Exam Professional Machine Learning Engineer topic 1 question 29 discussion

Supporting B ..https://cloud.google.com/architecture/data-preprocessing-for-ml-with-tf-transform-pt1#where_to_do_preprocessing

I think it should b B

I think the "high-throughput" is the critical differentiator between choosing B or D. Dataflow is designed for large-scale stream or batch processing, not for low-latency, single-request online prediction. The startup latency of a Dataflow job makes it unsuitable for this pattern.

Dataflow has autoscale. And in my experience, you use Cloud Functions to small stuff.

You are an ML engineer who has trained a model on a dataset that required computationally expensive preprocessing operations. You need to execute the same preprocessing at prediction time. You deployed the model on the Vertex AI platform for high‐throughput online prediction. Which architecture should you use? Answer is . Send incoming prediction requests to a Pub/Sub topic. Set up a Cloud Function that is triggered when messages are published to the Pub/Sub topic. Implement your preprocessing logic in the Cloud Function. Submit a prediction request to the Vertex AI platform using the transformed data. Write the predictions to an outbound Pub/Sub queue.

B is incorrect. Dataflow is a great option for large-scale data processing but may introduce additional complexity and overhead for a real-time prediction scenario where you just need to preprocess data on-the-fly. This is more appropriate for batch processing or when large volumes of data need to be processed in parallel. Option D is better as it leverages Pub/Sub, Cloud Functions, and AI Platform to preprocess data and obtain predictions without needing complex infrastructure or additional systems like Dataflow or Cloud Spanner.

Dataflow is superior to Cloud Functions for doing data transformations at high volume. The answer is clearly B.

D. The issue with B is that DataFlow does not work well with high throughput

B) Pub/Sub + Dataflow

Went with B, using dataflow for large amount data transformation is the best option

I went to B. A is completely wrong. C: 1st cloud spanner is not designed for high throughput, also it is not for preprocessing. D: cloud function could not be get enough resource to do the high computational transformation.

Because the concern here is high throughput and not specifically the latency so better to go with option B

B. Send incoming prediction requests to a Pub/Sub topic. Transform the incoming data using a Dataflow job. Submit a prediction request to AI Platform using the transformed data. Write the predictions to an outbound Pub/Sub queue https://dataintegration.info/building-streaming-data-pipelines-on-google-cloud

Went with B

I think it's D as B is not a good choice because it requires you to run a Dataflow job for each prediction request. This is inefficient and can lead to latency issues.

I go for D. Option B has Dataflow that it is more suitable for batch

It's B