Exam Professional Machine Learning Engineer topic 1 question 130 discussion

C. According Google: "Instead of deploying the model to an endpoint, you can use the RunInference API to serve machine learning models in your Apache Beam pipeline. This approach has several advantages, including flexibility and portability. However, deploying the model in Vertex AI offers many additional benefits, such as the platform's built-in tools for model monitoring, TensorBoard, and model registry governance. Vertex AI also provides the ability to use Optimized TensorFlow runtime in your endpoints. To do this, simply specify the TensorFlow runtime container when you deploy your model." https://cloud.google.com/blog/products/ai-machine-learning/streaming-prediction-with-dataflow-and-vertex

B. Load the model directly into the Dataflow job as a dependency, and use it for prediction. By loading the model directly into the Dataflow job as a dependency, you minimize the serving latency since the model is available within the pipeline itself. This way, you avoid additional network latency that would be introduced by invoking external services, such as Cloud Run, Vertex AI endpoints, or TFServing containers.

By loading the TensorFlow model directly into the Dataflow job, you ensure that inference happens inline within the pipeline on the worker nodes. Using external endpoints (Options A, C, and D) introduces extra latency due to network round trips, which is not ideal for real-time prediction in a cybersecurity context

the question asks *only* about minimizing latency. Doing everything in Dataflow minimizes latency over all the other options.

It's a toss up between B and C. I chose B because using vertex AI as an endpoint introduces network latency which naturally does not meet the criteria of "minimizing latency". However, choosing option B also implies that I have more overhead by directly running the model in the dataflow pipeline. Since the question didn't mention any limitations on resources, I assumed that the resources can be scaled accordingly to minimize latency. I might be overthinking on this option though seeing how most of Google questions have a strong preference on their "recommended platforms" like vertex AI. Most of the questions and the community answers seem to tend towards anything that mentions "vertex ai".

According Google: "Instead of deploying the model to an endpoint, you can use the RunInference API to serve machine learning models in your Apache Beam pipeline. This approach has several advantages, including flexibility and portability. However, deploying the model in Vertex AI offers many additional benefits, such as the platform's built-in tools for model monitoring, TensorBoard, and model registry governance. Vertex AI also provides the ability to use Optimized TensorFlow runtime in your endpoints. To do this, simply specify the TensorFlow runtime container when you deploy your model." https://cloud.google.com/blog/products/ai-machine-learning/streaming-prediction-with-dataflow-and-vertex

In this case, the best way to minimize the serving latency of the system log anomaly detection model is to deploy it to a Vertex AI endpoint. This will allow Dataflow to invoke the model directly, without having to load it into the job as a dependency. This will significantly reduce the serving latency, as Dataflow will not have to wait for the model to load before it can make a prediction. Option B would involve loading the model directly into the Dataflow job as a dependency. This would also add an additional layer of latency, as Dataflow would have to load the model into memory before it could make a prediction.

C. Deploy the model to a Vertex AI endpoint, and invoke this endpoint in the Dataflow job https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning

C I eliminate B because Dataflow is a batch-prediction solution, not real-time

Went with C

I believe it is C when deploying the model to a Vertex AI endpoint it provides a dedicated prediction service optimised for real-time inference. Vertex AI endpoints are designed for high performance and low latency, making them ideal for real-time prediction use cases. Dataflow can easily invoke the Vertex AI endpoint to perform predictions, minimising serving latency.

By loading the model directly into the Dataflow job as a dependency, you can perform predictions within the same job. This approach helps minimize serving latency since there is no need to make external calls to another service or endpoint. Instead, the model is directly available within the Dataflow pipeline, allowing for efficient and fast processing of the streaming data.

C. Deploy the model to a Vertex AI endpoint, and invoke this endpoint in the Dataflow job. The reason for this choice is that deploying the model to a Vertex AI endpoint and invoking it in the Dataflow job is the most efficient and scalable option for real-time prediction. Vertex AI provides a fully managed, serverless platform for deploying and serving machine learning models. It allows for high availability and low-latency serving of models, and can handle a large volume of requests in parallel. Invoking the model via an endpoint in the Dataflow job minimizes the latency for model prediction, as it avoids any unnecessary data transfers or containerization

Option B, loading the model directly into the Dataflow job as a dependency and using it for prediction, may not provide the optimal performance because Dataflow may not be optimized for low-latency predictions.

These are anwser https://cloud.google.com/dataflow/docs/notebooks/run_inference_tensorflow https://beam.apache.org/documentation/sdks/python-machine-learning/ https://beam.apache.org/documentation/transforms/python/elementwise/runinference/

C OR B. it is straightforward that it should be C as the follwing link https://cloud.google.com/architecture/detecting-anomalies-in-financial-transactions but B it seems a newer way. it keeps questionable.

C, for me.