Exam Professional Machine Learning Engineer topic 1 question 325 discussion

Actual exam question from Google's Professional Machine Learning Engineer

Question #: 325
Topic #: 1

[All Professional Machine Learning Engineer Questions]

You are building an ML model to predict customer churn for a subscription service. You have trained your model on Vertex AI using historical data, and deployed it to a Vertex AI endpoint for real-time predictions. After a few weeks, you notice that the model's performance, measured by AUC (area under the ROC curve), has dropped significantly in production compared to its performance during training. How should you troubleshoot this problem?

A. Monitor the training/serving skew of feature values for requests sent to the endpoint.
B. Monitor the resource utilization of the endpoint, such as CPU and memory usage, to identify potential bottlenecks in performance.
C. Enable Vertex Explainable AI feature attribution to analyze model predictions and understand the impact of each feature on the model's predictions.
D. Monitor the latency of the endpoint to determine whether predictions are being served within the expected time frame.

Show Suggested Answer

Suggested Answer: A 🗳️

by Duke_CT at June 20, 2025, 3:52 a.m.

Comments

Submit Cancel

qaz09

1 month ago

Selected Answer: A

Since the performance dropped after a few weeks, the solution is to monitor training/serving skew

upvoted 1 times

...

kirukkuman

1 month, 2 weeks ago

Selected Answer: A

A significant drop in AUC means the patterns the model learned are no longer effective on the new, live data. Detecting a skew or drift between the statistical properties of the training data and the serving data is the most direct way to confirm and diagnose this issue. Vertex AI Model Monitoring is designed specifically for this purpose.

upvoted 1 times

...

Duke_CT

1 month, 3 weeks ago

Selected Answer: C

I believe it's A.

upvoted 1 times

...