Exam Professional Machine Learning Engineer topic 1 question 173 discussion

This approach acknowledges that the skew might be due to a mismatch between the training data and the current production data distribution. By waiting for sufficient new production traffic, you can collect a more representative dataset that reflects the current state of the production environment. Retraining the model on this new data ensures that the model is better aligned with the production data distribution, which should resolve the skew.

C) - The model is adapting to the changing data distribution in production - Disabling alerts temporarily gives model a chance to adjust to new data *A) would hide/mask the skew; B) doesn't make sense because the monitoring job already uses the most recently trained data, it's just different from production data; D) is reactive and short term solution

C. Temporarily disable the alert. Enable the alert again after a sufficient amount of new production traffic has passed through the Vertex AI endpoint. Here's why: You've already retrained the model with more recent training data and deployed it back to the Vertex AI endpoint, but the alert persists. This suggests that the model is still adapting to the changing data distribution in production. Temporarily disabling the alert will give the model a chance to adjust to the new data distribution before the monitoring job starts firing alerts again. Once enough new traffic has passed through, you can re-enable the alert and continue monitoring the model's performance.

The baseline is calculated when you create a Vertex AI Model Monitoring job, and is only recalculated if you update the training dataset for the job.

Is B actually the correct answer? According to the documentation, training-serving skew detection can only be enabled if the original training data is available. Furthermore, the baseline is automatically recalculated when the training data is updated. So does this question imply that the model is trained on data without updating the original training-dataset? If so then B is clearly correct. If they updated the training dataset with new data and then retrained the model then the model monitoring job's baseline should automatically have been recalculated. I see no other valid answers in that case?

This option can help align the baseline distribution of the model monitoring job with the current distribution of the production data, and eliminate the false positive alerts.

B This option can help align the baseline distribution of the model monitoring job with the current distribution of the production data, and eliminate the false positive alerts.

the cause of the issue could be that the developer forgot to switch their monitoring job to the latest training dataset and the monitoring job still compares prod data with old training dataset and they of course have a skew

B. Update the model monitoring job to use the more recent training data that was used to retrain the model: This option directly aligns the model monitoring with the recently retrained model and ensures that the monitoring job reflects the characteristics of the latest training data.

A. Changing the sampling rate affects not training skew but cost efficiency: https://cloud.google.com/vertex-ai/docs/model-monitoring/overview#considerations B. The model monitoring job is already using the most recent data to detect skew. C&D are the same, except for D being more specific, so I would tend towards D.