Exam AWS Certified Machine Learning Engineer - Associate MLA-C01 topic 1 question 15 discussion

Exam question from Amazon's AWS Certified Machine Learning Engineer - Associate MLA-C01

Question #: 15
Topic #: 1

[All AWS Certified Machine Learning Engineer - Associate MLA-C01 Questions]

A company has deployed an XGBoost prediction model in production to predict if a customer is likely to cancel a subscription. The company uses Amazon SageMaker Model Monitor to detect deviations in the F1 score.
During a baseline analysis of model quality, the company recorded a threshold for the F1 score. After several months of no change, the model's F1 score decreases significantly.
What could be the reason for the reduced F1 score?

A. Concept drift occurred in the underlying customer data that was used for predictions.
B. The model was not sufficiently complex to capture all the patterns in the original baseline data.
C. The original baseline data had a data quality issue of missing values.
D. Incorrect ground truth labels were provided to Model Monitor during the calculation of the baseline.

Show Suggested Answer

Suggested Answer: A 🗳️

by GiorgioGss at Nov. 27, 2024, 3:53 p.m.

Disclaimers:

- ExamTopics website is not related to, affiliated with, endorsed or authorized by Amazon.
- Trademarks, certification & product names are used for reference only and belong to Amazon.

Comments

Submit Cancel

ninomfr64

5 months, 4 weeks ago

Selected Answer: A

A. Yes, concept drift is an evolution of data that invalidates the data model. It happens when the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. This causes problems because the predictions become less accurate as time passes. B. No, if it was the case the F1 would have been low since the begin and this is not justifying a change after months C. No, same as B D. No, incorrect labels in the baseline calculations would undermine F1 baseline value, but this is not explain a significant drop after months

upvoted 1 times

...

motk123

6 months, 2 weeks ago

Selected Answer: A

Concept Drift: Occurs when the statistical properties of the data used for predictions change over time, causing the model to underperform on current data. Why Not the Other Options? B. If the model complexity was insufficient, the issue would have been detected during the initial evaluation or baseline analysis, not after months of stable performance. C. A data quality issue would have impacted the model's performance immediately after deployment, not months later. D. Incorrect labels during baseline calculation could result in an inaccurate baseline F1 score, but it wouldn't explain a significant drop after stable performance over months.

upvoted 3 times

...

Saransundar

6 months, 3 weeks ago

Selected Answer: A

Concept Drift: Refers to the change in the statistical properties of the underlying data distribution over time --> Decrease F1 score --> perform poorly on new data

upvoted 3 times

...

GiorgioGss

7 months ago

Selected Answer: A

Option A could be the only one possible reason for drifting "after several months".

upvoted 2 times

...