Exam AWS Certified Machine Learning - Specialty All Questions

View all questions & answers for the AWS Certified Machine Learning - Specialty exam

Exam AWS Certified Machine Learning - Specialty topic 1 question 173 discussion

Exam question from Amazon's AWS Certified Machine Learning - Specialty

Question #: 173
Topic #: 1

[All AWS Certified Machine Learning - Specialty Questions]

A retail company uses a machine learning (ML) model for daily sales forecasting. The company's brand manager reports that the model has provided inaccurate results for the past 3 weeks.
At the end of each day, an AWS Glue job consolidates the input data that is used for the forecasting with the actual daily sales data and the predictions of the model. The AWS Glue job stores the data in Amazon S3. The company's ML team is using an Amazon SageMaker Studio notebook to gain an understanding about the source of the model's inaccuracies.
What should the ML team do on the SageMaker Studio notebook to visualize the model's degradation MOST accurately?

A. Create a histogram of the daily sales over the last 3 weeks. In addition, create a histogram of the daily sales from before that period.
B. Create a histogram of the model errors over the last 3 weeks. In addition, create a histogram of the model errors from before that period.
C. Create a line chart with the weekly mean absolute error (MAE) of the model.
D. Create a scatter plot of daily sales versus model error for the last 3 weeks. In addition, create a scatter plot of daily sales versus model error from before that period.

Show Suggested Answer

Suggested Answer: C 🗳️

by exam_prep at May 25, 2022, 11:43 p.m.

Disclaimers:

- ExamTopics website is not related to, affiliated with, endorsed or authorized by Amazon.
- Trademarks, certification & product names are used for reference only and belong to Amazon.

Comments

Submit Cancel

exam_prep

Highly Voted 3 years, 1 month ago

C is the correct answer.

upvoted 13 times

ef12052

2 months, 3 weeks ago

why? i think it's D

upvoted 1 times

...

ef12052

Most Recent 3 months ago

Selected Answer: C

https://docs.aws.amazon.com/forecast/latest/dg/predictor-monitoring-results.html

upvoted 1 times

ef12052

2 months, 3 weeks ago

change my mind, "weekly" -> not c, so i think it's D

upvoted 1 times

...

Carpediem78

3 months ago

Selected Answer: D

C. Creating a line chart of weekly Mean Absolute Error (MAE) → This can show changes in model performance over time, but it does not reveal whether errors are concentrated in specific sales ranges. D. Creating a scatter plot of daily sales vs. model errors for the past three weeks and the period before that → This allows for a direct visual analysis of the relationship between actual sales and prediction errors. → It helps identify whether errors are more significant in specific sales ranges and effectively compares model performance before and after degradation.

upvoted 1 times

...

learningcloud1

12 months ago

Selected Answer: B

Most accurately. C is a tempting answer. You will see the model degradation over time. You could see if it's slowly getting worse or was it sudden. B is more accurate. You will only have two histograms to compare, but you will easily see which direction the error move: Are we over or underestimating. In practice you would use both. I'm terms of the exam - most accurate - most added information, B gives more information than C. It's a preference though.

upvoted 1 times

learningcloud1

12 months ago

Although, following the docs it will be C: https://docs.aws.amazon.com/forecast/latest/dg/predictor-monitoring-results.html

upvoted 2 times

...

f3a4b7c

1 year, 1 month ago

Selected Answer: D

D is the correct answer

upvoted 1 times

...

3eb0542

1 year, 2 months ago

Selected Answer: D

Weekly MAE aggregates the error metrics over a larger time window, which can mask fluctuations and specific patterns in the model's performance on a daily basis. In situations where there are sudden changes or degradation in the model's accuracy within a week, this visualization might not capture those nuances effectively.

upvoted 2 times

...

AIWave

1 year, 4 months ago

Selected Answer: D

Degradation over time: line or scatter plot (options C, D) C is aggregrate weekly view and doesn't give any additional details. D compares the model's errors during 3 week period to the errors from before that period giving an accurate picture of anomalies

upvoted 3 times

...

3eb0542

1 year, 4 months ago

Selected Answer: D

GPT: To accurately visualize the degradation of the model over time and understand the source of inaccuracies, the ML team should focus on comparing the model's performance before and after the reported period of inaccuracies. The most appropriate option is: D. Create a scatter plot of daily sales versus model error for the last 3 weeks. In addition, create a scatter plot of daily sales versus model error from before that period.

upvoted 1 times

F1Fan

1 year, 3 months ago

Claude 3 Sonnet: Based on the evidence from AWS documentation and best practices, Option B: Create a histogram of the model errors over the last 3 weeks. In addition, create a histogram of the model errors from before that period, is the most accurate approach for the ML team to visualize the model's degradation. Histograms of model errors directly visualize the distribution and patterns of the model's inaccuracies, which is crucial for understanding the source of the problem. By comparing the error distributions before and after the 3-week period, the ML team can identify any significant shifts or changes that may indicate the cause of the model's degradation. This approach aligns with AWS best practices for model monitoring and visualization, as recommended by the Amazon SageMaker Model Monitor documentation. It provides a clear and focused visualization of the model's performance, enabling the ML team to gain insights and take appropriate actions to address the inaccuracies.

upvoted 1 times

...

kyuhuck

1 year, 4 months ago

Selected Answer: B

The best option to visualize the model's degradation most accurately would be to compare the model's errors over the relevant periods. This directly addresses the issue of model accuracy and allows for a clear comparison of model performance before and after the reported period of inaccuracy. Therefore, the most appropriate approach would be: B. Create a histogram of the model errors over the last 3 weeks. In addition, create a histogram of the model errors from before that period. This approach will allow the ML team to see if the distribution of errors has changed recently, indicating a degradation in model performance.

upvoted 2 times

...

DimLam

1 year, 8 months ago

Is there a reason to create weekly MAE plot, if the prediction is made on daily granularity?

upvoted 3 times

...

teka112233

1 year, 9 months ago

Selected Answer: C

this is the key sentence : At the end of each day, an AWS Glue job consolidates the input data that is used for the forecasting with the actual daily sales data and the predictions of the model and that is exactly what MAE do: mean absolute error (MAE) is a statistical measure of the difference between two continuous variables. It is calculated as the average of the absolute differences between the predicted and actual values so the answer is C

upvoted 1 times

...

loict

1 year, 9 months ago

Selected Answer: C

A. NO - Daily sales histogram does not help to see model error B. NO - Histogram of the model errors is good, but no point to have one for the first 3 weeks and another for older data C. YES - one chart of model errors is perfect D. NO - no point to have 2 charts again

upvoted 1 times

...

kaike_reis

1 year, 10 months ago

Selected Answer: C

C is correct.

upvoted 1 times

...

Mickey321

1 year, 11 months ago

Selected Answer: B

option B with histograms of model errors for the specific time periods is the most accurate and appropriate visualization to understand the model's degradation and identify the reasons behind the inaccuracies in the daily sales forecasting.

upvoted 3 times

...

Valcilio

2 years, 3 months ago

Selected Answer: C

C is the answer, line plots are good solutions for time series analysis.

upvoted 3 times

...

Jerry84

2 years, 5 months ago

Selected Answer: C

C is correct. We could view the "Degradation" as a trend. Line charts are usually very helpful to show if there is any trend in the data over the period of time under analysis. Histogram is normally used to visualizing distributions in your data.

upvoted 4 times

...

DD4

2 years, 10 months ago

Should be A because it is daily forecasting and histograms before and after will show the comparable degradation.

upvoted 1 times

venimus_vidimus_vicimus

2 years, 6 months ago

It only states to plot daily sales.. how should that help with the error? You need a plot of the actual and predicted values or or the errors - def not A

upvoted 1 times

...

Load full discussion...