Exam AWS Certified Machine Learning - Specialty All Questions

View all questions & answers for the AWS Certified Machine Learning - Specialty exam

Exam AWS Certified Machine Learning - Specialty topic 1 question 183 discussion

Exam question from Amazon's AWS Certified Machine Learning - Specialty

Question #: 183
Topic #: 1

[All AWS Certified Machine Learning - Specialty Questions]

A data scientist is evaluating a GluonTS on Amazon SageMaker DeepAR model. The evaluation metrics on the test set indicate that the coverage score is 0.489 and 0.889 at the 0.5 and 0.9 quantiles, respectively.
What can the data scientist reasonably conclude about the distributional forecast related to the test set?

A. The coverage scores indicate that the distributional forecast is poorly calibrated. These scores should be approximately equal to each other at all quantiles.
B. The coverage scores indicate that the distributional forecast is poorly calibrated. These scores should peak at the median and be lower at the tails.
C. The coverage scores indicate that the distributional forecast is correctly calibrated. These scores should always fall below the quantile itself.
D. The coverage scores indicate that the distributional forecast is correctly calibrated. These scores should be approximately equal to the quantile itself.

Show Suggested Answer

Suggested Answer: D 🗳️

by cognito_22 at May 7, 2022, 8:16 p.m.

Disclaimers:

- ExamTopics website is not related to, affiliated with, endorsed or authorized by Amazon.
- Trademarks, certification & product names are used for reference only and belong to Amazon.

Comments

Submit Cancel

cognito_22

Highly Voted 2 years, 3 months ago

Selected Answer: D

https://ts.gluon.ai/tutorials/forecasting/quick_start_tutorial.html

upvoted 10 times

example_

1 year, 11 months ago

https://ts.gluon.ai/stable/tutorials/forecasting/quick_start_tutorial.html

upvoted 4 times

...

backbencher2022

Most Recent 10 months, 1 week ago

Selected Answer: C

C is correct based on this blog - https://aws.amazon.com/blogs/machine-learning/training-debugging-and-running-time-series-forecasting-models-with-the-gluonts-toolkit-on-amazon-sagemaker/

upvoted 1 times

...

Mickey321

12 months ago

Selected Answer: D

D for me

upvoted 1 times

...

vbal

1 year ago

D: A well-calibrated model should have quantile coverage close to the desired coverage level (e.g., 90% quantile coverage should be close to 90%). If the quantile coverage is consistently off from the desired level, it may indicate the need to recalibrate the model or investigate the sources of uncertainty estimation errors.

upvoted 1 times

...

CKS1210

1 year, 1 month ago

Selected Answer: C

https://apps.microsoft.com/store/detail/move-mouse/9NQ4QL59XLBF?hl=en-us&gl=us

upvoted 1 times

...

WilianCB

1 year, 3 months ago

Selected Answer: D

I think it is D

upvoted 1 times

...

blanco750

1 year, 5 months ago

Selected Answer: C

Thanks to ChatGPT Given the coverage score results, the data scientist can conclude that the distributional forecast related the test set is well calibrated. Specifically, when the model predicts quantiles, around % of the true values should fall within the 0.5 quantile range, and around90% of the true values should fall within the 0.9 quantile range., the GluonTS on Amazon SageMaker DeepAR model performance on the test set was concerning the coverage of the predicted quantiles.

upvoted 1 times

wendaz

10 months ago

My chatgpt is latest: The coverage of a distributional forecast at a given quantile is the fraction of observations that fall below the predicted quantile. In a well-calibrated forecast, the coverage score should be approximately equal to the quantile itself. Given the information: Coverage score is 0.489 at the 0.5 quantile. Coverage score is 0.889 at the 0.9 quantile. For a well-calibrated forecast: At the 0.5 quantile (or median), the coverage should be approximately 0.5. At the 0.9 quantile, the coverage should be approximately 0.9. The provided coverage scores closely match the quantiles, with slight deviations. Therefore, the correct conclusion is: Option D: The coverage scores indicate that the distributional forecast is correctly calibrated. These scores should be approximately equal to the quantile itself.

upvoted 1 times

...

SANDEEP_AWS

1 year, 5 months ago

Selected Answer: C

Scores should always fall below the quantile itself. Ref: https://d1.awsstatic.com/asset-repository/Amazon%20Forecast%20Technical%20Guide%20to%20Time-Series%20Forecasting%20Principles.pdf -- Pg 18

upvoted 2 times

SANDEEP_AWS

1 year, 5 months ago

PDF Pg 23

upvoted 2 times

...

Tony_1406

1 year, 3 months ago

https://docs.aws.amazon.com/forecast/latest/dg/metrics.html#metrics-wQL A more concise doc.

upvoted 1 times

...