Exam AI-100 topic 4 question 4 discussion

Actual exam question from Microsoft's AI-100

Question #: 4
Topic #: 4

Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.
After you answer a question, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.
You are deploying an Azure Machine Learning model to an Azure Kubernetes Service (AKS) container.
You need to monitor the scoring accuracy of each run of the model.
Solution: You configure Azure Application Insights.
Does this meet the goal?

A. Yes
B. No

Show Suggested Answer

Suggested Answer: A 🗳️
References:
https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-enable-data-collection https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-enable-app-insights

by BwandoWando at May 29, 2021, 6:59 a.m.

Comments

Submit Cancel

rveney

2 years ago

B. No Configuring Azure Application Insights does not meet the goal of monitoring the scoring accuracy of each run of the model[1]. You can use Azure Machine Learning's model telemetry feature to monitor the scoring accuracy of each run of the model[1]. You can also use Azure Monitor to monitor the performance of your AKS cluster

upvoted 1 times

...

timosi

4 years ago

This is No. Application insight does not log model performance in terms of model prediction accuracy. That cannot be done without the real outcome. you can modify the scoring script to store the data in a blob to use the AML data drift monitor, but also this tool does not provide model performance accuracy. Right now, there is no tool or service that tracks model performance in the sense of model prediction vs model real outcome.

upvoted 2 times

...

BwandoWando

4 years, 1 month ago

This is a YES If you download from GITHUB the DP-100 sturdy materials and actual code example of MICROSOFT here https://microsoftlearning.github.io/mslearn-dp100/ , you can go to this notebook "16 - Monitor a Model.ipynb" you will need to modify the scoring file that every prediction that the model does, you will have to write into application insights. TLDR: 1. Modify scoring script 2. Use Application Insights within the scoring script to log the prediction

upvoted 4 times

...