Exam DP-100 topic 3 question 75 discussion

Actual exam question from Microsoft's DP-100

Question #: 75
Topic #: 3

You are creating a classification model for a banking company to identify possible instances of credit card fraud. You plan to create the model in Azure Machine
Learning by using automated machine learning.
The training dataset that you are using is highly unbalanced.
You need to evaluate the classification model.
Which primary metric should you use?

A. normalized_mean_absolute_error
B. AUC_weighted
C. accuracy
D. normalized_root_mean_squared_error
E. spearman_correlation

Show Suggested Answer

Suggested Answer: B 🗳️
AUC_weighted is a Classification metric.
Note: AUC is the Area under the Receiver Operating Characteristic Curve. Weighted is the arithmetic mean of the score for each class, weighted by the number of true instances in each class.
Incorrect Answers:
A: normalized_mean_absolute_error is a regression metric, not a classification metric.
C: When comparing approaches to imbalanced classification problems, consider using metrics beyond accuracy such as recall, precision, and AUROC. It may be that switching the metric you optimize for during parameter selection or model selection is enough to provide desirable performance detecting the minority class.
D: normalized_root_mean_squared_error is a regression metric, not a classification metric.
Reference:
https://docs.microsoft.com/en-us/azure/machine-learning/how-to-understand-automated-ml

by ac45863 at April 7, 2021, 11:25 p.m.

Comments

Submit Cancel

ac45863

Highly Voted 2 years, 2 months ago

It's correct. "...the AUC_weighted is a primary metric that calculates the contribution of every class based on the relative number of samples representing that class, hence is more robust against imbalance."

upvoted 16 times

...