Exam DP-100 topic 5 question 14 discussion

99% class 1 data and 1% class 2 data. If all prediction is class 1, we will attain 99% accuracy. So accuracy is incorrect answer

For evaluating a binary classification model, especially with imbalanced datasets, the Area Under the Receiver Operating Characteristic (AUC-ROC) Curve is an excellent metric. It's insensitive to class imbalance and provides a good summary of the model's performance across different classification thresholds.

AUC Curve (Area Under the Curve): The AUC-ROC (Receiver Operating Characteristic) curve is a performance measurement for classification problems at various threshold settings. AUC represents the degree or measure of separability, indicating how much the model is capable of distinguishing between classes. An AUC value of 0.5 suggests no discrimination (i.e., random guessing), whereas a value of 1.0 indicates perfect discrimination. The AUC-ROC curve is particularly useful for evaluating models on imbalanced datasets because it is insensitive to changes in the class distribution. It provides a single metric that captures the trade-off between sensitivity (true positive rate) and specificity (true negative rate).

The appropriate evaluation metric to use for assessing imbalance in a binary classification model is the AUC Curve (B). AUC (Area Under the Curve) is a measure of the model's ability to distinguish between positive and negative classes. AUC ranges from 0 to 1, where an AUC of 1 indicates perfect separation between the positive and negative classes, and an AUC of 0.5 indicates random chance. A high AUC value indicates that the model has a strong ability to correctly classify positive and negative instances, which is especially important in imbalanced datasets where one class may have significantly fewer instances than the other. Therefore, the AUC curve is a commonly used metric to evaluate the performance of binary classification models in the presence of class imbalance.

I guess weighted AUC is the best answer ...

What does it mean by "evaluate the model results for imbalance"? Does it mean evaluate the extent/degree of imbalance in the dataset? Or does it simply mean to evaluate the model when the underyling data is imbalanced?

AUC is the correct answer.

AUC seems to be the right answer as per this.. https://stats.stackexchange.com/questions/260164/auc-and-class-imbalance-in-training-test-dataset

I would assume AUC Curve is a typo, because AUC is Area Under Curve. Seems it would be the right answer though.

It should be E (Accuracy) The AUC is the area under ROC curve so it's a number not a curve. So there is no curve called AUC curve.

Should Accuracy not be included along with AUC?