Exam DP-100 topic 5 question 23 discussion

Actual exam question from Microsoft's DP-100

Question #: 23
Topic #: 5

DRAG DROP -
You have several machine learning models registered in an Azure Machine Learning workspace.
You must use the Fairlearn dashboard to assess fairness in a selected model.
Which three actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.
Select and Place:

Show Suggested Answer

Suggested Answer:

Step 1: Select a model feature to be evaluated.
Step 2: Select a binary classification or regression model.
Register your models within Azure Machine Learning. For convenience, store the results in a dictionary, which maps the id of the registered model (a string in name:version format) to the predictor itself.
Example:
model_dict = {}
lr_reg_id = register_model("fairness_logistic_regression", lr_predictor) model_dict[lr_reg_id] = lr_predictor svm_reg_id = register_model("fairness_svm", svm_predictor) model_dict[svm_reg_id] = svm_predictor
Step 3: Select a metric to be measured
Precompute fairness metrics.
Create a dashboard dictionary using Fairlearn's metrics package.
Reference:
https://docs.microsoft.com/en-us/azure/machine-learning/how-to-machine-learning-fairness-aml

by ranjsi01 at Jan. 25, 2022, 8:12 p.m.

Comments

Submit Cancel

ranjsi01

Highly Voted 3 years, 4 months ago

model, feature, metric (just switch a and b in answer area)

upvoted 11 times

giusecozza

2 years, 9 months ago

Yes, it seems the right order. Just take a look at the notebook from MS Learn: https://microsoftlearning.github.io/mslearn-dp100/instructions/15-detect-unfairness.html

upvoted 1 times

...

JTWang

2 years, 7 months ago

Answers do not need to be in order. Smple Code: sf = { 'Race': A_test.race, 'Sex': A_test.sex } from fairlearn.metrics._group_metric_set import _create_group_metric_set dash_dict = _create_group_metric_set(y_true=Y_test, predictions=ys_pred, sensitive_features=sf, prediction_type='binary_classification') https://learn.microsoft.com/zh-tw/azure/machine-learning/how-to-machine-learning-fairness-aml

upvoted 3 times

JTWang

2 years, 7 months ago

My fault, the answers need to be in order.

upvoted 2 times

...

phdykd

Highly Voted 2 years, 3 months ago

The three actions that you should perform in sequence to use the Fairlearn dashboard to assess fairness in a selected model are: Select a binary classification or regression model: Fairlearn is a toolkit for assessing and improving fairness in binary classification and regression models. Therefore, you need to select a model that falls into one of these two categories. Select a metric to be measured: After selecting the model, you need to choose a metric to be measured. Fairlearn provides a range of fairness metrics, such as demographic parity, equalized odds, and equal opportunity, that can be used to assess how the model performs across different groups. Select a model feature to be evaluated: Once you have selected the model and the metric, you need to choose a model feature to be evaluated. This could be any feature that you believe may have an impact on the fairness of the model, such as race, gender, or age. You can use Fairlearn to analyze the model's performance across different subgroups based on this feature.

upvoted 6 times

...

evangelist

Most Recent 11 months, 4 weeks ago

Fairlearn works with binary classification or regression models, so you need to select one of these first. Next, you need to choose what metric you want to use to measure fairness. Finally, you select which feature of the model you want to evaluate for fairness.

upvoted 3 times