Exam Professional Machine Learning Engineer topic 1 question 225 discussion

Actual exam question from Google's Professional Machine Learning Engineer

Question #: 225
Topic #: 1

[All Professional Machine Learning Engineer Questions]

You work for a telecommunications company. You’re building a model to predict which customers may fail to pay their next phone bill. The purpose of this model is to proactively offer at-risk customers assistance such as service discounts and bill deadline extensions. The data is stored in BigQuery and the predictive features that are available for model training include:

- Customer_id
- Age
- Salary (measured in local currency)
- Sex
- Average bill value (measured in local currency)
- Number of phone calls in the last month (integer)
- Average duration of phone calls (measured in minutes)

You need to investigate and mitigate potential bias against disadvantaged groups, while preserving model accuracy.

What should you do?

A. Determine whether there is a meaningful correlation between the sensitive features and the other features. Train a BigQuery ML boosted trees classification model and exclude the sensitive features and any meaningfully correlated features.
B. Train a BigQuery ML boosted trees classification model with all features. Use the ML.GLOBAL_EXPLAIN method to calculate the global attribution values for each feature of the model. If the feature importance value for any of the sensitive features exceeds a threshold, discard the model and tram without this feature.
C. Train a BigQuery ML boosted trees classification model with all features. Use the ML.EXPLAIN_PREDICT method to calculate the attribution values for each feature for each customer in a test set. If for any individual customer, the importance value for any feature exceeds a predefined threshold, discard the model and train the model again without this feature.
D. Define a fairness metric that is represented by accuracy across the sensitive features. Train a BigQuery ML boosted trees classification model with all features. Use the trained model to make predictions on a test set. Join the data back with the sensitive features, and calculate a fairness metric to investigate whether it meets your requirements.

Show Suggested Answer

Suggested Answer: D 🗳️

by pikachu007 at Jan. 13, 2024, 7:27 a.m.

Comments

Submit Cancel

fitri001

9 months, 4 weeks ago

Selected Answer: D

Fairness Metric: Defining a metric like parity (equal accuracy) or calibration (similar predicted probabilities) across sensitive features like age, sex, or salary allows you to quantify potential bias. Model Training with All Features (Initially): Training the model with all features provides a baseline performance and allows you to identify potentially biased features later. Test Set Predictions: Making predictions on a held-out test set ensures the evaluation is based on unseen data and avoids overfitting. Joining Back Sensitive Features: Reintroducing sensitive features after prediction allows you to calculate fairness metrics for different customer groups. Iterative Refinement: Based on the fairness metric results, you can determine if further mitigation strategies are needed.

upvoted 3 times

fitri001

9 months, 4 weeks ago

A. Excluding Features Based on Correlation: While correlated features might indicate bias, simply excluding them can discard valuable information and potentially reduce model accuracy. B. Global Attribution for Feature Removal: Using global feature importance might not reveal bias impacting specific customer groups. Additionally, discarding a feature solely based on importance could affect model performance. C. Individual Attribution for Model Discarding: While individual attribution can identify per-customer bias, discarding the model entirely based on a single instance might be overly cautious and lead to starting from scratch frequently.

upvoted 1 times

...

pinimichele01

10 months, 1 week ago

Selected Answer: D

https://cloud.google.com/vertex-ai/docs/evaluation/intro-evaluation-fairness

upvoted 1 times

...

prtikare

11 months ago

Answer is A

upvoted 1 times

...

shadz10

1 year ago

Selected Answer: D

https://cloud.google.com/vertex-ai/docs/evaluation/intro-evaluation-fairness

upvoted 1 times

...

pikachu007

1 year, 1 month ago

Selected Answer: D

Direct Bias Assessment: It directly measures model fairness using a relevant metric, providing clear insights into potential issues. Preserving Information: It avoids prematurely removing features, potentially capturing valuable predictive signals while mitigating bias. Aligning with Goals: It allows tailoring the fairness metric to specific ethical and business objectives.

upvoted 2 times

...