Exam Professional Machine Learning Engineer topic 1 question 114 discussion

The game is more enjoyable - the better and "business metrics" points me to user engagement as best metric

I do not agree with C as User engagement would not help directly to evaluate whether players are enjoying more balanced matches due to the model’s performance. As the goal of the model is to assign players to teams based on their skill level so that teams are balanced and enjoyable for all participants I will go with B as the more appropriate answer.

This question doesn't specify how "additional revenue" is measured. Most businesses I've worked for would love "D" for all our models instead of anything else. That being said, C is the only measurable business metric there.

focusing on user engagement through the number of battles played daily provides a clearer indication of whether the model successfully creates balanced and enjoyable matches, which is the core objective. If players find battles more engaging due to fairer competition, they're more likely to keep playing. This can then translate to long-term benefits like increased retention and potential monetization opportunities.

Tempted by B but "user engagement" is the keyword.

Looking for "business metrics to track," I think C could be the most important metric. Although, option B is also a good choice.

C: Business metric i.e. outcome driven

"Business metrics" does suggest that the question is looking for metrics that are relevant to the business goals of the company, rather than purely technical metrics. In that case, C.could be a good choice. User engagement is an important metric for any online service, as it reflects how much users are enjoying and using the product. In the context of a multiplayer game, the number of battles played daily per user can indicate how well the model is doing in creating balanced teams that are enjoyable to play against. If the model is successful in creating balanced teams, then users are likely to play more games, which would increase user engagement. Therefore, C could be a suitable choice to track the performance of the model.

The focus is to obtain a model that assigns players to teams with players with similar level of skill (or average team 1 skill == average team 2 skill) A: A fast queue assignment may not focus on pearing players with the same levels of skills. A random assignment would work. B: This would be an option but is more difficult to measure than C, we don’t know If we have a measure of skill level. Also, for new players this metric would not be available at the beginning. I think “There are many new players every day.” is a key point important to discard answer B. C: Players play more games daily ← players enjoy the game more frequently and the other way round should also apply. Easy to measure also for new players. D:This focus on costs and revenue not on players matchmaking. I would go with C.

C because "user engagement" is a business metric https://support.google.com/analytics/answer/11109416?hl=en

Went with C

This is B, as it directly relates to our model's ability to predict player ability. There are many factors beyond our model which will impact user engagement (e.g. whether the game is actually enjoyable) so it's not a good measurement of the model performance

Answer C

The question is asking about "available players". Therefore, the business metric is the user engagement.

Asks for >business metric<, and problem states "user research indicates that the game is more enjoyable when battles have players with similar skill levels.", which means more battles per user if your model is performing well.

It's C. The question specifically asks for a business metric. Precision and recall are not business metrics, but user engagement is

The template uses the 'ability' to create teams. For this, we can conclude that the system measures the player's skill. So, nothing better than comparing the predict ability with the actual ability to understand the performance of the model.