Exam Professional Machine Learning Engineer topic 1 question 299 discussion

Actual exam question from Google's Professional Machine Learning Engineer

Question #: 299
Topic #: 1

[All Professional Machine Learning Engineer Questions]

Your team is experimenting with developing smaller, distilled LLMs for a specific domain. You have performed batch inference on a dataset by using several variations of your distilled LLMs and stored the batch inference outputs in Cloud Storage. You need to create an evaluation workflow that integrates with your existing Vertex AI pipeline to assess the performance of the LLM versions while also tracking artifacts. What should you do?

A. Develop a custom Python component that reads the batch inference outputs from Cloud Storage, calculates evaluation metrics, and writes the results to a BigQuery table.
B. Use a Dataflow component that processes the batch inference outputs from Cloud Storage, calculates evaluation metrics in a distributed manner, and writes the results to a BigQuery table.
C. Create a custom Vertex AI Pipelines component that reads the batch inference outputs from Cloud Storage, calculates evaluation metrics, and writes the results to a BigQuery table.
D. Use the Automatic side-by-side (AutoSxS) pipeline component that processes the batch inference outputs from Cloud Storage, aggregates evaluation metrics, and writes the results to a BigQuery table.

Show Suggested Answer

Suggested Answer: C 🗳️

by 5091a99 at March 4, 2025, 11:14 p.m.

Comments

Submit Cancel

4d742d7

3 weeks, 1 day ago

Selected Answer: D

Use the AutoSxS pipeline component to quickly evaluate and compare your distilled LLMs—all integrated with Vertex AI Pipelines, with minimal development overhead and full artifact lineage support. Let me know if you'd like help configuring the component or reviewing its outputs!

upvoted 1 times

...

Begum

1 month, 2 weeks ago

Selected Answer: C

Vertrx AI component can be used. No need to complicate solution.

upvoted 1 times

...

tmpuserx

1 month, 3 weeks ago

Selected Answer: D

AutoSxS said is meant for models comparism

upvoted 1 times

...

5091a99

3 months, 4 weeks ago

Selected Answer: C

Answer C: Vertex AI Pipeline. - The Flow already includes Pipelines, which allow for more flexibility in model training, evaluation and metadata storage. No need to go outside of the environment.

upvoted 1 times

...