Exam Certified Generative AI Engineer Associate topic 1 question 80 discussion

Actual exam question from Databricks's Certified Generative AI Engineer Associate

Question #: 80
Topic #: 1

[All Certified Generative AI Engineer Associate Questions]

A Generative AI Engineer has created a RAG application which can help employees interpret HR documentation. The prototype application is now working with some positive feedback from internal company testers. Now the Generative AI Engineer wants to formally evaluate the system’s performance and understand where to focus their efforts to further improve the system

How should the Generative AI Engineer evaluate the system?

A. Use ROUGE score to comprehensively evaluate the quality of the final generated answers.
B. Use an LLM-as-a-judge to evaluate the quality of the final answers generated.
C. Curate a dataset that can test the retrieval and generation components of the system separately. Use MLflow’s built in evaluation metrics to perform the evaluation on the retrieval and generation components.
D. Benchmark multiple LLMs with the same data and pick the best LLM for the job.

Show Suggested Answer

Suggested Answer: C 🗳️

by DavidMiller at May 3, 2025, 4:22 p.m.

Comments

Submit Cancel

seaun

1 week, 6 days ago

Selected Answer: C

Option C

upvoted 1 times

...

DavidMiller

2 months ago

Selected Answer: C

Option C is correct because it takes a modular, scientific approach to debugging and optimizing a RAG system—by isolating variables and evaluating each part methodically.

upvoted 1 times

...