Exam Certified Data Engineer Professional topic 1 question 114 discussion

Actual exam question from Databricks's Certified Data Engineer Professional

Question #: 114
Topic #: 1

[All Certified Data Engineer Professional Questions]

A data team’s Structured Streaming job is configured to calculate running aggregates for item sales to update a downstream marketing dashboard. The marketing team has introduced a new promotion, and they would like to add a new field to track the number of times this promotion code is used for each item. A junior data engineer suggests updating the existing query as follows. Note that proposed changes are in bold.

Original query:

Proposed query:

Which step must also be completed to put the proposed query into production?

A. Specify a new checkpointLocation
B. Remove .option('mergeSchema', 'true') from the streaming write
C. Increase the shuffle partitions to account for additional aggregates
D. Run REFRESH TABLE delta.‛/item_agg‛

Show Suggested Answer

Suggested Answer: A 🗳️

by MDWPartners at May 29, 2024, 5:55 p.m.

Comments

Submit Cancel

Deb9753

Highly Voted 1 year ago

Answer: A When updating the schema of a streaming job, specifying a new checkpoint location ensures that the streaming query starts fresh with the new schema. This avoids issues that might arise from schema mismatches between the previous state and the new schema. This is especially relevant when adding new fields because the existing state might not be compatible with the new schema.

upvoted 5 times

...

KadELbied

Most Recent 1 month, 1 week ago

Selected Answer: A

Suretly A

upvoted 1 times

...

SeRo42

5 months ago

Selected Answer: A

Are filter conditions in count() allowed?

upvoted 1 times

...

robodog

9 months, 4 weeks ago

Selected Answer: A

A answer

upvoted 1 times

...

MDWPartners

1 year ago

Selected Answer: A

This checkpoint location preserves all of the essential information that identifies a query. Each query must have a different checkpoint location. Multiple queries should never have the same location. For more information, see the Structured Streaming Programming Guide. https://docs.databricks.com/en/structured-streaming/query-recovery.html

upvoted 2 times

...