Exam Certified Data Engineer Professional topic 1 question 97 discussion

Actual exam question from Databricks's Certified Data Engineer Professional

Question #: 97
Topic #: 1

[All Certified Data Engineer Professional Questions]

A data architect has heard about Delta Lake’s built-in versioning and time travel capabilities. For auditing purposes, they have a requirement to maintain a full record of all valid street addresses as they appear in the customers table.

The architect is interested in implementing a Type 1 table, overwriting existing records with new values and relying on Delta Lake time travel to support long-term auditing. A data engineer on the project feels that a Type 2 table will provide better performance and scalability.

Which piece of information is critical to this decision?

A. Data corruption can occur if a query fails in a partially completed state because Type 2 tables require setting multiple fields in a single update.
B. Shallow clones can be combined with Type 1 tables to accelerate historic queries for long-term versioning.
C. Delta Lake time travel cannot be used to query previous versions of these tables because Type 1 changes modify data files in place.
D. Delta Lake time travel does not scale well in cost or latency to provide a long-term versioning solution.
E. Delta Lake only supports Type 0 tables; once records are inserted to a Delta Lake table, they cannot be modified.

Show Suggested Answer

Suggested Answer: D 🗳️

by 60ties at Nov. 15, 2023, 1:44 p.m.

...

vctrhugo

1 year, 5 months ago

Selected Answer: D

Delta Lake’s time travel feature allows you to access previous versions of the data, which can be useful for auditing purposes. However, if you’re planning to use time travel as a long-term versioning solution, it’s important to know that it may not scale well in terms of cost or latency. This is because every time you perform a write operation, a new version of the data is created, which can consume significant storage over time. Additionally, querying older versions of the data may require scanning through many files, which can increase query latency.

upvoted 4 times

...

60ties

1 year, 7 months ago

Selected Answer: D

D makes more sense

upvoted 4 times

...