Exam Certified Data Engineer Associate topic 1 question 77 discussion

I had the exam today and option A & B weren't exist, correct answer is D.

None is never a solution

Pipelines supports both language

The correct response is A. None of these changes will need to be made. Delta Live Tables supports both Python and SQL, as well as streaming and batch sources. This means that the existing medallion-based multi-hop architecture can be maintained, and the pipeline can continue to use both Python and SQL for different layers. Therefore, no changes are necessary when migrating to Delta Live Tables.

Delta Live Tables (DLT) is designed to support a medallion-based architecture (raw → bronze → silver → gold) and allows for a combination of Python and SQL in pipeline definitions. It also supports both batch and streaming sources.

The correct answer is: A. None of these changes will need to be made. Explanation: Delta Live Tables (DLT) supports both Python and SQL: The data engineer can continue writing transformations for the raw, bronze, and silver layers in Python. The data analyst can work on the gold layer in SQL. Medallion-based architecture: Delta Live Tables is well-suited for the medallion architecture (raw -> bronze -> silver -> gold). It is commonly used to build reliable and maintainable data pipelines. Streaming sources: Delta Live Tables fully supports streaming inputs and can handle both batch and streaming sources natively. Flexibility in implementation: Delta Live Tables does not impose restrictions that require pipelines to be written entirely in either SQL or Python. Both languages can coexist in the same pipeline as needed. Thus, no major changes are required for the migration to Delta Live Tables.

A is correct

Right answer is not listed here. The right answer is "Different notbook may jused for SQL and Python"

D is the answer

A. None of these changes will need to be made. You can continue using the medallion-based architecture, and you do not need to switch entirely to SQL or Python. Delta Live Tables will work with your existing streaming sources and support both SQL and Python.

When migrating to Delta Live Tables, you can continue using the medallion-based architecture, work with streaming sources, and write the pipeline in either SQL or Python. Therefore, no major changes are required for the pipeline in this scenario.

D: Delta Live Tables is primarily designed to work with batch processing rather than streaming. This means that when migrating a pipeline to Delta Live Tables, any streaming sources used in the original pipeline will need to be replaced with batch sources. In the scenario described, where the raw source of the pipeline is a streaming input, the data engineer and data analyst will need to modify their pipeline to read data from a batch source instead. This could involve changing the way data is ingested and processed to align with batch processing paradigms rather than streaming. Additionally, Delta Live Tables enables the integration of both SQL and Python code within a pipeline, so there's no strict requirement to write the pipeline entirely in SQL or Python. Both the data engineer's Python code for the raw, bronze, and silver layers and the data analyst's SQL code for the gold layer can still be used within the Delta Live Tables environment. Overall, the key change needed when migrating to Delta Live Tables in this scenario is transitioning from a streaming input source to a batch source to align with the batch processing nature of Delta Live Tables.

A is correct

Cleared the exam today . Option A and B were not available in the exam . There was a different option which was correct.

B - DLT support medallion architecture (see example in : https://docs.databricks.com/en/delta-live-tables/transform.html#combine-streaming-tables-and-materialized-views-in-a-single-pipeline) C - DLT can mix Python and SQL using multiple notebooks (according to https://docs.databricks.com/en/delta-live-tables/tutorial-python.html You cannot mix languages within a Delta Live Tables source code file. You can use multiple notebooks or files with different languages in a pipeline) D - DLT manage streaming sources using streaming tables (ex : https://docs.databricks.com/en/delta-live-tables/load.html#load-data-from-a-message-bus) E - DLT support python and sql (https://docs.databricks.com/en/delta-live-tables/tutorial-python.html and https://docs.databricks.com/en/delta-live-tables/tutorial-sql.html) Correct answer is A by elimination

I think the answer is A

It should be A. Medallion architecture can be used in DLT pipeline https://www.databricks.com/glossary/medallion-architecture "Databricks provides tools like Delta Live Tables (DLT) that allow users to instantly build data pipelines with Bronze, Silver and Gold tables from just a few lines of code."