Exam DP-600 topic 1 question 74 discussion

Actual exam question from Microsoft's DP-600

Question #: 74
Topic #: 1

HOTSPOT -

You have a Fabric tenant that contains lakehouse named Lakehouse1. Lakehouse1 contains a Delta table with eight columns.

You receive new data that contains the same eight columns and two additional columns.

You create a Spark DataFrame and assign the DataFrame to a variable named df. The DataFrame contains the new data.

You need to add the new data to the Delta table to meet the following requirements:

• Keep all the existing rows.
• Ensure that all the new data is added to the table.

How should you complete the code? To answer, select the appropriate options in the answer area.

NOTE: Each correct selection is worth one point.

Show Suggested Answer

Suggested Answer:

by SJSull at April 5, 2024, 9:22 a.m.

Comments

Submit Cancel

SJSull

Highly Voted 7 months ago

append, ("mergeschema", "true") - schema evolution, same columns and 2 added new columns

upvoted 50 times

...

9878eb9

Highly Voted 6 months, 2 weeks ago

- append - ("mergeschema", "true") https://learn.microsoft.com/en-us/azure/databricks/delta/update-schema Add columns with automatic schema update Columns that are present in the DataFrame but missing from the table are automatically added as part of a write transaction when: write or writeStream have .option("mergeSchema", "true") spark.databricks.delta.schema.autoMerge.enabled is true

upvoted 20 times

...

vernillen

Most Recent 4 months, 4 weeks ago

1. Append --> Because of "Keep all the existing rows" 2. mergeSchema, true --> Because there will be 2 additional columns, so it's they are not deleted from the new version. If they would have been deleted from the new version, you would have to use "overwriteSchema, true". Since that option replaces the existing schema with the schema of the new dataframe. mergeSchema, on the contrary, allows the addition of new columns rather than overwriting the schema.

upvoted 5 times

...

4371883

6 months, 1 week ago

append mergeschema="true" append as all existing rows must be kept. according to this: https://learn.microsoft.com/en-us/azure/databricks/delta/update-schema overwriteschema is for "change a column’s type or name or drop a column" mergeschema is for "Columns that are present in the DataFrame but missing from the table are automatically added"

upvoted 7 times

...

Exam DP-600 All Questions

View all questions & answers for the DP-600 exam