Exam Certified Data Engineer Professional topic 1 question 118 discussion

Actual exam question from Databricks's Certified Data Engineer Professional

Question #: 118
Topic #: 1

[All Certified Data Engineer Professional Questions]

A member of the data engineering team has submitted a short notebook that they wish to schedule as part of a larger data pipeline. Assume that the commands provided below produce the logically correct results when run as presented.

Which command should be removed from the notebook before scheduling it as a job?

A. Cmd 2
B. Cmd 3
C. Cmd 4
D. Cmd 5

Show Suggested Answer

Suggested Answer: D 🗳️

by csrazdan at Sept. 4, 2024, 4:29 a.m.

Comments

Submit Cancel

KadELbied

1 month ago

Selected Answer: D

CMD 6 E - display(finalDF)

upvoted 1 times

...

arekm

5 months, 2 weeks ago

Selected Answer: D

D - Cmd 5 which is display(finalDF) is costly, produces a nicely formatted HTML table which is unnecessary during the production run. On top that there is a limit on how much output the notebook can generate. This causes me some problems in the past for legacy code.

upvoted 1 times

...

smashit

7 months, 1 week ago

There is a limitation when you use display command inside a job. if the output exceeds 20MB it will throw an error. may be that was the reason.

upvoted 1 times

...

csrazdan

9 months, 2 weeks ago

Selected Answer: D

The display function is built specifically for the notebook environment and will not work for Spark. Should you want to print contacts of DF in Spark then replace it with df.show()

upvoted 2 times

...