Exam Certified Data Engineer Professional topic 1 question 23 discussion

Actual exam question from Databricks's Certified Data Engineer Professional

Question #: 23
Topic #: 1

[All Certified Data Engineer Professional Questions]

Which statement characterizes the general programming model used by Spark Structured Streaming?

A. Structured Streaming leverages the parallel processing of GPUs to achieve highly parallel data throughput.
B. Structured Streaming is implemented as a messaging bus and is derived from Apache Kafka.
C. Structured Streaming uses specialized hardware and I/O streams to achieve sub-second latency for data transfer.
D. Structured Streaming models new data arriving in a data stream as new rows appended to an unbounded table.
E. Structured Streaming relies on a distributed network of nodes that hold incremental state values for cached stages.

Show Suggested Answer

Suggested Answer: D 🗳️

by 8605246 at Aug. 6, 2023, 10:49 a.m.

Comments

Submit Cancel

8605246

Highly Voted 1 year, 4 months ago

correct; The key idea in Structured Streaming is to treat a live data stream as a table that is being continuously appended. This leads to a new stream processing model that is very similar to a batch processing model. You will express your streaming computation as standard batch-like query as on a static table, and Spark runs it as an incremental query on the unbounded input table. Let’s understand this model in more detail. https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html

upvoted 5 times

...