Exam Professional Data Engineer topic 1 question 100 discussion

Actual exam question from Google's Professional Data Engineer

Question #: 100
Topic #: 1

[All Professional Data Engineer Questions]

You have a requirement to insert minute-resolution data from 50,000 sensors into a BigQuery table. You expect significant growth in data volume and need the data to be available within 1 minute of ingestion for real-time analysis of aggregated trends. What should you do?

A. Use bq load to load a batch of sensor data every 60 seconds.
B. Use a Cloud Dataflow pipeline to stream data into the BigQuery table.
C. Use the INSERT statement to insert a batch of data every 60 seconds.
D. Use the MERGE statement to apply updates in batch every 60 seconds.

Show Suggested Answer

Suggested Answer: B 🗳️

by jvg637 at March 19, 2020, 5:39 p.m.

Comments

Submit Cancel

jvg637

Highly Voted 5 years, 3 months ago

I think we need a pipeline, so it's B to me.

upvoted 29 times

...

MaxNRG

Highly Voted 3 years, 6 months ago

Selected Answer: B

Is B, if we expect a growth we’ll need some buffer (that will be pub-sub) and the dataflow pipeline to stream data in big query. The tabledata.insertAll method is not valid here.

upvoted 7 times

...

Erg_de

Most Recent 8 months, 1 week ago

Selected Answer: B

For real-time analysis and quick data availability, the appropriate option is the combination of the pipeline with BigQuery.

upvoted 1 times

...

Helinia

1 year, 7 months ago

“need the data to be available within 1 minute of ingestion for real-time analysis” → low latency requirement → Dataflow streaming The database can either be BQ or BigTable for this kind of requirement in data volume and latency. But it mentioned that the destination has to be BQ, so B.

upvoted 1 times

...