Exam Professional Data Engineer All Questions

View all questions & answers for the Professional Data Engineer exam

Exam Professional Data Engineer topic 1 question 45 discussion

Actual exam question from Google's Professional Data Engineer

Question #: 45
Topic #: 1

[All Professional Data Engineer Questions]

You work for a manufacturing plant that batches application log files together into a single log file once a day at 2:00 AM. You have written a Google Cloud
Dataflow job to process that log file. You need to make sure the log file in processed once per day as inexpensively as possible. What should you do?

A. Change the processing job to use Google Cloud Dataproc instead.
B. Manually start the Cloud Dataflow job each morning when you get into the office.
C. Create a cron job with Google App Engine Cron Service to run the Cloud Dataflow job.
D. Configure the Cloud Dataflow job as a streaming job so that it processes the log data immediately.

Show Suggested Answer

Suggested Answer: C 🗳️

by [deleted] at March 21, 2020, 7:34 a.m.

Comments

Submit Cancel

[Removed]

Highly Voted 4 years, 8 months ago

Answer: C

upvoted 22 times

...

[Removed]

Highly Voted 4 years, 8 months ago

Answer: C Description: Scheduler for adhoc jobs – 3 jobs free and $0.10 per job

upvoted 13 times

...

grshankar9

Most Recent 4 months, 1 week ago

Selected Answer: C

App Engine Cron is limited to scheduling tasks within your App Engine application, whereas Cloud Scheduler can trigger actions on various Google Cloud services like Cloud Functions, Pub/Sub topics, or external HTTP endpoints.

upvoted 1 times

...

axantroff

1 year ago

Selected Answer: C

Service was renamed, but the answer is still - C

upvoted 1 times

...

imran79

1 year, 1 month ago

C. Using the Google App Engine Cron Service to run the Cloud Dataflow job allows you to automate the execution of the job. By creating a cron job, you can ensure that the Dataflow job is triggered exactly once per day at a specified time. This approach is automated, reliable, and fits the requirement of processing the log file once per day.

upvoted 1 times

...

itsmynickname

1 year, 4 months ago

C. For a modern solution, Cloud Scheduler

upvoted 5 times

...

Maurilio_Cardoso

1 year, 6 months ago

Selected Answer: C

Currently, Cloud Scheduler takes over the scheduling functions.

upvoted 2 times

...

jin0

1 year, 9 months ago

I don't understand why that dataflow is used for processing? even though it should be processed once per a day?? is it more suitable for processing by using Dataproc instead?

upvoted 2 times

mark1223jkh

6 months, 2 weeks ago

Actually, google recommends Dataflow over Dataproc for both batch and streaming. Dataproc is only recommended if you are coming from hadoop, spark, ....

upvoted 2 times

...

captainbu

1 year, 10 months ago

Selected Answer: C

C was correct but nowadays you'd schedule a Dataflow job with Cloud Scheduler: https://cloud.google.com/community/tutorials/schedule-dataflow-jobs-with-cloud-scheduler

upvoted 6 times

...

Ender_H

2 years, 2 months ago

Selected Answer: C

Correct Answer: C. ❌ A: Dataproc is a managed Apache Spark and Apache Hadoop service, makes no sense to use it ❌ B: This might sound as the cheapest, but is highly error prone, besides, anyone in charge of this has a salary and I doubt is a low one. ✅ C: This is the easiest/fastest/cheapest way to trigger job runs, you can even set retry attempts. source: https://cloud.google.com/appengine/docs/flexible/nodejs/scheduling-jobs-with-cron-yaml. ❌ D: Setting this would be much more expensive than the cron-job

upvoted 2 times

...

noob_master

2 years, 5 months ago

Selected Answer: C

Answer: C

upvoted 1 times

...

anji007

3 years, 1 month ago

Ans: C

upvoted 2 times

...

Chelseajcole

3 years, 1 month ago

I know probably this question is testing on if you know corn.yaml and its function in App Engine. But why B will be more expensive? Human capital cost? Let's say if hiring a person click the button will be cheaper than launch an app engine, should we reconsider B?

upvoted 3 times

AmirN

2 years, 5 months ago

Would you rather pay someone $100,000 a year to click 'run' on jobs all day, or have them automate it and do more cutting edge work? This would be opportunity cost.

upvoted 3 times

...