Exam Professional Data Engineer topic 1 question 139 discussion

Job generators (they would be the publishers). Job runners = subscribers Question mentions that it must scale (of which push subscription has automatic scaling) and can accommodate additional new applications (this can be solved by having multiple subscriptions, with each relating to a unique application) to a central topic

A. App Engine API: While scalable, it introduces a central point of failure and might not be as performant as Pub/Sub for high-volume data. C. Cloud SQL: Not designed for real-time data sharing and continuous updates, leading to potential bottlenecks and performance issues. D. Cloud Spanner: Offers strong consistency and global distribution, but its pricing model might be less suitable for high-volume, cost-sensitive workloads compared to Pub/Sub.

B to decouple jobs being generated and run. Pub/Sub also scales seamlessly

B. Use a Cloud Pub/Sub topic to publish jobs, and use subscriptions to execute them. Scalability: Cloud Pub/Sub is a highly scalable messaging service that can handle a significant volume of messages and subscribers. It can easily accommodate increases in usage as your data pipeline scales. Decoupling: Using Pub/Sub decouples the job generators from the job runners, which is a good architectural choice for flexibility and scalability. Job generators publish messages to a topic, and job runners subscribe to that topic to execute jobs when they are available. Adding New Applications: With Cloud Pub/Sub, adding new applications (new publishers or subscribers) is straightforward. You can simply create new publishers to send jobs or new subscribers to consume jobs without impacting existing components.

key words here: job generators (pushlish message on pub/sub) and job runners(subscribe message for further analysis). You may add as much as pushlishing job and subscribing job to same topic. So Answer B. Using API , app engine is also good approach but its more complex than pub/sub.

B is the answer.

A Since it's application i will go with

use pubsub

I would tend to think B , one of the use of pub/sub is decoupling app

I choose B