Exam Professional Data Engineer topic 1 question 25 discussion

I would choose D. A and B are wrong since don't notify anything to the monitoring tool. C has no filter on what will be notified. We want only some tables.

D as the key requirement is to have notification on a particular table. It can be achieved using advanced log filter to filter only the table logs and create a project sink to Cloud Pub/Sub for notification. Refer GCP documentation - Advanced Logs Filters: https://cloud.google.com/logging/docs/view/advanced-queries A is wrong as advanced filter will help in filtering. However, there is no notification sends. B is wrong as it would send all the logs and BigQuery does not provide notifications. C is wrong as it would send all the logs.

D is the correct answer because: - we need to advance filtering to filter the logs for the specific table - we need to use monitoring tool for notification.

Good point by MaxNRG about reducing the number of logs sending to Pub/Sub

Theorically Pub/Sub could filters log to forward the right ones to the correct topic. https://cloud.google.com/pubsub/docs/subscription-message-filter So C could be accepted, but It's better if filtering is performed earlier, so in this case D is more performing

D. Using the Stackdriver API, create a project sink with an advanced log filter to export to Pub/Sub, and subscribe to the topic from your monitoring tool. This approach allows you to set up a custom log sink with an advanced filter that targets the specific table and then export the log entries to Google Cloud Pub/Sub. Your monitoring tool can subscribe to the Pub/Sub topic, providing you with instant notifications when relevant events occur without being inundated with notifications from other tables. Options A and B do not offer the same level of customization and specificity in targeting notifications for a particular table. Option C is almost correct but doesn't mention the use of an advanced log filter in the sink configuration, which is typically needed to filter the logs to a specific table effectively. Using the Stackdriver API for more advanced configuration is often necessary for fine-grained control over log filtering.

D makes sense.

D should be the answer

A and B mention nothing about notifications and C would push all data. It's D.

D makes sense

"advanced log filter" is the key word here, all other options push all data ...

D is the best choice

Using the Stackdriver API, create a project sink with advanced log filter to export to Pub/Sub, and subscribe to the topic from your monitoring tool.

D. Option B doesn't make sense

correct answer -> Using the Stackdriver API, create a project sink with advanced log filter to export to Pub/Sub, and subscribe to the topic from your monitoring tool. Option C is also most likely right answer but it doesn't have the filter. We don't want all the tables. We only want one. So the correct answer is D. Logging sink - Using a Logging sink, you can direct specific log entries to your business logic. In this example, you can use Cloud Audit logs for Compute Engine which use the resource type gce_firewall_rule to filter for the logs of interest. You can also add an event type GCE_OPERATION_DONE to the filter to capture only the completed log events. Here is the Logging filter used to identify the logs. You can try out the query in the Logs Viewer. Pub/Sub topic – In Pub/Sub, you can create a topic to which to direct the log sink and use the Pub/Sub message to trigger a cloud function. Reference: https://cloud.google.com/blog/products/management-tools/automate-your-response-to-a-cloud-logging-event

explained by MaxNRG

as explained by MaxNRG