Exam AWS Certified Machine Learning Engineer - Associate MLA-C01 topic 1 question 46 discussion

✅ Explanation: SageMaker Pipelines + AWS Glue Integration Goal: You want SageMaker Pipelines to wait for AWS Glue jobs to finish, since their outputs are needed during the model development phase. You also want to keep operational overhead low. Why Callback steps are the best option: Callback steps in SageMaker Pipelines allow you to pause the pipeline execution until an external process completes. You can: Use the callback step to trigger an AWS Glue workflow (or job). Use a Lambda function or a service like EventBridge or Step Functions to report back to SageMaker when the Glue job completes. This approach provides tight integration while maintaining low operational overhead—you don’t need to build a whole orchestration layer.

https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-pipelines/tabular/custom_callback_pipelines_step/sagemaker-pipelines-callback-step.html This shows how to use the Callback Step and how to include a Glue ETL job as part of a SageMaker ML pipeline.

I think it's B when I consider 'LEAST operational overhead'.

https://aws.amazon.com/blogs/machine-learning/extend-amazon-sagemaker-pipelines-to-include-custom-steps-using-callback-steps/

LEAST effort: https://aws.amazon.com/blogs/machine-learning/extend-amazon-sagemaker-pipelines-to-include-custom-steps-using-callback-steps/ The example is exactly for the same use-case as the question.

This approach allows you to directly integrate the outputs of AWS Glue jobs into SageMaker Pipelines, leveraging the processing steps to handle the data. It minimizes additional orchestration overhead and keeps the workflow streamlined within SageMaker Pipelines