Exam AWS Certified Machine Learning - Specialty topic 1 question 311 discussion

https://aws.amazon.com/it/blogs/aws/new-amazon-athena-for-apache-spark/

Amazon Athena makes it easy to interactively run data analytics and exploration using Apache Spark without the need to plan for, configure, or manage resources reference :-https://docs.aws.amazon.com/athena/latest/ug/notebooks-spark.html

B - SageMaker Athena and redshift does not support Apache spark scripts EMR requires managing infra

SageMaker provide serverless, spark cluster notebook, pay as you go if you remember to close it when you finished your work.

Amazon Athena does not natively support running Python code directly. Amazon Athena is primarily a serverless, interactive query service that allows you to analyze data in Amazon S3 using standard SQL. Use Apache Spark from within Amazon SageMaker. Amazon SageMaker allows you to run Jupyter notebooks and provides managed Apache Spark integration, which means you don't need to manage the underlying compute resources yourself. You can also use SageMaker to perform the analysis and pay only for the resources you consume during the execution of your queries.

A and not B also because of paying for queries that you run. Notebooks will continue to run and cost money

https://docs.aws.amazon.com/athena/latest/ug/notebooks-spark-working-with-notebooks.html

https://docs.aws.amazon.com/athena/latest/ug/notebooks-spark-editor.html

Correct Answer: A https://docs.aws.amazon.com/athena/latest/ug/notebooks-spark.html

It's A - Using Apache Spark on Amazon Athena https://aws-sdk-pandas.readthedocs.io/en/3.2.1/tutorials/041%20-%20Apache%20Spark%20on%20Amazon%20Athena.html

Serverless, Python, and Notebook are key elements for making the decision. It's B

https://docs.amazonaws.cn/en_us/athena/latest/ug/notebooks-spark-getting-started.html

Just thinking out loud, how can it be not Redshift as well? The question also mentions pay for queries, and handle petabyte of data. Spark is an integration possible with Amazon Redshift, and Redshift has serverless version too. https://aws.amazon.com/blogs/aws/new-amazon-redshift-integration-with-apache-spark/

A: No - Athena does not support python code B: Yes - Sagemaker is serverless and SageMaker Processing allows you to run Spark jobs from a Jupyter notebook using Python. You only pay for resources used during processing jobs. C: No - involves managing the EMR cluster. You pay for running EC2 instances whether in use or not. D: No - Redshift can't run spark jobs and no native support for python/Jupiter notebooks