Exam AWS Certified Solutions Architect - Associate SAA-C03 topic 1 question 258 discussion

No, D should be correct. "LEAST operational overhead" => Should you fully manage service like Glue instead of manually like the answer A.

This solution meets the requirements with the least operational overhead because AWS Glue is a fully managed ETL service that makes it easy to move data between data stores. AWS Glue can read .csv files from an S3 bucket and write the data into Parquet format in another S3 bucket. The AWS Lambda function can be triggered by an S3 PUT event when a new .csv file is uploaded, and it can start the AWS Glue ETL job to convert the file to Parquet format. This solution does not require managing any servers or clusters, which reduces operational overhead.

AWS Glue can run your extract, transform, and load (ETL) jobs as new data arrives. For example, you can configure AWS Glue to initiate your ETL jobs to run as soon as new data becomes available in Amazon Simple Storage Service (S3). Clearly you don't need a lambda function to initiate the ETL job. https://aws.amazon.com/glue/#:~:text=to%20initiate%20your-,ETL,-jobs%20to%20run Option A requires writing code to perform the file conversion. In the exam option D would the best answer.

D is correct

A. introduces significant operational overhead. This approach requires managing the Lambda, handling concurrency, and ensuring proper error handling for large file sizes, which can be challenging. B. adds unnecessary complexity and operational overhead. Managing the Spark job, handling scalability, and coordinating the Lambda invocations for each file upload can be cumbersome. C. introduces additional complexity and may not be the most efficient solution. It involves managing Glue resources, scheduling Lambda, and querying data even when no new files are uploaded. Option D leverages AWS Glue's ETL capabilities, allowing you to define and execute a data transformation job at scale. By invoking the ETL job using an Lambda function for each S3 PUT event, you can ensure that files are efficiently converted to Parquet format without the need for manual intervention. This approach minimizes operational overhead and provides a streamlined and scalable solution.

Both A and D can works, but A is more simple. It's more close to the "Least Operational effort".

The maximum size for a Lambda event payload is 256 KB - so (A) didn't work with 1GB Files. Glue is recommended for the Parquet Transformation of AWS.

ANS - d https://aws.amazon.com/blogs/database/how-to-extract-transform-and-load-data-for-analytic-processing-using-aws-glue-part-2/ - READ ARTICLE -

Here A is the correct answer. The reason here is the least operational overhead. A ==> S3 - Lambda - S3 D ==> S3 - Lambda - Glue - S3 Also, glue cannot convert on fly automatically, you need to write some code there. If you write the same code in lambda it will convert the same and push the file to S3 Lambda has max memory of 128 MB to 10 GB. So, it can handle it easily. And we need to consider cost also, glue cost is more. Hope many from this forum realize these differences.

A is unlikely to work as Lambda may struggle with 1GB size: "< 64 MB, beyond which lambda is likely to hit memory caps", see https://stackoverflow.com/questions/41504095/creating-a-parquet-file-on-aws-lambda-function

Should be D as Glue is self managed service and provides tel job for converting cab files to parquet off the shelf.

https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/three-aws-glue-etl-job-types-for-converting-data-to-apache-parquet.html

AWS Glue is right solution here.

I am thinking D. A says lambda will download the .csv...but to where? that seem manual based on that

I think A

https://www.examtopics.com/discussions/amazon/view/83201-exam-aws-certified-solutions-architect-associate-saa-c02/