Exam AWS Certified Data Analytics - Specialty topic 1 question 92 discussion

I think that answer should be "A". Lambda is the most cost effective solution and satisfies both memory and time requirements. Additionally lambda support "tumbling windows" https://aws.amazon.com/blogs/compute/using-aws-lambda-for-streaming-analytics/ so in my opinion "A" is the best option in this question.

Glue DataBrew supports window functions https://docs.aws.amazon.com/databrew/latest/dg/recipe-actions.functions.window.html will go with D

A Python script is already provided, and rewriting it in Scala or Spark would be unnecessary work. Time windowing here is not referencing streaming, the files are already inside S3.

A - lambda invocation tumbling window functions

A, not D. Confucius says "Do not use a cannon to kill a mosquito."

In normal answer sure I will go for D, but most cost effective and only require 1GB of memory, I will go for A AWS Lambda is a serverless compute service that runs code in response to events and automatically scales based on the incoming traffic. With Lambda, the user only pays for the compute time that the function uses, making it a cost-effective option. Since the Python script has been tested to run within a few minutes with 1 GB of memory, AWS Lambda can easily handle the processing requirements for this project. In addition, since the data generated each night is relatively small (2 MB per bed), AWS Lambda's maximum payload size of 512 MB is more than enough to handle the incoming data. The processed data can also be easily uploaded to the Amazon S3 bucket. Therefore, AWS Lambda with a Python script would be the most cost-effective solution for this project, as it provides a serverless and scalable environment for running the Python script with the required memory and processing capabilities.

MOST cost-effective way is Lambda

Answer A was impossible at the time exam DAS-C01 came out (13 APR 2020.) The AWS blog post announcing the new windowing feature for Lambda is dated 15 DEC 2020.

in my opinion is D. time windowing , 1 GB of memory and a couple of minutes of execution...Glue is better

Lamda to handle tumbling window - data should be limited to 1MB These include: Window start and end: the beginning and ending timestamps for the current tumbling window. State: an object containing the state returned from the previous window, which is initially empty. The state object can contain up to 1 MB of data. isFinalInvokeForWindow: indicates if this is the last invocation for the tumbling window. This only occurs once per window period. isWindowTerminatedEarly: a window ends early only if the state exceeds the maximum allowed size of 1 MB. Therefore its D - As glue job with Pyspark can handle this volume of data aggregation by each bed.

Maybe the question is testing do you know PySpark can do windowing function. PySpark is also Python script. In term of cost effective, Glue is cheaper cpmpare to EMR. So overall, I vote for D.

The process is already tested with python, so there is not a concern on the fact that we might need a PySpark job to work with dataframes, etc. Also it seems a bit overkill to set up a Spark job to process 2mb files. Again, if the quote saying that "the script is tested with python" was not present, I would choose PySpark or Scala. The Python mention might also be for us to choose PySpark over Scala, but it is not a easy question and I think both answers are quite right. I think it must be A

A is way cost effective.

Go for A due to cost effectiveness as Glue is much expensive as compare to Lambda

The data size for every bed is 2 MB. In this case, Lambda should faster than Glub job

A: This new feature introduces the concept of a tumbling window, which is a fixed-size, non-overlapping time interval of up to 15 minutes. To use this, you specify a tumbling window duration in the event-source mapping between the stream and the Lambda function. When you apply a tumbling window to a stream, items in the stream are grouped by window and sent to the processing Lambda function. The function returns a state value that is passed to the next invocation of the tumbling window. https://aws.amazon.com/blogs/compute/using-aws-lambda-for-streaming-analytics/

D over A for tumbling window