Exam AWS Certified Data Engineer - Associate DEA-C01 topic 1 question 97 discussion

C - This option ensures low latency by using a short buffer interval (5 seconds). The use of KCL allows for customized processing logic and timely delivery of data to S3. This makes it a strong candidate for minimal latency. D - While this option provides low latency with a 5-second buffer interval, it introduces unnecessary complexity by using Apache Flink for what seems to be a straightforward data ingestion task. This option is overkill for the given use case and may add more operational overhead than necessary.

Kinesis Data Streams cannot deliver directly to S3. Data has to go through Firehose. A is correct but is not lowest latency. I would go with D, as we can set the buffer interval to a low value. We do not need Flink, tho. That's a bit confusing.

Why could not be A? https://aws.amazon.com/blogs/big-data/optimize-downstream-data-processing-with-amazon-data-firehose-and-amazon-emr-running-apache-spark/ It uses Data Firehose + Kinesis Data Streams

Answer:D

Use data streams and KCL, option A would be right but the default buffer for Firehose does not allow it to be correct. D adds extra components that are not needed for delivery of data.

https://aws.amazon.com/about-aws/whats-new/2023/12/amazon-kinesis-data-firehose-zero-buffering/

its C - option D uses 1/ Analytics which summarizes data and gence has delay then passses to 2/ Firehose for deliver and Firehose doesnt say its using zero buffering

Firehose uses multi-part upload for S3 destination when you configure a buffer time interval less than 60 seconds to offer lower latencies. Due to multi-part upload for S3 destination, you will see some increase in S3 PUT API costs if you choose a buffer time interval less than 60 seconds.

I think the answer is C. Kinesis Data Firehose has a minimum buffer interval of 60 seconds (1 minute) or 1 MB of data.