Exam AWS Certified Data Analytics - Specialty topic 1 question 136 discussion

store data in S3 not in Kinesis

A & B out as you cannot store data for more than 7 days in KDS. C - Possibly but won't be cost-effective. D - Cost-Effective with least amount of operational overhead.

B is the right answer, Kinesis Data Stream can keep raw data up to 365 days. Anytime they ask about click-stream analytics it should go to Kinesis Data Analytics.

B: I passed the test

analytics application will process the raw data only once so why store in S3?

The solution that meets the requirements with the least amount of operational effort is Option B: Use Amazon Kinesis Data Streams to collect the data. Use Amazon Kinesis Data Analytics with Apache Flink to process the data in real-time. Set the retention period of the Kinesis data stream to 8,760 hours. This solution uses Amazon Kinesis Data Streams to collect the data and processes it in real-time using Amazon Kinesis Data Analytics with Apache Flink. This allows for near-real-time clickstream analytics without the need for additional data processing or storage. Additionally, the retention period of the Kinesis data stream can be set to 8,760 hours (1 year), which allows other applications to have immediate access to the raw data for up to 1 year without the need for additional storage or processing. This solution requires the least amount of operational effort as it does not require additional steps for data processing or storage.

For those debating B and D, I am going with B. Least operational overhead, and the giveway is the hours : "The maximum value of a stream's retention period is 8760 hours (365 days)." https://docs.aws.amazon.com/kinesis/latest/APIReference/API_IncreaseStreamRetentionPeriod.html. Be wary of the attempt in the question to use 8.760 hours which i think is sneaky.

Focus on "LEAST amount of operational effort" so you should eliminate EMR from the picture. D recommends EMR and it is operational overhead. B - make sense. Since it can keep data for 365 days and there is no operational overhead.

It says the raw data need to be accessed by other applications, if we put in the Kinesis Stream, how other application can access it? Besides, KDS is not good for store data. S3 is the right choice.

A Kinesis data stream stores records for 24 hours by default, up to 365 days (8,760 hours). https://aws.amazon.com/blogs/big-data/retaining-data-streams-up-to-one-year-with-amazon-kinesis-data-streams/

C. Using Amazon Managed Streaming for Apache Kafka (Amazon MSK) to collect the data, Amazon EMR with Apache Flink to process the data, and setting the log retention hours to 8,760 would meet the requirements with the least amount of operational effort. Amazon MSK is a fully managed service that makes it easy to set up, maintain, and scale Apache Kafka clusters. Amazon EMR can be used to process data from an Amazon MSK stream in real time, and the log retention hours can be set to 8,760 to retain the data for up to 1 year. This solution would require minimal effort to set up and maintain, and would allow other applications to access the raw data for up to 1 year.

B for {{ LEAST amount of operational effort }}

I vote for B. I think C with EMR requires more operational effort.

KDS can keep the data for one year.

Answer is B, KDS can natively store raw data for up to 1 year

Answer-B

Retention and Minimal Operational KDS(8760 hours (365 days)) + KDA(Flink available out of the box) >> KDS + EMR with Flink + S3