Exam AWS Certified Data Analytics - Specialty topic 1 question 4 discussion

I don't understand why everyone is choosing C. First of all, KPL does not send data to Kinesis Firehose, it sends data to Kinesis Data Streams, so C is very much incorrect. Second, the term KPL agent, there is no such thing. We would install Kinesis Agent on EC2 and not KPL Agent, so B and C are incorrect. In Option D, you see that it is using Cloudwatch logs which already offers what the customer wants... and implementing an Opensearch with Kibana for it would be overengineering and duplicating the same solution in another tool, duplicating data and cost. In option A it is using CloudWatch logs and Athena, which is easy to configure and works well. The answer does not say it, but the customer could use Cloudwatch Dashboards and Cloudwatch Metrics generated from the log stream. Activating EC2 detailed monitoring is not necessary though. Another thing: you guys are saying that Data Streams cannot deliver data to ES, but actually by using a Lambda you can pretty much do this... This question is very strange... But, by elimination, I would stay with option A.

D is correct. CloudWatch subscription is realtime. Push logs to Kinesis Firehose or Streams which can push into ElasticSearch for aggregation and dashboard. CloudWatch-Subs -> Amazon Streams - > Firehose -> ElasticSearch -> Dashbaord

This is the only valid option. KDS doesn't have integration with Opensearch

C is correct, KDS cannot send data to OpenSearch as stated in answer D. Also answer D does not mention how the logs will get ingested from the EC2 instances to CloudWatch. logs.

real-time capabilities are not a strict requirement and minimal delay is the primary consideration, then using Kinesis might indeed be considered overkill for your specific use case. In such scenarios, a simpler and cost-effective solution, such as Option A (CloudWatch + S3 + Athena), could be more suitable. This architecture allows for periodic log analysis without the need for real-time streaming. Option A provides a straightforward setup with CloudWatch for log collection, S3 for storage, and Athena for fast and interactive log analytics. It is a serverless solution that can meet your requirements while minimizing complexity. Consider your specific needs, the frequency of log analysis, and the trade-offs between simplicity, cost, and real-time capabilities when making your decision. If real-time insights are unnecessary, a less complex solution like Option A might be more appropriate.

correct answer

Very weird one: I'm also inbetween C & D but bot are making no sense because of: C: KPL can't write into Firehose. The normal process flow would be KPL -> KDS -> KDF. D: Is weird because they KDS can't write directly to OpenSearch. The normal process flow would be CloudWatch -> KDS -> Lambda -> OpenSearch. In addition to that, the question is talking about logs of the application itself and not metrics/logs of the EC2 - Instance (I'm not a native english speaker but this is how I've understood the question). I don't think that CloudWatch is the right tool for that. In the end I go with C (hoping KPL agent is somewhat of a typo and they mean just Agent).

KPL cannot send data directly to KDF. "You can send data to your Kinesis Data Firehose Delivery stream using different types of sources: You can use a Kinesis data stream, the Kinesis Agent, or the Kinesis Data Firehose API using the AWS SDK. You can also use Amazon CloudWatch Logs, CloudWatch Events, or AWS IoT as your data source." Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-write.html# So, C is incorrect. D would be 100% correct IF it had KDF instead of KDS. I am guessing it's a typo or who knows what !! I am gonna pretend this question won't show up in the exam :)

D. I thought in a different option but: A. "detailed monitoring on Amazon EC2" is for metrics, and report 1 minute period, so is NOT real time BC. Do not exist KPL agent D. Is the real time option

So what is the answer here?

The correct answer is D, although in reality none is correct. I had this same question but instead of KDS it was Kinesis Data Firehose. Answer A B C were also among the answers and were all incorrect. Answer C will cause some delays while CloudWatch subscriptions are near-real time

C - i guess correct D - says KDS --> OpenSearch does not work.

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/integrations.html#integrations-kinesis

Clearly C, D can't be correct, Kinesis Data Streams cant send data directly to OpenSearch

going w D

D is the answer

I think D is not correct, as Kinesis Data Streams can't push directly to OpenSearch Service: "You can still use other sources to load streaming data, such as Amazon Kinesis Data Firehose and Amazon CloudWatch Logs, which have built-in support for OpenSearch Service. Others, like Amazon S3, Amazon Kinesis Data Streams, and Amazon DynamoDB, use AWS Lambda functions as event handlers." You need Lambda to do that. https://docs.aws.amazon.com/opensearch-service/latest/developerguide/integrations.html So C is the correct for me. Yes, Amazon Kinesis Producer Library (KPL) agent does not exist, but i think they are refering just to the agent https://docs.aws.amazon.com/streams/latest/dev/writing-with-agents.html