Exam AWS Certified DevOps Engineer - Professional DOP-C02 topic 1 question 146 discussion

I think it's B, Amazon Lookout for metrics can detect anomalies from S3 bucket and trigger Lambda https://aws.amazon.com/lookout-for-metrics/

Using KDA for anomaly detection, means you can use the built-in RCF (Random Cut Forest) ML algorithm. Option B: Firehose and S3 adds latency. Further, Amazon Lookout for Metrics is more suitable for business metrics than network traffic. Options C&D require you to implement your own Lambda function, which means more error prone and more maintenance.

B for me

- Data Streaming: Use Amazon Kinesis Data Firehose to deliver VPC flow logs from CloudWatch Logs to an Amazon S3 bucket. - Anomaly Detection: Amazon Lookout for Metrics will monitor the data in the S3 bucket and automatically detect anomalies in the network traffic. - Event Response: When Lookout for Metrics detects an anomaly, it triggers an AWS Lambda function. The Lambda function will then publish an event to the Amazon EventBridge event bus, which can further initiate automated responses, notifications, or alerts.

Although option A uses Kinesis Data Analytics for anomaly detection, setting up and maintaining custom analytics and anomaly detection logic is more complex and less efficient compared to using a managed service like Lookout for Metrics.

A is wrong because kinesis data analytics output must be either kinesis data stream or firehose, can't be lambda directly so there is a missing component

I've reviewed most of the comments, and it seems like everyone is just repeating themselves. I've "googled" and looked at the references. I found examples of both kinesis data streams, kinesis data analytics and firehose. The one step in "A" I have a problem with is "Create an AWS Lambda function to use as the output of the data stream." How can Lambda be an output of a data stream "over time"? I don't think you can identify an anomaly "over time" unless you've got persistent storage for the data (which can be reparsed as necessary to compare past with present). I'm leaning towards "B" unless someone can convince me otherwise (and not by just repeating what others have already said).

Option B involves using Amazon Lookout for Metrics, which is not designed for real-time anomaly detection.

i think B

Lookout for Metrics automatically detects and diagnoses anomalies (outliers from the norm) in business and operational data. It’s a fully managed ML service, which uses specialized ML models to detect anomalies based on the characteristics of your data. You don’t need ML experience to use Lookout for Metrics. Kinesis Data Analytics Studio provides an interactive notebook experience powered by Apache Zeppelin and Apache Flink to analyze streaming data. It also helps productionize your analytics application by building and deploying code as a Kinesis data analytics application straight from the notebook. https://aws.amazon.com/blogs/machine-learning/smart-city-traffic-anomaly-detection-using-amazon-lookout-for-metrics-and-amazon-kinesis-data-analytics-studio/

A. If you google "detecting anomalies in vpc flow logs" every article suggests Kinesis Data Analytics

I'll go with A. Mainly because Kinesis data analytics has anomoly detection using a random cut forest function: https://docs.aws.amazon.com/kinesisanalytics/latest/dev/app-anomaly-detection.html

B - Amazon Lookout for Metrics Automatically detect anomalies within metrics and identify their root causes. So would fit the requirements

Option A is preferable for scenarios requiring real-time processing and anomaly detection in streaming data, such as VPC flow logs, with the capability to quickly initiate responses to detected anomalies. It offers a more streamlined and immediate approach to monitoring and responding to network traffic anomalies, making it highly suitable for the company's needs regarding their critical compute infrastructure with predictable traffic patterns. Option B might still be considered if the company's workflow is more adapted to batch processing and the delays inherent in data delivery and processing are acceptable. However, for immediate anomaly detection and response, Option A stands out as the more appropriate solution.

Kinesis Data Firehose determines how often to write to S3 by buffer settings, which is not realtime enough to handle VPC flow log, which can be fatal depending on the content of the `CRITICAL compute infrastructure`. Kinesis Data Analytics has machine learning solutions such as RANDOM_CUT_FOREST in addition to fixed detection by normal SQL.

B without a doubt

Option B is the most suitable for the scenario. Kinesis Data Firehose: It allows the streaming of data to an S3 bucket, providing a durable storage solution. Lookout for Metrics: It is designed to detect anomalies in your data and can be configured to monitor the data stored in the S3 bucket for anomalies.