Exam AWS Certified Data Engineer - Associate DEA-C01 topic 1 question 63 discussion

Amazon Kinesis Data Streams is a fully managed service that allows for seamless integration of diverse data sources, including IoT sensors. By using Kinesis Data Streams as the ingestion mechanism, the company can avoid the overhead of setting up and managing an Apache Kafka cluster or other data ingestion pipelines.

near real time = Kinesis Data streams

Option C is the best solution to meet the requirements

Option C is the best solution to meet the requirements with the least operational overhead: Use Amazon Kinesis Data Streams to ingest real-time sensor data Store the nested JSON data in Amazon DynamoDB for low latency queries The key advantages of Option C are: Kinesis Data Streams fully manages real-time data ingestion with auto-scaling and persistence DynamoDB provides single digit millisecond latency for queries DynamoDB natively supports nested JSON data models Fully managed services minimize operational overhead In contrast: Option A requires managing Kafka clusters Option B uses Lambda which can't provide persistent storage Option D requires integrating SQS, Glue, and RDS leading to complexity

Option C, using Amazon Kinesis Data Streams to capture the sensor data and storing it in Amazon DynamoDB for querying, is the best solution to meet the requirements with the least operational overhead. This solution is well-optimized for real-time data ingestion, supports the desired data format, and provides the necessary query performance.