Exam AWS Certified Solutions Architect - Professional SAP-C02 topic 1 question 75 discussion

The most cost-effective and efficient solution that meets the design requirements would be option B, Design the application to store each incoming record in an Amazon DynamoDB table properly configured for the scale. Configure the DynamoDB Time to Live (TTL) feature to delete records older than 120 days. DynamoDB is a NoSQL key-value store designed for high scale and performance. It is fully managed by AWS and can easily handle millions of small records per minute. Additionally, with the TTL feature, you can set an expiration time for each record, so that the data can be automatically deleted after the specified time period.

A. No, because millions of writes to a single .csv file would cause read and write latency B. Yes, because DynamoDB can support peaks of more than 20 million requests per second. C. No, because creating nightly cron is unnecessary, and a relation database isn't designed to ingest millions of small records per minute D. No, because S3 supports 210,000 PUT requests per minute (3,500 requests per second * 60 seconds per min) which is far less than 1,000,000+ writes per minute

Read the requirements: MOST cost-effective and meets the design requirements. Note "retrieved with low latency". DynamoDB latency is single digits, where as S3 is 100-200 milliseconds.

For those who said B, how many WCU is needed for dynamoDB? Given: 1 million records per minute 4KB per record This translates to approximately 16,667 records per second (1,000,000 / 60) For DynamoDB WCU calculation: 1 WCU = 1 write per second for items up to 1KB For items larger than 1KB, the WCU is rounded up to the next 1KB For 4KB items, each write will consume 4 WCUs Therefore: WCUs needed = (Records per second) × (Item size in KB rounded up) WCUs = 16,667 × 4 WCUs = 66,668 WCUs First, you need to increase the quotas for that table by submitting a support ticket. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html Second, this is very expensive. Obviously, combine it with kinesis data agent and firehorse that write to S3 will be much reliable options but it will increase the cost significantly. But still cheaper than the dynamo db options. https://calculator.aws/#/estimate?id=87f1df21449660b0b9d61a6c1153632b1983d2e4

Option B is too expensive.

If you really think about being cost effective than Option D is the right choice

Why Option D Might Be Cost-Effective: Lower Storage Costs: S3 storage is generally cheaper than DynamoDB when dealing with large amounts of data (e.g., $0.023/GB/month for S3 Standard vs. $0.25/GB/month for DynamoDB on-demand). Batching Reduces API Call Costs: By batching multiple records into a single object, you reduce the number of PUT requests to S3. This can lead to lower API costs compared to writing each record individually to DynamoDB. Lifecycle Policies for Data Expiry: S3 lifecycle policies automatically clean up data older than 120 days, similar to DynamoDB's TTL feature.

D. Design the application to batch incoming records before writing them to an Amazon S3 bucket. Update the metadata for the object to contain the list of records in the batch and use the Amazon S3 metadata search feature to retrieve the data. Configure a lifecycle policy to delete the data after 120 days.

Obviously it is DynamoDB. Although as a side node I would say it is probably a very bad choice as it would be astronomically expensive for millions of writes per minute…. A Kinesis Data Streams would make much more sense especially that the data is only needed for 3 months…

B, dynamodb is the best option

For small records less than 4 KB, DynamoDB can efficiently handle the ingestion of millions o records per minute from devices around the world, meeting the application's design requirements for low-latency data access. Additionally, DynamoDB's Time to Live (TTL) feature allows for automatic deletion of items after a specific period, aligning with the requirement to store data for only 120 days.

A = S3 is not great with small files and searching for data based on index (a common pattern is to store object metadata in a database like DDB, OpenSearch or RDS/Aurora). Many small files can lead to high costs for retrieval B = correct C = single-table design, high volume write/retrieval os small object and no need for complex query are better served and cost less with DDB rather than RDS D = more efficient than A, but still S3 metadata search feature is limited

see uC6rW1aB's answer

B is the best for cost-effective. D is more cost for S3 request

Ref: https://aws.amazon.com/dynamodb/pricing/on-demand/ DynamoDB read requests can be either strongly consistent, eventually consistent, or transactional. A strongly consistent read request of up to 4 KB requires one read request unit. For items larger than 4 KB, additional read request units are required.

Dinamo DB is at least 5X more expensive than S3 for this use case. There are million of writing and each is 4K, total disk space is 10-15TB.

Although both B and D are correct, Option D is more cost effective.