Exam AWS Certified Data Analytics - Specialty topic 1 question 7 discussion

It should be A, large number of small files ins3 will slow down reads

Merge the files in Amazon S3 to form larger files will definitely increase read performance. So option A is the right choice.

Everyone is saying A, which is write but why because 1000's shard and per shard capacity is 1 mb , So 1000's of files per second . which require merge to improve the query performance.

A for sure

A: I passed the test

A. This bit of AWS documentation: https://docs.aws.amazon.com/athena/latest/ug/performance-tuning-s3-throttling.html says "If possible, avoid having a large number of small files. Amazon S3 has a limit of 5500 requests per second, and your Athena queries share this same limit. If you scan millions of small objects in a single query, your query will likely be throttled by Amazon S3."

A. Merge the files in Amazon S3 to form larger files. To improve query performance when using Amazon Athena to access data from an Amazon S3 bucket, the streaming application should merge the files in S3 to form larger files. When the streaming application writes data to S3 every 10 seconds, it creates small files, which can lead to a large number of small files over time. This can lead to performance degradation in Athena queries as more small files mean more metadata needs to be scanned, and more file operations are required to read data. By merging small files into larger files, the number of files in the bucket can be reduced, which can significantly improve Athena query performance. Increasing the number of shards in Kinesis Data Streams, adding more memory and CPU capacity to the streaming application, or writing files to multiple S3 buckets are not directly related to the issue of degraded query performance in Athena.

s3 has a limit of 5500 requests per second, combining reduces the requests https://docs.aws.amazon.com/athena/latest/ug/performance-tuning.html

Correct answer is A as merging files to form a bigger file can help optimize and improve query performance. Option B is wrong as increasing shards would only increase the ingestion flow. Options C & D are wrong as it does not improve Athena's query performance.

Merging small files into larger files will reduce the number of compute activities and speed up the process

https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/

Answer should be A

A as merging small files into one large file will result in less meta data to maintain for the Data Catalog to maintain which results in Athena to scan data faster

Things can be done to increase performance of Athena are use columnar formats, use small number of large files, use partitions. So the answer should be A.

Answer should be A

Answer is A

A is the correct answer. merge small files into larger files works as expected