Exam AWS Certified Data Analytics - Specialty topic 1 question 60 discussion

Exam question from Amazon's AWS Certified Data Analytics - Specialty

Question #: 60
Topic #: 1

[All AWS Certified Data Analytics - Specialty Questions]

A company is streaming its high-volume billing data (100 MBps) to Amazon Kinesis Data Streams. A data analyst partitioned the data on account_id to ensure that all records belonging to an account go to the same Kinesis shard and order is maintained. While building a custom consumer using the Kinesis Java SDK, the data analyst notices that, sometimes, the messages arrive out of order for account_id. Upon further investigation, the data analyst discovers the messages that are out of order seem to be arriving from different shards for the same account_id and are seen when a stream resize runs.
What is an explanation for this behavior and what is the solution?

A. There are multiple shards in a stream and order needs to be maintained in the shard. The data analyst needs to make sure there is only a single shard in the stream and no stream resize runs.
B. The hash key generation process for the records is not working correctly. The data analyst should generate an explicit hash key on the producer side so the records are directed to the appropriate shard accurately.
C. The records are not being received by Kinesis Data Streams in order. The producer should use the PutRecords API call instead of the PutRecord API call with the SequenceNumberForOrdering parameter.
D. The consumer is not processing the parent shard completely before processing the child shards after a stream resize. The data analyst should process the parent shard completely first before processing the child shards.

Show Suggested Answer

Suggested Answer: D 🗳️

by Priyanka_01 at Aug. 15, 2020, 5:41 a.m.

Disclaimers:

- ExamTopics website is not related to, affiliated with, endorsed or authorized by Amazon.
- Trademarks, certification & product names are used for reference only and belong to Amazon.

Comments

Submit Cancel

Priyanka_01

Highly Voted 3 years, 8 months ago

D:https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-after-resharding.html the parent shards that remain after the reshard could still contain data that you haven't read yet that was added to the stream before the reshard. If you read data from the child shards before having read all data from the parent shards, you could read data for a particular hash key out of the order given by the data records' sequence numbers. Therefore, assuming that the order of the data is important, you should, after a reshard, always continue to read data from the parent shards until it is exhausted. Only then should you begin reading data from the child shards.

upvoted 80 times

...

pk349

Most Recent 2 years, 1 month ago

D: I passed the test

upvoted 1 times

...

cloudlearnerhere

2 years, 7 months ago

Selected Answer: D

Correct answer is D as Kinesis Data Streams resharding causes the existing data to be in parent shard and new data is routed to child shards. The issue can occur if the consumers starts processing the child shard without completely processing the parent shard.

upvoted 4 times

...