Exam AWS Certified Solutions Architect - Associate SAA-C03 topic 1 question 164 discussion

Amazon SQS supports dead-letter queues (DLQ), which other queues (source queues) can target for messages that can't be processed (consumed) successfully. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html

By integrating both the sender and processor applications with an SQS, messages can be reliably sent from the sender to the processor application for processing. SQS provides at-least-once delivery, ensuring that messages are not lost in transit. If a message fails to process, it can be retained in the queue and retried without impacting the processing of other messages. Configuring a DLQ allows for the collection of messages that repeatedly fail to process, providing visibility into failed messages for troubleshooting and analysis. A is not the optimal choice as it involves managing and configuring an EC2 instance running a Redis, which adds operational overhead and maintenance requirements. B is not the most operationally efficient solution as it introduces additional complexity by using Amazon Kinesis data streams and integrating with the Kinesis Client Library for message processing. D, using SNS, is not the best fit for the scenario as it is more suitable for pub/sub messaging and broadcasting notifications rather than the specific requirement of message processing between two applications.

Dead Letter queues are the most suitable here.

Ans C - Use SQS and dead letter queue. Not ideal because max visibility timeout for SQS is 12h and can't extended by user. I did consider B: KInesis which is for real-time processing - altho' the question doesn't say its not real-time. Anyway, SQS is basic and fits.

C) This option works. There is a 12h maximum visibility timeout, but: "If you don't know how long it takes to process a message, create a heartbeat for your consumer process: Specify the initial visibility timeout (for example, 2 minutes) and then—as long as your consumer still works on the message—keep extending the visibility timeout by 2 minutes every minute." https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/working-with-messages.html

None of the options fit. A) Not operationally efficient B) Kinesis is for real-time processing D) SNS is not suitable for work queuing. C) While this may be the "correct" answer, it also doesn't really fit the problem statement. Maximum visibility timeout for SQS is 12h, also can't be extended by the consumer. "If your consumer needs longer than 12 hours, consider using Step Functions." https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html

C is the only option with dead letter que which meets the requirement of retaining messages that fail to process without impacting other messages.

Implement an AWS service to handle messages between the two applications = Amazon Simple Queue Service If the messages fail to process, they must be retained = a dead-letter queue

SQS provides a fully managed message queuing service that meets all the requirements: SQS can handle the sending and processing of 1,000 messages per hour Messages can be retained for up to 14 days to allow the full 2 days for processing Using a dead-letter queue will retain failed messages without impacting other processing SQS requires minimal operational overhead compared to running your own message queue server

Answer is B), the reason is: - Because messages might up to 2 days to be processed. Visibility timeout of SQS is 12 hours, so after 12 hours another consumer might take a message from the queue which is currently being processed.

Answer C, In Question if Keyword have Processing Failed >> SQS

solution that meets these requirements and is the MOST operationally efficient will be option C. SQS is buffer between 2 APPs.

The visibility timeout must not be more than 12 hours. ( For SQS ) Jobs may take 2 days to process

operationally efficient = Serverless SQS is serverless

more realistic option is C. only problem with this is the limit of the visibility timeout is 12H max. as the second application take 2 days to process, there will be a duplicate of processing messages in the queue. this might complicate things.

SQS has a limit 12h for visibility time out

Option C, using Amazon SQS, is a valid solution that meets the requirements of the company. However, it may not be the most operationally efficient solution because SQS is a managed message queue service that requires additional operational overhead to handle the retention of messages that failed to process. Option B, using Amazon Kinesis Data Streams, is more operationally efficient for this use case because it can handle the retention of messages that failed to process automatically and provides the ability to process and analyze streaming data in real-time.