Exam AWS Certified Data Analytics - Specialty topic 1 question 122 discussion

Whenever the requirement states it is Operational Analytics, I consider using ES. Also, here QuickSight cannot refresh data for every 5 secs automatically, which in case of ELK is possible. So, Answer is C.

A,C is wrong because "You can configure the values for OpenSearch buffer size (1 MB to 100 MB) or buffer interval (60 to 900 seconds)", the requirement is "5 seconds" to refresh data. https://aws.amazon.com/tw/kinesis/data-firehose/faqs/ B is wrong since EMR requires more effort. hence D.

A,C is wrong because "You can configure the values for OpenSearch buffer size (1 MB to 100 MB) or buffer interval (60 to 900 seconds)", the requirement is "5 seconds" to refresh data. https://aws.amazon.com/tw/kinesis/data-firehose/faqs/ B is wrong since EMR requires more effort. hence D.

C: I passed the test

The solution that meets the requirements and requires the least possible implementation effort is option C: Use Amazon Kinesis Data Firehose to push the data into an Amazon OpenSearch Service (Amazon Elasticsearch Service) cluster. Visualize the data by using an OpenSearch Dashboards (Kibana). Amazon Kinesis Data Firehose can be used to stream data to an Amazon Elasticsearch Service cluster with minimal configuration. Amazon OpenSearch Service is an open-source, distributed search and analytics engine that is fully compatible with Elasticsearch. OpenSearch Dashboards (Kibana) can be used to visualize and explore the data in near-real time, with automatic refreshing every 5 seconds. This solution eliminates the need for a custom application or custom coding, and it requires minimal configuration and maintenance effort. It also provides a scalable, fully managed, and cost-effective way to store, analyze, and visualize streaming data.

What is the need of refreshing the dashboard for every 5seconds if Firehose can deliver for every 60 seconds? Go with D(don't use quicksight spice, query the data directly from quicksight) or go with B(but more dev effort)

Answer is C. Simple to deploy.

It is C. Simple to deploy = not a custom Java application. QuickSight can't be refreshed fast enough.

Answer B is correct

It should be C. There is no need to use JavaScript libraries to generate a dashboard as the end user is not public using a webpage but data analysts within the organization. Firehose, ElasticSearch and Kibana are able to handle this requirement very well. And there is no custom code development required either.

In the requirement, it want to refresh the dashboard every 5 seconds, but not need to process data in 5 seconds.

I vote for B. For C, it use kinesis anadata firehose. But the data firehose has a buffer with a minimum 60 seconds or 1 MB. We are not sure about the message size and frequence, so the deliver interval maybe much more than 5 seconds(it is required in the question). So B will not match the requirement. For B, the spark in a EMR can process data in realtime. It will develop application using d3.js for visualization. But we still need a storage for the realtime data, which is output of spark stream, and used by d3.js to get and display. Although C isn't clear enough and needs much efford, but it match the requirement.

QuickSight can only be updated every 15 minutes. So the answer is C.

B and C are close. As simple to deploy rules out D3.js as it's not simple and quicksight is not meant for real-time (or near real-time) refresh! hence answer is C

Kinesis firehose can write to destination RES R - REdshift , E - elastic search , S - S3 . when it can write directly to elastic search why write to S3 and then connect to elastic search. So the answer is C directly write to elastic search and use kibana to refresh every 5 secs and reporting display.

I think it's B because Quicksight can't refresh data every 5sec and Firehose has a minila batch size of 60sec so the only remaining option is B even if it requires custom code

B and C are close. But B is not simple to deploy. so C.