Exam AWS Certified Data Analytics - Specialty topic 1 question 8 discussion

IThink its C. Refer the below link https://aws.amazon.com/premiumsupport/knowledge-center/high-jvm-memory-pressure-elasticsearch/

I agree with Option C: Unbalanced shard allocations across nodes or too many shards in a cluster can cause JVMMemoryPressue. Resolution - Reduce the number of shards by deleting old or unused indices. https://aws.amazon.com/premiumsupport/knowledge-center/high-jvm-memory-pressure-elasticsearch/

its a C

C: I passed the test

C. Decrease the number of Amazon ES shards for the index. To improve the performance of Amazon ES in this scenario, the number of shards for the index should be decreased. Currently, the index has 1,000 shards, which is likely causing high overhead and slowing down query performance. In general, it's recommended to have 20-30 GB of data per shard for efficient indexing and query performance in Amazon ES. However, having too many shards can lead to inefficient resource utilization and slow query performance. Additionally, since the cluster is configured with 3 dedicated master nodes, increasing the memory of the master nodes may not have a significant impact on performance. Decreasing the number of data nodes may also not be an effective solution, as this could reduce the capacity of the cluster to handle the 1 TB of daily data ingestion. Increasing the number of shards for the index would further exacerbate the performance issues, as more shards would lead to more overhead and slower query performance.

According to this link https://aws.amazon.com/premiumsupport/knowledge-center/opensearch-high-jvm-memory-pressure/ It clearly states that this issue can be caused due to "Unbalanced shard allocations across nodes or too many shards in a cluster."

Correct answer is C as one of the causes of JVMMemoryPressure error can be Unbalanced shard allocations across nodes or too many shards in a cluster. and can be resolved by reducing the number of shards by deleting old or unused indices. Option A is wrong because dedicated master nodes are only used to increase cluster stability. Therefore, this option won't help you improve the performance of the cluster. Option B is incorrect because these nodes carry all the data in your indexes (storage) and do all the processing for your requests (CPU). If you decrease the number of data nodes, the performance of the cluster still won't improve. Option D is incorrect. The JVMMemoryPressure error signifies that there is an unbalanced shard allocations across nodes. This means that there are too many shards in the Amazon ES cluster and not the other way around. To improve the performance of the cluster, you must decrease the number of shards.

Selected Answer: C

Answer is C.

Ans - C I got this question for my certification I gave on Dec 9th 2021.

C is the right answer

Answer: C. Decrease the number of Amazon ES shards for the index. Memory pressure in the JVM can result if: You have unbalanced shard allocations across nodes You have too many shards in a cluster Fewer shards can yield better performance if JVMMemoryPressure errors are encountered Delete old or unused indices

C is correct. from documentation -- https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/sizing-domains.html Here the shard size is 1.5*1000 (GB)/1000 (number of shards)= 1.5 GB which is much less than recommended size of shards. The overarching goal of choosing a number of shards is to distribute an index evenly across all data nodes in the cluster. However, these shards shouldn't be too large or too numerous. A good rule of thumb is to try to keep shard size between 10–50 GiB. Large shards can make it difficult for Elasticsearch to recover from failure, but because each shard uses some amount of CPU and memory, having too many small shards can cause performance issues and out of memory errors. In other words, shards should be small enough that the underlying Amazon ES instance can handle them, but not so small that they place needless strain on the hardware.

C is the right answer

The question says the domain is running one index, if so how can we assume there are old unused indices. shouldn't we reindex to adjust the number of shards?

C is correct

From link https://aws.amazon.com/premiumsupport/knowledge-center/high-jvm-memory-pressure-elasticsearch/ You can resolve high JVM memory pressure issues by reducing traffic to the cluster. To reduce traffic to the cluster, follow these best practices: Reduce the number of shards by deleting old or unused indices.