Exam Professional Data Engineer topic 1 question 19 discussion

Ans: A B: Wrong eVM wont solve the problem of larger storage prices. C: May be, but nothing mentioned in terms of what to tune in the question, also this is like-for-like migration so tuning may not be part of the migration. D: Again, this is like-for-like so need to define what is hot data and which is cold data, also persistent disk costlier than cloud storage.

A like-for-like migration to Cloud Dataproc that replicates on-premises Hadoop would require each node to have 50 TB of persistent disk, which is costly. Instead, you can minimize storage costs by leveraging Google Cloud Storage (GCS). Cloud Dataproc seamlessly integrates with GCS through the Hadoop connector, allowing you to store your data cost-effectively in Cloud Storage and run ephemeral clusters that read data directly from GCS. This approach eliminates the need for each node to carry 50 TB of expensive persistent disk storage while still supporting your Hadoop workload.

. Migrate some of the cold data into Google Cloud Storage, and keep only the hot data in Persistent Disk. Google Cloud Storage (GCS) is a cost-effective alternative to Persistent Disk for storing less frequently accessed ("cold") data. Hot data that requires fast access can remain on Persistent Disk, reducing storage costs while maintaining performance. Cloud Dataproc supports HDFS-to-GCS integration, allowing Hadoop jobs to access data in GCS seamlessly.D

You are most of the people looking at like for like migration would require 50TB persistent storage but missing to look at CIO concern about cost of block storage...considering CIO concern the option here is cloud storage... moreover that is recommended as well ..

Option A: Put the data into Google Cloud Storage. This is the best option. Google Cloud Dataproc is designed to work well with Google Cloud Storage. Using GCS instead of Persistent Disk can save money, and GCS offers advantages such as higher durability and the ability to share data across multiple clusters.

I have seen this question in other places and I believe that you store the older data in Cloud Storage and retain processing data in persistent disk. D

Answer: D Question: A like-for- like migration of the cluster would require 50 TB of Google Persistent Disk per node. which means Persistent is still required.

the question is talking about block storage , GCS is object storage !

GCS is cost-effective and also Google's recommendation!

Minimize cost then GCS

A - is the right answer.

A - dataproc - storage - cost effective is cloud storage

Cloud Storage

Cloud Storage is google recommended one

Vote for 'A"

A, Moving the data to GCS will reduce the cost of running the dataproc clusters all the time \

Correct A