Exam Professional Data Engineer topic 1 question 158 discussion

Correct: C If you create a Dataproc cluster with internal IP addresses only, attempts to access the Internet in an initialization action will fail unless you have configured routes to direct the traffic through a NAT or a VPN gateway. Without access to the Internet, you can enable Private Google Access, and place job dependencies in Cloud Storage; cluster nodes can download the dependencies from Cloud Storage from internal IPs.

Should be C: https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/init-actions

A: Incorrect, Proxy allows for connection to a Cloud SQL instance, which unless you have the dependencies stored there (doesn't seem viable or smart), would achieve nothing. B: Incorrect, will allow for a connection to the internet to be made for installing the dependencies, however this goes against the companies security policies so should not be considered. C: Correct, only one that makes sense and is best practise (see https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/init-actions) D: Incorrect, provides access to a shared VPC network, however doesn't necessarily provide a way to access the dependencies. And even if it did, would go against company security policy.

c looks good

Security Compliance: This option aligns with your company's security policies, which prohibit public Internet access from Cloud Dataproc nodes. Placing the dependencies in a Cloud Storage bucket within your VPC security perimeter ensures that the data remains within your private network. VPC Security: By placing the dependencies within your VPC security perimeter, you maintain control over network access and can restrict access to the necessary nodes only. Dataproc Initialization Action: You can use a custom initialization action or script to fetch and install the dependencies from the secure Cloud Storage bucket to the Dataproc cluster nodes during startup. By copying the dependencies to a secure Cloud Storage bucket and using an initialization action to install them on the Dataproc nodes, you can meet your security requirements while providing the necessary dependencies to your cluster.

C is correct

C seems good

Answer C, It needs practical experience to understand this question. You create cluster with some package/software i.e dependencies such as python packages that you store in .zip file, then you save a jar file to run the cluster as an application such as you need java while running spark session. and some config yaml file. These dependencies you can save in bucket and can use to configure cluster from external window , sdk or api. without going into UI. Then you need to use VPC to access these files

C is the answer. https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/network#and_vpc-sc_networks With VPC Service Controls, administrators can define a security perimeter around resources of Google-managed services to control communication to and between those services.

Without access to the internet, you can enable Private Google Access and place job dependencies in Cloud Storage; cluster nodes can download the dependencies from Cloud Storage from internal IPs.

https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/network#create_a_cloud_dataproc_cluster_with_internal_ip_address_only

When creating a Dataproc cluster, you can specify initialization actions in executables or scripts that Dataproc will run on all nodes in your Dataproc cluster immediately after the cluster is set up. Initialization actions often set up job dependencies, such as installing Python packages, so that jobs can be submitted to the cluster without having to install dependencies when the jobs are run

Correct: C

c it is!

Should be C

Should be C

I think the correct answer might be C instead, due to https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/network#create_a_cloud_dataproc_cluster_with_internal_ip_address_only