Exam Professional Data Engineer topic 1 question 202 discussion

Therefore, the correct option is C. Use Transfer Service for on-premises data to copy data from your on-premises environment to Cloud Storage. Use the BigQuery Data Transfer Service to import data into BigQuery. Option A is incorrect because Cloud Scheduler is not designed for data transfer, but rather for scheduling the execution of Cloud Functions, Cloud Run, or App Engine applications. Option B is incorrect because Transfer Appliance is designed for large-scale data transfers from on-premises environments to Google Cloud and is not suitable for transferring data on a daily basis. Option D is also incorrect because the BigQuery Data Transfer Service dataset copy feature is designed for copying datasets between BigQuery projects and not suitable for copying data from on-premises environments to BigQuery.

"Your on-premises environment cannot be accessed from the public internet" statement suggests that inbound traffic from internet is NOT allowed however, it doesn't mean that outbound internet connectivity from on-prem resources is not possible. Any on-prem system with outbound internet access can copy/transfer the CSV files. CSV files are located on a filesystem, therefore you cannot copy them with BQ Transfer Service. Leaving only possible option; first copy CSVs to cloud storage then run BQ Transfer Service pls refer to https://cloud.google.com/bigquery/docs/dts-introduction#supported_data_sources

I'm torn on this question. Okay no access from public internet... does that mean they don't have private lines (e.g. Ded/Partner interconnects)? Poorly worded. IMO it can either be: Answer B or C based on interpretation of Public Internet.

I vote B, because in "Your on-premises environment cannot be accessed from the public internet.", it would only allow data to be extracted internally within the company. So Transfer Appliance is the most appropriate tool.

Transfer Service for on-premises data is designed specifically for this scenario. It uses a private, secure agent-based approach to move data from on-premises environments to Google Cloud Storage.

They don't define "cannot be accessed from the public internet" - does this mean no incoming traffic, or no traffic or any kind regardless of the initiation point? We simply do not know, and so are left guessing. C? Probably, but could be B, just depending.

the correct option is C

I would say B. It is the ONLY option that is possible without data being accessible over the public (unless we assume that a direct interconnect is already set up, which seems farfetched). Also, nowhere does it say how up-to-date the data needs to be that we are querying or how often we need to query, only that the data increases in size by 100gb per day (indicating that its going to be a lot of data)

Answer C, What is wrong with B ? Key words = Daily transfer .. so no to transfer appliance,

C is the answer. https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#storage-transfer-service-for-large-transfers-of-on-premises-data Storage Transfer Service for on-premises data enables transfers from network file system (NFS) storage to Cloud Storage. https://cloud.google.com/bigquery/docs/cloud-storage-transfer-overview The BigQuery Data Transfer Service for Cloud Storage lets you schedule recurring data loads from Cloud Storage buckets to BigQuery.

C D-no answer because bq transfer service don't support from on-prem

Transfer Service for on-premises is optimal for on-premises google ( large files (< 1 TB) and bandwidth available and scheduling) https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#transfer-options https://cloud.google.com/blog/products/storage-data-transfer/introducing-storage-transfer-service-for-on-premises-data BigQuery Data Transfer Service is good for gcs to bigquery https://cloud.google.com/bigquery/docs/cloud-storage-transfer

I will go with C

I will g with C https://cloud.google.com/architecture/migration-to-google-cloud-transferring-your-large-datasets#transfer-options

C is correct, b is suitable for weekly . https://cloud.google.com/transfer-appliance/docs/4.0/overview

Ans C https://cloud.google.com/storage-transfer/docs/on-prem-agent-best-practices

I would go with option C. You need a service to transfer data from on-premises to cloud storage. so "Transfer service" is the best option & additionally you can easily configure the network so that data flows through private network. cloud scheduler on other hand is used mostly for automation. You can schedule a service but in my view cannot be used solo to transfer data.