Exam Professional Machine Learning Engineer topic 1 question 189 discussion

The choice is between B and D, both good BUT: Importing and making batch predictions is quite straightforward in BQ ML https://cloud.google.com/bigquery/docs/making-predictions-with-imported-tensorflow-models if not pre-processing needed on the data. If we need a more complete pipeline I'd chose D, but the tables need partitioning (100GB is the limit in Vertex AI): https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions#input_data_requirements

Managed Service: Vertex AI batch prediction

A) - BigQuery ML is not designed for the scale of a 10TB dataset - Batch Prediction performs efficient batch inference on large GCS datasets - AVRO is a binary format, more compact and efficient to process than CSV *B uses BQML; C uses CSV format; exporting to GCS is more efficient than performing Vertex AI predictions directly on BQ for this volumetry.

It should be A. The "CREATE MODEL" statement in BigQuery ML is meant for BigQuery-specific models, and do not support models like TensorFlow SavedModel out of the box. This option would not work for using a TensorFlow model stored in Cloud Storage.

My answer is D.

https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions#input_data_requirements

My Answer: D The historical dataset is stored in BigQuery, which can be directly accessed by Vertex AI. Vertex AI offers batch prediction capabilities, allowing you to apply the model to the data stored in BigQuery without the need to export it. So, This approach leverages the scalability of Google Cloud infrastructure and avoids unnecessary data movement, being not necessary to export data to Cloud Store (options A and C), nor Import the TensorFlow model to BQ (option B).

https://cloud.google.com/bigquery/docs/making-predictions-with-imported-tensorflow-models#:~:text=Import%20TensorFlow%20models,-To%20import%20TensorFlow&text=In%20the%20Google%20Cloud%20console%2C%20go%20to%20the%20BigQuery%20page.&text=In%20the%20query%20editor%2C%20enter,MODEL%20statement%20like%20the%20following.&text=The%20preceding%20query%20imports%20a,BigQuery%20ML%20model%20named%20imported_tf_model%20.

https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-tensorflow#limitations

Has to be B, because D has limitations: BigQuery data source tables must be no larger than 100 GB. https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions#input_data_requirements

Same platform as data == less computation required to load and pass it to model

It could either be B or D. It seems like most of the limitations of B are mentioned in the problem (https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-tensorflow#limitations) but some of them are not and we are left questioning if the model will match the remaining requirements. Therefore, I would go for D, which can import data from BigQuery. https://cloud.google.com/vertex-ai/docs/predictions/get-batch-predictions#bigquery

Limitations of other options: A and C. Exporting data: Exporting 10 TB of data to Cloud Storage incurs additional storage costs, transfer time, and potential data management complexities. B. BigQuery ML: While BigQuery ML supports some TensorFlow models, it might have limitations with certain model architectures or features. Additionally, it might not be as optimized for large-scale batch inference as Vertex AI.