Exam SnowPro Advanced Data Engineer topic 1 question 38 discussion

Actual exam question from Snowflake's SnowPro Advanced Data Engineer

Question #: 38
Topic #: 1

[All SnowPro Advanced Data Engineer Questions]

A Data Engineer is working on a continuous data pipeline which receives data from Amazon Kinesis Firehose and loads the data into a staging table which will later be used in the data transformation process. The average file size is 300-500 MB.
The Engineer needs to ensure that Snowpipe is performant while minimizing costs.
How can this be achieved?

A. Increase the size of the virtual warehouse used by Snowpipe.
B. Split the files before loading them and set the SIZE_LIMIT option to 250 MB.
C. Change the file compression size and increase the frequency of the Snowpipe loads.
D. Decrease the buffer size to trigger delivery of files sized between 100 to 250 MB in Kinesis Firehose.

Show Suggested Answer

Suggested Answer: D 🗳️

by stopthisnow at Nov. 12, 2023, 11:18 p.m.

Comments

Submit Cancel

charemski

1 month, 2 weeks ago

Selected Answer: D

SIZE_LIMIT is not supported in Snowpipe https://docs.snowflake.com/en/sql-reference/sql/create-pipe

upvoted 1 times

...

DIPARJ

4 months, 3 weeks ago

Selected Answer: B

It should be option B

upvoted 1 times

...

Snow_P

9 months ago

Selected Answer: D

https://docs.snowflake.com/en/user-guide/data-load-considerations-prepare

upvoted 1 times

...

stopthisnow

9 months, 1 week ago

Selected Answer: D

Various tools can aggregate and batch data files. One convenient option is Amazon Kinesis Firehose. Firehose allows defining both the desired file size, called the buffer size, and the wait interval after which a new file is sent (to cloud storage in this case), called the buffer interval. For more information, see the Kinesis Firehose documentation. If your source application typically accumulates enough data within a minute to populate files larger than the recommended maximum for optimal parallel processing, you could decrease the buffer size to trigger delivery of smaller files. Keeping the buffer interval setting at 60 seconds (the minimum value) helps avoid creating too many files or increasing latency.

upvoted 1 times

...