Exam Professional Data Engineer topic 1 question 29 discussion

Description: Best practices of bigtable states that rowkey should not be only timestamp or have timestamp at starting. It’s better to have sensorid and timestamp as rowkey

Answer D

D is correct

Looks like D is the best option Reference: https://cloud.google.com/bigtable/docs/schema-design#time-based

D. Use a row key of the form <sensorid>#<timestamp>. By using the sensor ID as the prefix in the row key, you can achieve better distribution of data across Bigtable tablets. This can help balance the workload and prevent hotspots in the table. Additionally, placing the timestamp after the sensor ID allows you to perform range scans for a specific sensor and retrieve data efficiently within a time frame. Option C (using a row key of the form <timestamp>#<sensorid>) can work for some use cases but may not be as efficient for range scans when you want to retrieve data for a specific sensor within a time range. Option A (using a row key of the form <timestamp>) may lead to hotspots and inefficient range scans because it doesn't consider sensor IDs. Option B (using a row key of the form <sensorid>) is not optimal because it doesn't allow for efficient time-based filtering and could lead to uneven data distribution in Bigtable.

D is right Best practices of bigtable states that rowkey should not be only timestamp or have timestamp at starting. It’s better to have sensorid and timestamp as rowkey. Reference: https://cloud.google.com/bigtable/docs/schema-design

#<sensorid>#<timestamp> ------> low cardinality # high cardinality This is current Bigtable Best Practice (to avoid Hotspots on the inserts)

Discard: A -> timestamp unique id could not be unique in the case that sensors transmit data at the same time. B -> sensorId repeated id for messages coming from the same sensor C -> a bad performance choice D -> BEST CHOICE. Each time BigTable looks for data in a table it does a scan and sort operations. By starting each unique id by sensorId it will make it easier to group and sort data since it has the lowest cardinality https://cloud.google.com/bigtable/docs/schema-design#general-concepts

as I look at https://cloud.google.com/bigtable/docs/schema-design#row-keys asia#india#bangalore asia#india#mumbai they didn't have # ahead of this first value. asia#india#bangalore OR #asia#india#bangalore Are both valid?

ANSWER: D

Answer is D

A. Use a row key of the form <timestamp>. ---> Incorrect, because google says don't use a timestamp by itself or at the beginning of a row key. B. Use a row key of the form <sensorid>. --->Incorrect, because google says Include a timestamp as part of your row key. C. Use a row key of the form <timestamp>#<sensorid>. ---> Incorrect, because google says don't use a timestamp by itself or at the beginning of a row key. D. Use a row key of the form >#<sensorid>#<timestamp>. ---> Correct answer, because of option A,B,C reasons. - Timestamp isn't by itself, neither at the beginning. - Timestamp is included. Reference: https://cloud.google.com/bigtable/docs/schema-design#row-keys

Ans: D

Vote for 'D' - Store multiple delimited values in each row key. (But avoid starting with Timestamp) "Row keys to avoid" https://cloud.google.com/bigtable/docs/schema-design

Correct D

Should be D Reverse of timestamp even better but no options for that. Also changing sensor ID if they are in sequential to hash or changing data to bits even better. Idea is not to use timestamp or sequential ID as first key.

The correct answer is D. Refer to the link https://cloud.google.com/bigtable/docs/schema-design for Big table schema design. C is not the right answer becuase Timestamps If you often need to retrieve data based on the time when it was recorded, it's a good idea to include a timestamp as part of your row key. Using the timestamp by itself as the row key is not recommended, as most writes would be pushed onto a single node. For the same reason, avoid placing a timestamp at the start of the row key. For example, your application might need to record performance-related data, such as CPU and memory usage, once per second for a large number of machines. Your row key for this data could combine an identifier for the machine with a timestamp for the data (for example, machine_4223421#1425330757685).