Exam Certified Associate Developer for Apache Spark topic 1 question 50 discussion

Actual exam question from Databricks's Certified Associate Developer for Apache Spark

Question #: 50
Topic #: 1

[All Certified Associate Developer for Apache Spark Questions]

Which of the following code blocks returns a DataFrame containing a column dayOfYear, an integer representation of the day of the year from column openDate from DataFrame storesDF?
Note that column openDate is of type integer and represents a date in the UNIX epoch format – the number of seconds since midnight on January 1st, 1970.
A sample of storesDF is displayed below:

A. (storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp"))
. withColumn("dayOfYear", dayofyear(col("openTimestamp"))))
B. storesDF.withColumn("dayOfYear", get dayofyear(col("openDate")))
C. storesDF.withColumn("dayOfYear", dayofyear(col("openDate")))
D. (storesDF.withColumn("openDateFormat", col("openDate").cast("Date"))
. withColumn("dayOfYear", dayofyear(col("openDateFormat"))))
E. storesDF.withColumn("dayOfYear", substr(col("openDate"), 4, 6))

Show Suggested Answer

Suggested Answer: A 🗳️

by peekaboo15 at April 13, 2023, 9:30 p.m.

Comments

Submit Cancel

juliom6

8 months, 3 weeks ago

Selected Answer: A

A is correct: from pyspark.sql.functions import col, dayofyear storesDF = spark.createDataFrame([(0, 1100746394), (1, 1474410343)], ['storeId', 'openDate']) storesDF = (storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp")).withColumn("dayOfYear", dayofyear(col("openTimestamp")))) display(storesDF)

upvoted 1 times

...

newusername

9 months ago

Selected Answer: A

A is correct

upvoted 1 times

...

thanab

10 months, 3 weeks ago

Selected Answer: A

storesDF.withColumn("openTimestamp", col("openDate").cast("Timestamp")).withColumn("dayOfYear", dayofyear(col("openTimestamp")))

upvoted 2 times

...

singh100

1 year ago

A. dayofyear function in PySpark's functions module expects the column openDate to be of type timestamp rather than long.

upvoted 1 times

...

4be8126

1 year, 3 months ago

Selected Answer: A

The correct answer is C. Option A is correct because it casts the openDate column to a timestamp using cast("Timestamp") and then uses the dayofyear function to extract the day of the year from the timestamp. Option B is incorrect because it contains syntax errors, including the "get" keyword, which is not necessary or valid in this context. Option C is close, but it does not cast the openDate column to a timestamp, which is necessary to use the dayofyear function. Option D is incorrect because it converts column "openDate" to a date format, which is unnecessary for extracting the day of the year. Additionally, the dayofyear() function can be applied directly to the "openDate" column. Option E is incorrect because it uses the substr() function to extract a substring from the "openDate" column, which does not correspond to the day of the year.

upvoted 1 times

...

peekaboo15

1 year, 3 months ago

The answer should be A. Unixtime should be cast to timestamp first

upvoted 2 times

...