Exam DP-600 topic 1 question 53 discussion

Actual exam question from Microsoft's DP-600

Question #: 53
Topic #: 1

HOTSPOT -
You have a Fabric tenant that contains a lakehouse named Lakehouse1. Lakehouse1 contains a table named Nyctaxi_raw. Nyctaxi_row contains the following table:

You create a Fabric notebook and attach it to Lakehouse1.
You need to use PySpark code to transform the data. The solution must meet the following requirements:
Add a column named pickupDate that will contain only the date portion of pickupDateTime.
Filter the DataFrame to include only rows where fareAmount is a positive number that is less than 100.
How should you complete the code? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.

Show Suggested Answer

Suggested Answer:

by fabric1 at Feb. 13, 2024, 12:52 p.m.

Comments

Submit Cancel

XiltroX

Highly Voted 1 year, 3 months ago

The correct answers: 1. withColumn 2. cast('date') 3. .filter('fareAmount.......

upvoted 44 times

...

Momoanwar

Highly Voted 1 year, 4 months ago

We need to add column not rename existing column. Here is the correct answer : df.withColumn('pickupDate', df['pickupDateTime'].cast(DateType())) \ .filter("fareAmount > 0 AND fareAmount < 100")

upvoted 12 times

...

c119533

Most Recent 10 months, 1 week ago

Tested answer df = spark.read.format("delta").load("Tables/factinternetsales") df2 = df.withColumn("pickupdate", df['pickupDateTime'].cast('date')).filter("fareAmount > 0 AND fareAmount < 100") df2.show()

upvoted 2 times

...

woliveiras

1 year ago

The correct answers: 1. withColumn 2. cast('date') 3. .filter('fareAmount Tested!!!

upvoted 8 times

...

72bd3bc

1 year ago

why the expression with where is not correct? Is it because it includes 100? not less than 100?

upvoted 1 times

...

PiyushT

1 year, 1 month ago

.filter(fareamount >0 and <100) does not work, I tried the code myself. You have to use same condition but with "col" like this .filter(col(fareamount) > 0 and col(fareamount) < 100)

upvoted 2 times

...

stilferx

1 year, 1 month ago

IMHO, withColumn -> cast(dateType()) -> filter("fareAmount > 0 AND fareAmount < 100") is correct. As colleagues said, cast(dateType()), not cast('date'). Let's consider it a typo from Microsoft.

upvoted 6 times

...

earlqq

1 year, 3 months ago

A bit incorrect. .withcolumn .cast(date) .filter(fareamount >0 and <100)

upvoted 8 times

...