Exam Certified Associate Developer for Apache Spark topic 1 question 43 discussion

Actual exam question from Databricks's Certified Associate Developer for Apache Spark

Question #: 43
Topic #: 1

[All Certified Associate Developer for Apache Spark Questions]

The code block shown below contains an error. The code block is intended to use SQL to return a new DataFrame containing column storeId and column managerName from a table created from DataFrame storesDF. Identify the error.
Code block:
storesDF.createOrReplaceTempView("stores")
storesDF.sql("SELECT storeId, managerName FROM stores")

A. The createOrReplaceTempView() operation does not make a Dataframe accessible via SQL.
B. The sql() operation should be accessed via the spark variable rather than DataFrame storesDF.
C. There is the sql() operation in DataFrame storesDF. The operation query() should be used instead.
D. This cannot be accomplished using SQL – the DataFrame API should be used instead.
E. The createOrReplaceTempView() operation should be accessed via the spark variable rather than DataFrame storesDF.

Show Suggested Answer

Suggested Answer: B 🗳️

by 4be8126 at May 1, 2023, 3:17 p.m.

Comments

Submit Cancel

jds0

11 months, 2 weeks ago

Selected Answer: B

B is correct. 'storeDF' has not attribute or method `sql` Test code below: from pyspark.sql import SparkSession spark = SparkSession.builder.appName("MyApp").getOrCreate() data = [ (0, 3, "John"), (1, 1, "Jane"), (2, 2, "Jack"), ] storesDF = spark.createDataFrame(data, ["storeID", "customerSatisfaction", "managerName"]) storesDF.createOrReplaceTempView("stores") try: storesDF.sql("SELECT storeId, managerName FROM stores") except AttributeError as e: print(e) finally: spark.sql("SELECT storeId, managerName FROM stores").show()

upvoted 2 times

...

juliom6

1 year, 8 months ago

Selected Answer: B

B is correct: storesDF = spark.createDataFrame([('1', 'juan'), ('2', 'perez')], ['storeId', 'managerName']) storesDF.createOrReplaceTempView("stores") spark.sql("SELECT storeId, managerName FROM stores").show()

upvoted 2 times

...

4be8126

2 years, 2 months ago

Selected Answer: B

Option B is correct because the sql() function is not a method of a DataFrame object. It is actually a method of the SparkSession object spark. Therefore, the correct way to execute a SQL statement using Spark SQL is to call sql() on the SparkSession object as follows: spark.sql("SELECT storeId, managerName FROM stores") In the code block provided in the question, sql() is called on a DataFrame object, which will result in a DataFrame object without executing the SQL statement. Therefore, option B correctly identifies the error in the code block.

upvoted 2 times

...