Question 1

Given an instance of SparkSession named spark, reviewing the following code what's the output ?1. from pyspark.sql.types import IntegerType2. from pyspark.sql.functions import col3. import pyspark.sql.functions as f4.5.6. a = [1002, 3001, 4002, 2003, 2002, 3004, 1003, 4006]7. b = spark8. .createDataFrame(a, IntegerType())9. .withColumn("x", col("value") 00)10.11. c = b12. .groupBy(col("x"))13. .agg(f.count("x"), f.sum("value"))14. .drop("x")15. .toDF("count", "total")16. .orderBy(col("count").desc(), col("total"))17. .limit(1)18. .show()

Options :

A : 1. |count|total|2. | 2| 8008|

B : 1. |count|total|2. | 3| 7006|

C : 1. |count|total|2. | 8|20023|

D : 1. |count|total|2. | 1| 3001|

Answer: B

Question 2

When the property spark.sql.optimizer.dynamicPartitionPruning.enabled is set to true, what optimization happens in spark ?

Options :

A : It allows you to dynamically switch join strategies.

B : Spark dynamically handles skew in sort-merge join by splitting (and replicating if needed) skewed partitions with this property enabled.

C : You to read only as much data as you need.

D : It allows you to dynamically select physical plans based on cost.

Answer: C

Question 3

Which of the following code blocks reads all CSV files in directory filePath into a single DataFrame, withcolumn names defined in the CSV file headers?Content of directory filePath:1._SUCCESS2._committed_27545464516997471243._started_27545464516997471244.part-00000-tid-2754546451699747124-10eb85bf-8d91-4dd0-b60b-2f3c02eeecaa-298-1-c000.csv.gz5.part-00001-tid-2754546451699747124-10eb85bf-8d91-4dd0-b60b-2f3c02eeecaa-299-1-c000.csv.gz6.part-00002-tid-2754546451699747124-10eb85bf-8d91-4dd0-b60b-2f3c02eeecaa-300-1-c000.csv.gz7.part-00003-tid-2754546451699747124-10eb85bf-8d91-4dd0-b60b-2f3c02eeecaa-301-1-c000.csv.gzspark.option("header",True).csv(filePath)

Options :

A : spark.read.format("csv").option("header",True).option("compression","zip").load(filePath)

B : spark.read().option("header",True).load(filePath)

C : spark.read.format("csv").option("header",True).load(filePath)

D : spark.read.load(filePath)

Answer: C

Question 4

When joining two dataframes, if there is a need to evaluate the keys in both of the DataFrames or tables and include all rows from the right DataFrame as well as any rows in the left DataFrame that have a match in the righ DataFrame also If there is no equivalent row in the left DataFrame, we want to instert null: which join type should we select ? df1.join(person, joinExpression, joinType)

Options :

A : val joinType = “rightAnti”

B : val joinType = “rightSemi”

C : val joinType = “rightOuter”

D : val joinType = “right_outer”

Answer: D

Question 5

Which of the following is a viable way to improve Spark's performance when dealing with large amounts of data, given that there is only a single application running on the cluster?

Options :

A : Increase values for the properties spark.default.parallelism and spark.sql.shuffle.partitions

B : Decrease values for the properties spark.default.parallelism and spark.sql.partitions

C : Increase values for the properties spark.sql.parallelism and spark.sql.partitions

D : Increase values for the properties spark.sql.parallelism and spark.sql.shuffle.partitions

E : Increase values for the properties spark.dynamicAllocation.maxExecutors, spark.default.parallelism, and spark.sql.shuffle.partitions

Answer: A

Confidently Practice Online with Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Exam Cram

Buying Options: