Question 1

A Spark application needs to read multiple Parquet files from a directory where the files have differing but compatible schemas. The data engineer wants to create a DataFrame that includes all columns from all files. Which code should the data engineer use to read the Parquet files and include all columns using Apache Spark?

Options :

A : spark.read.parquet("/data/parquet/")

B : spark.read.option("mergeSchema", True).parquet("/data/parquet/")

C : spark.read.format("parquet").option("inferSchema", "true").load("/data/parquet/")

D : spark.read.parquet("/data/parquet/").option("mergeAllCols", True)

Answer: B

Question 2

A Spark application suffers from too many small tasks due to excessive partitioning. How can this be fixed without a full shuffle?

Options :

A : Use the distinct() transformation to combine similar partitions

B : Use the coalesce() transformation with a lower number of partitions

C : Use the sortBy() transformation to reorganize the data

D : Use the repartition() transformation with a lower number of partitions

Answer: B

Question 3

A Spark developer is developing a Spark application to monitor task performance across a cluster. One requirement is to track the maximum processing time for tasks on each worker node and consolidate this information on the driver for further analysis. Which technique should the developer use?

Options :

A : Broadcast a variable to share the maximum time among workers.

B : Configure the Spark UI to automatically collect maximum times.

C : Use an RDD action like reduce() to compute the maximum time.

D : Use an accumulator to record the maximum time on the driver.

Answer: C

Question 4

A data engineer observes that an upstream streaming source sends duplicate records, where duplicates share the same key and have at most a 30-minute difference inevent_timestamp. The engineer adds: dropDuplicatesWithinWatermark("event_timestamp", "30 minutes") What is the result?

Options :

A : It is not able to handle deduplication in this scenario

B : It removes duplicates that arrive within the 30-minute window specified by the watermark

C : It removes all duplicates regardless of when they arrive

D : It accepts watermarks in seconds and the code results in an error

Answer: B

Question 5

A developer initializes a SparkSession:

spark = SparkSession.builder \ .appName("Analytics Application") \ .getOrCreate() Which statement describes thesparkSparkSession?ThegetOrCreate()method explicitly destroys any existing SparkSession and creates a new one.

Options :

A : ThegetOrCreate()method explicitly destroys any existing SparkSession and creates a new one.

B :

A SparkSession is unique for eachappName, and callinggetOrCreate()with the same name will return an existing SparkSession once it has been created.

C : If a SparkSession already exists, this code will return the existing session instead of creating a new one.

D : A new SparkSession is created every time thegetOrCreate()method is invoked.

Answer: C

Confidently Practice Online with Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Cram

Buying Options: