Question 1

Which feature of Spark Connect should be considered when designing an application that plans to enable remote interaction with a Spark cluster?

Options :

A : It is primarily used for data ingestion into Spark from external sources.

B : It provides a way to run Spark applications remotely in any programming language.

C : It can be used to interact with any remote cluster using the REST API.

D : It allows for remote execution of Spark jobs.

Answer: D

Question 2

A data engineer observes that an upstream streaming source sends duplicate records, where duplicates share the same key and have at most a 30-minute difference inevent_timestamp. The engineer adds: dropDuplicatesWithinWatermark("event_timestamp", "30 minutes") What is the result?

Options :

A : It is not able to handle deduplication in this scenario

B : It removes duplicates that arrive within the 30-minute window specified by the watermark

C : It removes all duplicates regardless of when they arrive

D : It accepts watermarks in seconds and the code results in an error

Answer: B

Question 3

A Spark application needs to read multiple Parquet files from a directory where the files have differing but compatible schemas. The data engineer wants to create a DataFrame that includes all columns from all files. Which code should the data engineer use to read the Parquet files and include all columns using Apache Spark?

Options :

A : spark.read.parquet("/data/parquet/")

B : spark.read.option("mergeSchema", True).parquet("/data/parquet/")

C : spark.read.format("parquet").option("inferSchema", "true").load("/data/parquet/")

D : spark.read.parquet("/data/parquet/").option("mergeAllCols", True)

Answer: B

Question 4

A Spark developer is developing a Spark application to monitor task performance across a cluster. One requirement is to track the maximum processing time for tasks on each worker node and consolidate this information on the driver for further analysis. Which technique should the developer use?

Options :

A : Broadcast a variable to share the maximum time among workers.

B : Configure the Spark UI to automatically collect maximum times.

C : Use an RDD action like reduce() to compute the maximum time.

D : Use an accumulator to record the maximum time on the driver.

Answer: C

Question 5

A data engineer is working ona Streaming DataFrame streaming_df with the given streaming data:

Which operation is supported with streamingdf ?

Options :

A : streaming_df. select (countDistinct ("Name") )

B : streaming_df.groupby("Id") .count ()

C : streaming_df.orderBy("timestamp").limit(4)

D : streaming_df.filter (col("count") < 30).show()

Answer: D

Confidently Practice Online with Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Cram

Buying Options: