Question 1

Which feature of Spark Connect should be considered when designing an application that plans to enable remote interaction with a Spark cluster?

Options :

A : It is primarily used for data ingestion into Spark from external sources.

B : It provides a way to run Spark applications remotely in any programming language.

C : It can be used to interact with any remote cluster using the REST API.

D : It allows for remote execution of Spark jobs.

Answer: D

Question 2

A Spark engineer must select an appropriate deployment mode for the Spark jobs. What is the benefit of using cluster mode in Apache Spark™?

Options :

A :

In cluster mode, resources are allocated from a resource manager on the cluster, enabling better performance and scalability for large jobs

B :

In cluster mode, the driver is responsible for executing all tasks locally without distributing them across the worker nodes.

C :

In cluster mode, the driver runs on the client machine, which can limit the application's ability to handle large datasets efficiently.

D :

In cluster mode, the driver program runs on one of the worker nodes, allowing the application to fully utilize the distributed resources of the cluster.

Answer: D

Question 3

A data engineer observes that an upstream streaming source sends duplicate records, where duplicates share the same key and have at most a 30-minute difference inevent_timestamp. The engineer adds: dropDuplicatesWithinWatermark("event_timestamp", "30 minutes") What is the result?

Options :

A : It is not able to handle deduplication in this scenario

B : It removes duplicates that arrive within the 30-minute window specified by the watermark

C : It removes all duplicates regardless of when they arrive

D : It accepts watermarks in seconds and the code results in an error

Answer: B

Question 4

A data scientist of an e-commerce company is working with user data obtained from its subscriber database and has stored the data in a DataFrame df_user. Before further processing the data, the data scientist wants to create another DataFrame df_user_non_pii and store only the non-PII columns in this DataFrame. The PII columns in df_user are first_name, last_name, email, and birthdate. Which code snippet can be used to meet this requirement?

Options :

A : df_user_non_pii = df_user.drop("first_name", "last_name", "email", "birthdate")

B : df_user_non_pii = df_user.drop("first_name", "last_name", "email", "birthdate")

C : df_user_non_pii = df_user.dropfields("first_name", "last_name", "email", "birthdate")

D : df_user_non_pii = df_user.dropfields("first_name, last_name, email, birthdate")

Answer: A

Question 5

A data engineer needs to write a Streaming DataFrame as Parquet files. Given the code:

Given the code:

Options :

A :

format("parquet") .option("location", "path/to/destination/dir")

B :

CopyEdit .option("format", "parquet") .option("destination", "path/to/destination/dir")

C :

.option("format", "parquet") .option("location", "path/to/destination/dir")

D :

format("parquet") .option("path", "path/to/destination/dir")

Answer: D

Confidently Practice Online with Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Cram

Buying Options: