Question 1

A Spark application needs to read multiple Parquet files from a directory where the files have differing but compatible schemas. The data engineer wants to create a DataFrame that includes all columns from all files. Which code should the data engineer use to read the Parquet files and include all columns using Apache Spark?

Options :

A : spark.read.parquet("/data/parquet/")

B : spark.read.option("mergeSchema", True).parquet("/data/parquet/")

C : spark.read.format("parquet").option("inferSchema", "true").load("/data/parquet/")

D : spark.read.parquet("/data/parquet/").option("mergeAllCols", True)

Answer: B

Question 2

A Spark developer is developing a Spark application to monitor task performance across a cluster. One requirement is to track the maximum processing time for tasks on each worker node and consolidate this information on the driver for further analysis. Which technique should the developer use?

Options :

A : Broadcast a variable to share the maximum time among workers.

B : Configure the Spark UI to automatically collect maximum times.

C : Use an RDD action like reduce() to compute the maximum time.

D : Use an accumulator to record the maximum time on the driver.

Answer: C

Question 3

Given this code:

withWatermark("event_time","10 minutes") .groupBy(window("event_time","15 minutes")) .count() What happens to data that arrives after the watermark threshold?

Options :

A :

Records that arrive later than the watermark threshold (10 minutes) will automatically be included in the aggregation if they fall within the 15-minute window.

B :

Any data arriving more than 10 minutes after the watermark threshold will be ignored and not included in the aggregation.

C :

Data arriving more than 10 minutes after the latest watermark will still be included in the aggregation but will be placed into the next window.

D :

The watermark ensures that late data arriving within 10 minutes of the latest event_time will be processed and included in the windowed aggregation.

Answer: B

Question 4

A data scientist of an e-commerce company is working with user data obtained from its subscriber database and has stored the data in a DataFrame df_user. Before further processing the data, the data scientist wants to create another DataFrame df_user_non_pii and store only the non-PII columns in this DataFrame. The PII columns in df_user are first_name, last_name, email, and birthdate. Which code snippet can be used to meet this requirement?

Options :

A : df_user_non_pii = df_user.drop("first_name", "last_name", "email", "birthdate")

B : df_user_non_pii = df_user.drop("first_name", "last_name", "email", "birthdate")

C : df_user_non_pii = df_user.dropfields("first_name", "last_name", "email", "birthdate")

D : df_user_non_pii = df_user.dropfields("first_name, last_name, email, birthdate")

Answer: A

Question 5

Given a DataFramedfthat has 10 partitions, after running the code: result = df.coalesce(20) How many partitions will the result DataFrame have?

Options :

A : 10

B : Same number as the cluster executors

C : 1

D : 20

Answer: A

Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Mock Exam – Practice Online Confidently

Buying Options: