Question 1

A data engineer is working with a large JSON dataset containing order information. The dataset is stored in a distributed file system and needs to be loaded into a Spark DataFrame for analysis. The data engineer wants to ensure that the schema is correctly defined and that the data is read efficiently. Which approach should the data scientist use to efficiently load the JSON data into a Spark DataFrame with a predefined schema?

Options :

A :

Use spark.read.json() to load the data, then use DataFrame.printSchema() to view the inferred schema, and finally use DataFrame.cast() to modify column types.

B : Use spark.read.json() with the inferSchema option set to true

C :

Use spark.read.format("json").load() and then use DataFrame.withColumn() to cast each column to the desired data type.

D : Define a StructType schema and use spark.read.schema(predefinedSchema).json() to load the data.

Answer: D

Question 2

Which feature of Spark Connect should be considered when designing an application that plans to enable remote interaction with a Spark cluster?

Options :

A : It is primarily used for data ingestion into Spark from external sources.

B : It provides a way to run Spark applications remotely in any programming language.

C : It can be used to interact with any remote cluster using the REST API.

D : It allows for remote execution of Spark jobs.

Answer: D

Question 3

Given this code:

withWatermark("event_time","10 minutes") .groupBy(window("event_time","15 minutes")) .count() What happens to data that arrives after the watermark threshold?

Options :

A :

Records that arrive later than the watermark threshold (10 minutes) will automatically be included in the aggregation if they fall within the 15-minute window.

B :

Any data arriving more than 10 minutes after the watermark threshold will be ignored and not included in the aggregation.

C :

Data arriving more than 10 minutes after the latest watermark will still be included in the aggregation but will be placed into the next window.

D :

The watermark ensures that late data arriving within 10 minutes of the latest event_time will be processed and included in the windowed aggregation.

Answer: B

Question 4

Given a DataFramedfthat has 10 partitions, after running the code: result = df.coalesce(20) How many partitions will the result DataFrame have?

Options :

A : 10

B : Same number as the cluster executors

C : 1

D : 20

Answer: A

Question 5

A data engineer needs to write a Streaming DataFrame as Parquet files. Given the code:

Given the code:

Options :

A :

format("parquet") .option("location", "path/to/destination/dir")

B :

CopyEdit .option("format", "parquet") .option("destination", "path/to/destination/dir")

C :

.option("format", "parquet") .option("location", "path/to/destination/dir")

D :

format("parquet") .option("path", "path/to/destination/dir")

Answer: D

Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Mock Exam – Practice Online Confidently

Buying Options: