Databricks Certified Data Engineer Professional Practice Test 1
Question 1
At Horizon Travel, supplier CSV feeds contain commas inside quoted customer names. A change was deployed to address: An ingestion reader splits quoted names into extra columns. Which test result provides the strongest evidence that the requirement is now met?
- Parse a quoted comma and confirm original column boundaries are preserved
- Compare output content length and checksum to the original object
- Parse an order with three line items and confirm all three survive
- Ingest an unknown field and confirm it is retained for triage
Explanation
Correct Answer
A:Parse a quoted comma and confirm original column boundaries are preserved. This tests the expected result against an ingestion reader splits quoted names into extra columns, not simply whether a job or object exists. CSV quoting and escape configurations must match the source serialization.
Incorrect Answers
B:Compare output content length and checksum to the original object. This validate action addresses a different case (binary payload metadata), where a line-oriented text reader corrupts the raw binary payload. binaryFile input preserves raw file bytes and path metadata rather than decoding lines. In the stated problem, an ingestion reader splits quoted names into extra columns; this choice would not establish the required malformed csv quotes decision.
C:Parse an order with three line items and confirm all three survive. This validate action addresses a different case (xml repeating elements), where an XML ingest incorrectly flattens multiple line items into one text field. XML ingestion must specify the repeating row element and preserve nested structure. In the stated problem, an ingestion reader splits quoted names into extra columns; this choice would not establish the required malformed csv quotes decision.
D:Ingest an unknown field and confirm it is retained for triage. This validate action addresses a different case (auto loader rescued data), where unknown or mismatched fields disappear during the bronze load. The rescued data column can preserve records with unexpected schema content for later remediation. In the stated problem, an ingestion reader splits quoted names into extra columns; this choice would not establish the required malformed csv quotes decision.
Question 2
At Oak Valley Retail, a Python UDF fails on certain malformed nested payloads. The known risk is: Scattered print statements do not reveal the failing object shape. After remediation, which ongoing signal best detects recurrence?
- Monitor duplicate business keys and retry/batch-ID counts
- Watch parse error counts and debugger-found edge-case regressions
- Monitor state rows and dropped late-event counts
- Monitor Python worker execution time and error rates
Explanation
Correct Answer
B:Watch parse error counts and debugger-found edge-case regressions. This watches the failure mode described by scattered print statements do not reveal the failing object shape. Debugger breakpoints and targeted local repros help inspect code path and values. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor duplicate business keys and retry/batch-ID counts. This monitor action addresses a different case (streaming idempotent sink), where a retry of a completed batch produces duplicate sink rows. foreachBatch can execute a batch again and requires idempotent external writes for exactly-once effects. In the stated problem, scattered print statements do not reveal the failing object shape; this choice would not establish the required notebook debugger decision.
C:Monitor state rows and dropped late-event counts. This monitor action addresses a different case (event-time watermarking), where old keys remain in state indefinitely without a lateness bound. Watermarks provide a lateness threshold for bounded stateful streaming operations. In the stated problem, scattered print statements do not reveal the failing object shape; this choice would not establish the required notebook debugger decision.
D:Monitor Python worker execution time and error rates. This monitor action addresses a different case (vectorized udf), where row-by-row Python invocation dominates stage runtime. Pandas UDFs process batches through Arrow rather than per-row Python calls. In the stated problem, scattered print statements do not reveal the failing object shape; this choice would not establish the required notebook debugger decision.
Question 3
Quality assurance at Horizon Travel checks a corrected workload where a dashboard repeatedly groups detailed events by day and product. The original fault was: Interactive queries scan billions of detail rows for identical totals. Which verification is most decisive?
- Compare query file skipping and fact totals before and after
- Reconcile total revenue against the approved source aggregates
- Reconcile gold totals and dashboard response time against direct fact queries
- Inject malformed input and confirm it never contaminates gold
Explanation
Correct Answer
C:Reconcile gold totals and dashboard response time against direct fact queries. This tests the expected result against interactive queries scan billions of detail rows for identical totals, not simply whether a job or object exists. A summarized gold table can serve a stable reporting grain efficiently.
Incorrect Answers
A:Compare query file skipping and fact totals before and after. This validate action addresses a different case (liquid clustering for fact models), where rigid daily partitions offer little pruning for customer filters. Liquid clustering enables query-aware data layout without high-cardinality partitioning. In the stated problem, interactive queries scan billions of detail rows for identical totals; this choice would not establish the required aggregate gold model decision.
B:Reconcile total revenue against the approved source aggregates. This validate action addresses a different case (fact table grain), where summing revenue double-counts orders with multiple items. A dimensional fact must have one clearly documented row grain. In the stated problem, interactive queries scan billions of detail rows for identical totals; this choice would not establish the required aggregate gold model decision.
D:Inject malformed input and confirm it never contaminates gold. This validate action addresses a different case (medallion bronze silver gold), where malformed source events contaminate metrics without an intermediate quality boundary. Bronze, silver and gold layers separate raw capture, validated refinement and business presentation. In the stated problem, interactive queries scan billions of detail rows for identical totals; this choice would not establish the required aggregate gold model decision.
Question 4
A deployment owner at Meadowland Software needs to implement a remedy: a dimension table requires exactly one active row per supplier identifier. The present failure is: An upstream replay creates multiple active versions. Which change should be applied?
- Declare a supported fail-on-violation expectation for the required field
- Apply a timestamp validation rule and route invalid source events to quarantine
- Apply the compatible target schema migration and update the transformation
- Quarantine or reconcile duplicate active-key records before publishing silver
Explanation
Correct Answer
D:Quarantine or reconcile duplicate active-key records before publishing silver. This directly changes the feature or logic behind an upstream replay creates multiple active versions. An explicit uniqueness check identifies duplicate business keys before serving data. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Declare a supported fail-on-violation expectation for the required field. This implement action addresses a different case (expectation fail action), where the pipeline continues despite critical null records. Fail-on-violation expectations stop updates rather than silently filtering rows. In the stated problem, an upstream replay creates multiple active versions; this choice would not establish the required uniqueness quality rule decision.
B:Apply a timestamp validation rule and route invalid source events to quarantine. This implement action addresses a different case (quarantine bad records), where dropping invalid rows silently hides upstream quality failures. Quarantine preserves invalid events separately while allowing valid records through. In the stated problem, an upstream replay creates multiple active versions; this choice would not establish the required uniqueness quality rule decision.
C:Apply the compatible target schema migration and update the transformation. This implement action addresses a different case (controlled schema evolution), where the silver writer rejects every new record after the upstream schema change. Schema evolution must be reviewed rather than enabling unrestricted acceptance for every column. In the stated problem, an upstream replay creates multiple active versions; this choice would not establish the required uniqueness quality rule decision.
Question 5
A deployment owner at Granite Edge Labs needs to implement a remedy: a Spark stream consumes from a Kafka topic into Delta. The present failure is: Restarting with a fresh checkpoint replays previously ingested messages. Which change should be applied?
- Configure the Kafka stream and persistent checkpoint path
- Configure cloudFiles ingestion into a Delta bronze streaming table
- Configure the supported XML reader with the correct rowTag and schema
- Read objects with the supported binaryFile source and retain content/path fields
Explanation
Correct Answer
A:Configure the Kafka stream and persistent checkpoint path. This directly changes the feature or logic behind restarting with a fresh checkpoint replays previously ingested messages. A stable streaming checkpoint maintains Kafka offsets and sink progress across restarts. It is the concrete configuration for the stated incident.
Incorrect Answers
B:Configure cloudFiles ingestion into a Delta bronze streaming table. This implement action addresses a different case (auto loader file discovery), where listing the full historical directory every run exceeds the ingestion SLA. Auto Loader is designed for incremental new-file discovery with persisted state. In the stated problem, restarting with a fresh checkpoint replays previously ingested messages; this choice would not establish the required kafka offset durability decision.
C:Configure the supported XML reader with the correct rowTag and schema. This implement action addresses a different case (xml repeating elements), where an XML ingest incorrectly flattens multiple line items into one text field. XML ingestion must specify the repeating row element and preserve nested structure. In the stated problem, restarting with a fresh checkpoint replays previously ingested messages; this choice would not establish the required kafka offset durability decision.
D:Read objects with the supported binaryFile source and retain content/path fields. This implement action addresses a different case (binary payload metadata), where a line-oriented text reader corrupts the raw binary payload. binaryFile input preserves raw file bytes and path metadata rather than decoding lines. In the stated problem, restarting with a fresh checkpoint replays previously ingested messages; this choice would not establish the required kafka offset durability decision.
Question 6
At Orion Capital, account subscription tiers change over time and historical reports must reflect old tiers. A change was deployed to address: A single current tier overwrites history in the dimension. Which test result provides the strongest evidence that the requirement is now met?
- Join reports across marts and verify consistent country totals
- Query facts as of past dates and verify the appropriate historical tier
- Reproduce an account merge and verify historical facts retain correct links
- Query old and new rows and confirm existing measures and keys remain valid
Explanation
Correct Answer
B:Query facts as of past dates and verify the appropriate historical tier. This tests the expected result against a single current tier overwrites history in the dimension, not simply whether a job or object exists. SCD Type 2 stores effective start/end intervals for distinct historical dimension versions.
Incorrect Answers
A:Join reports across marts and verify consistent country totals. This validate action addresses a different case (conformed dimensions), where the same country appears under conflicting dimension keys across marts. A conformed dimension provides common attribute definitions across fact models. In the stated problem, a single current tier overwrites history in the dimension; this choice would not establish the required scd type 2 effective dates decision.
C:Reproduce an account merge and verify historical facts retain correct links. This validate action addresses a different case (surrogate dimension keys), where joining facts directly on mutable business IDs links history incorrectly. Stable surrogate keys decouple dimensional identity from changing source identifiers. In the stated problem, a single current tier overwrites history in the dimension; this choice would not establish the required scd type 2 effective dates decision.
D:Query old and new rows and confirm existing measures and keys remain valid. This validate action addresses a different case (dimension evolution), where a destructive table recreation would invalidate downstream consumers. Additive schema evolution can preserve contracts when null defaults are acceptable. In the stated problem, a single current tier overwrites history in the dimension; this choice would not establish the required scd type 2 effective dates decision.
Question 7
At Westhaven Services, a CDC source occasionally emits two records for one key at the same sequence number. A change is required because: The processing order for tied source events is nondeterministic. Which implementation action most directly corrects the configuration?
- Replace a scalar Python UDF with supported column expressions or a Pandas UDF
- Declare CDC keys, sequence and delete semantics in the supported Lakeflow API
- Include the unique tie-breaker in the supported sequence expression
- Configure the supported SCD Type 2 flow with keys and sequence
Explanation
Correct Answer
C:Include the unique tie-breaker in the supported sequence expression. This directly changes the feature or logic behind the processing order for tied source events is nondeterministic. A deterministic sequencing contract needs a unique ordered event position. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Replace a scalar Python UDF with supported column expressions or a Pandas UDF. This implement action addresses a different case (vectorized udf), where row-by-row Python invocation dominates stage runtime. Pandas UDFs process batches through Arrow rather than per-row Python calls. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
B:Declare CDC keys, sequence and delete semantics in the supported Lakeflow API. This implement action addresses a different case (auto cdc keys), where applying events as append-only rows leaves outdated active accounts. AUTO CDC or applicable APPLY CHANGES needs stable keys and sequence ordering for stateful upserts. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
D:Configure the supported SCD Type 2 flow with keys and sequence. This implement action addresses a different case (scd type 2 history), where overwrite-style updates destroy historical state needed for point-in-time reporting. SCD Type 2 records effective history instead of replacing the previous version. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
Question 8
A deployment owner at Riverbend Energy needs to implement a remedy: a large Delta fact receives unpredictable queries filtered by customer and region. The present failure is: Rigid daily partitions offer little pruning for customer filters. Which change should be applied?
- Alter the dimension schema and backfill the new field safely
- Map both marts to the governed conformed dimension
- On change, close the prior version and insert a new dimension version
- Enable liquid clustering and execute an eligible OPTIMIZE process
Explanation
Correct Answer
D:Enable liquid clustering and execute an eligible OPTIMIZE process. This directly changes the feature or logic behind rigid daily partitions offer little pruning for customer filters. Liquid clustering enables query-aware data layout without high-cardinality partitioning. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Alter the dimension schema and backfill the new field safely. This implement action addresses a different case (dimension evolution), where a destructive table recreation would invalidate downstream consumers. Additive schema evolution can preserve contracts when null defaults are acceptable. In the stated problem, rigid daily partitions offer little pruning for customer filters; this choice would not establish the required liquid clustering for fact models decision.
B:Map both marts to the governed conformed dimension. This implement action addresses a different case (conformed dimensions), where the same country appears under conflicting dimension keys across marts. A conformed dimension provides common attribute definitions across fact models. In the stated problem, rigid daily partitions offer little pruning for customer filters; this choice would not establish the required liquid clustering for fact models decision.
C:On change, close the prior version and insert a new dimension version. This implement action addresses a different case (scd type 2 effective dates), where a single current tier overwrites history in the dimension. SCD Type 2 stores effective start/end intervals for distinct historical dimension versions. In the stated problem, rigid daily partitions offer little pruning for customer filters; this choice would not establish the required liquid clustering for fact models decision.
Question 9
Operations at Cedar Brook Commerce owns a workload where a customer-to-events join has one giant default customer ID. The previously identified issue was: A single join partition spills heavily to disk. Which metric or audit check best detects drift?
- Monitor hot-key frequencies and spill metrics
- Monitor plan stability and unexpected AQE fallbacks
- Monitor shuffle size per task and batch runtime
- Monitor plans that lose partition pruning
Explanation
Correct Answer
A:Monitor hot-key frequencies and spill metrics. This watches the failure mode described by a single join partition spills heavily to disk. Adaptive skew handling or key salting can mitigate a hot join key. It supplies a regression signal after remediation.
Incorrect Answers
B:Monitor plan stability and unexpected AQE fallbacks. This monitor action addresses a different case (adaptive execution), where static join strategy performs poorly at varying input volumes. Adaptive Query Execution can adjust supported shuffle partitions and join strategies at runtime. In the stated problem, a single join partition spills heavily to disk; this choice would not establish the required skewed joins decision.
C:Monitor shuffle size per task and batch runtime. This monitor action addresses a different case (shuffle partition count), where the current partition count is too low for data volume. Shuffle partition sizing affects memory pressure and task parallelism. In the stated problem, a single join partition spills heavily to disk; this choice would not establish the required skewed joins decision.
D:Monitor plans that lose partition pruning. This monitor action addresses a different case (partition pruning), where a query wraps business_date in a transformation and reads many partitions. Direct predicates on partition keys can enable partition pruning. In the stated problem, a single join partition spills heavily to disk; this choice would not establish the required skewed joins decision.
Question 10
At Westridge Research, a source string price unexpectedly arrives where a decimal is required. A change is required because: Implicit type coercion risks invalid numeric values being stored. Which implementation action most directly corrects the configuration?
- Apply a supported watermark and deduplication expression before writing
- Use TRY_CAST or equivalent safe conversion before writing the typed target
- Use MERGE with matched update and not-matched insert branches
- Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition
Explanation
Correct Answer
B:Use TRY_CAST or equivalent safe conversion before writing the typed target. This directly changes the feature or logic behind implicit type coercion risks invalid numeric values being stored. Schema validation and explicit casts make type conversion failures observable. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Apply a supported watermark and deduplication expression before writing. This implement action addresses a different case (streaming dedup watermark), where a plain dropDuplicates retains state without a practical bound for never-ending keys. Watermarked streaming deduplication can bound retained duplicate state under an explicit lateness policy. In the stated problem, implicit type coercion risks invalid numeric values being stored; this choice would not establish the required silver schema enforcement decision.
C:Use MERGE with matched update and not-matched insert branches. This implement action addresses a different case (cdc merge semantics), where the current INSERT-only target contains duplicate customer keys. A keyed Delta MERGE can update existing rows and insert new keys. In the stated problem, implicit type coercion risks invalid numeric values being stored; this choice would not establish the required silver schema enforcement decision.
D:Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition. This implement action addresses a different case (expectation drop action), where invalid rows currently reach downstream aggregates. A Lakeflow expectation configured to drop invalid rows filters violations from target output. In the stated problem, implicit type coercion risks invalid numeric values being stored; this choice would not establish the required silver schema enforcement decision.