Databricks Certified Data Engineer Professional Practice Test 15
Question 140
Quality assurance at Fairhaven Transport checks a corrected workload where a customer dimension must preserve previous plan tier values. The original fault was: Overwrite-style updates destroy historical state needed for point-in-time reporting. Which verification is most decisive?
- Restart and confirm previously committed input is not reprocessed unexpectedly
- Replay update and delete events and confirm one accurate current row per key
- Confirm dashboard results refresh after source data updates
- Change a customer tier and confirm old and current history rows are queryable
Explanation
Correct Answer
D:Change a customer tier and confirm old and current history rows are queryable. This tests the expected result against overwrite-style updates destroy historical state needed for point-in-time reporting, not simply whether a job or object exists. SCD Type 2 records effective history instead of replacing the previous version.
Incorrect Answers
A:Restart and confirm previously committed input is not reprocessed unexpectedly. This validate action addresses a different case (streaming checkpoint identity), where a newly selected checkpoint location makes the stream reread historical input. Streaming checkpoint metadata records progress and state for restart continuity. In the stated problem, overwrite-style updates destroy historical state needed for point-in-time reporting; this choice would not establish the required scd type 2 history decision.
B:Replay update and delete events and confirm one accurate current row per key. This validate action addresses a different case (auto cdc keys), where applying events as append-only rows leaves outdated active accounts. AUTO CDC or applicable APPLY CHANGES needs stable keys and sequence ordering for stateful upserts. In the stated problem, overwrite-style updates destroy historical state needed for point-in-time reporting; this choice would not establish the required scd type 2 history decision.
C:Confirm dashboard results refresh after source data updates. This validate action addresses a different case (materialized view choice), where each dashboard query recomputes the same expensive aggregation. A materialized view maintains a precomputed query result through refresh. In the stated problem, overwrite-style updates destroy historical state needed for point-in-time reporting; this choice would not establish the required scd type 2 history decision.
Question 141
At Harborline Utilities, a 5-MB region dimension is joined to a multi-terabyte event table. A change is required because: The join triggers a costly shuffle of the large fact dataset. Which implementation action most directly corrects the configuration?
- Broadcast the dimension where appropriate or tune automatic broadcast threshold
- Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition
- Change to a left join and explicitly handle missing dimension fields
- Apply LEFT JOIN and CASE WHEN or COALESCE to the region output
Explanation
Correct Answer
A:Broadcast the dimension where appropriate or tune automatic broadcast threshold. This directly changes the feature or logic behind the join triggers a costly shuffle of the large fact dataset. Broadcasting an appropriately sized dimension can avoid a large shuffle join. It is the concrete configuration for the stated incident.
Incorrect Answers
B:Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition. This implement action addresses a different case (expectation drop action), where invalid rows currently reach downstream aggregates. A Lakeflow expectation configured to drop invalid rows filters violations from target output. In the stated problem, the join triggers a costly shuffle of the large fact dataset; this choice would not establish the required broadcast lookup table decision.
C:Change to a left join and explicitly handle missing dimension fields. This implement action addresses a different case (join with nullable keys), where rows with null external IDs disappear from an inner join. SQL null equality and join type determine whether unmatched rows survive. In the stated problem, the join triggers a costly shuffle of the large fact dataset; this choice would not establish the required broadcast lookup table decision.
D:Apply LEFT JOIN and CASE WHEN or COALESCE to the region output. This implement action addresses a different case (conditional enrichment), where an unconditional inner join discards rows with missing region codes. A left join with explicit CASE logic preserves rows while enriching those that match. In the stated problem, the join triggers a costly shuffle of the large fact dataset; this choice would not establish the required broadcast lookup table decision.
Question 142
At Granite Edge Labs, mobile events can arrive several hours after their event timestamp. A change was deployed to address: Reports silently discard legitimate late events under an aggressive watermark. Which test result provides the strongest evidence that the requirement is now met?
- Replay update and delete events and confirm one accurate current row per key
- Replay representative late events and verify expected acceptance or exclusion
- Use representative nulls and decimals to verify conversion and result accuracy
- Reverse input arrival order and verify identical final target state
Explanation
Correct Answer
B:Replay representative late events and verify expected acceptance or exclusion. This tests the expected result against reports silently discard legitimate late events under an aggressive watermark, not simply whether a job or object exists. A watermark trades supported lateness for state size and timeliness.
Incorrect Answers
A:Replay update and delete events and confirm one accurate current row per key. This validate action addresses a different case (auto cdc keys), where applying events as append-only rows leaves outdated active accounts. AUTO CDC or applicable APPLY CHANGES needs stable keys and sequence ordering for stateful upserts. In the stated problem, reports silently discard legitimate late events under an aggressive watermark; this choice would not establish the required late event semantics decision.
C:Use representative nulls and decimals to verify conversion and result accuracy. This validate action addresses a different case (udf return schema), where returned values cannot be converted to the declared Spark return type. Pandas UDF return values must conform to the declared Spark data type. In the stated problem, reports silently discard legitimate late events under an aggressive watermark; this choice would not establish the required late event semantics decision.
D:Reverse input arrival order and verify identical final target state. This validate action addresses a different case (cdc duplicate sequence), where the processing order for tied source events is nondeterministic. A deterministic sequencing contract needs a unique ordered event position. In the stated problem, reports silently discard legitimate late events under an aggressive watermark; this choice would not establish the required late event semantics decision.
Question 143
At Eastern Star Foods, a cloud object store receives a large and growing set of landing files. A change is required because: Listing the full historical directory every run exceeds the ingestion SLA. Which implementation action most directly corrects the configuration?
- Configure CSV quote/escape and malformed-record capture
- Configure rescued-data handling and route affected rows to a review table
- Configure cloudFiles ingestion into a Delta bronze streaming table
- Implement checkpoint-aware incremental ingestion and a safe merge downstream
Explanation
Correct Answer
C:Configure cloudFiles ingestion into a Delta bronze streaming table. This directly changes the feature or logic behind listing the full historical directory every run exceeds the ingestion SLA. Auto Loader is designed for incremental new-file discovery with persisted state. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Configure CSV quote/escape and malformed-record capture. This implement action addresses a different case (malformed csv quotes), where an ingestion reader splits quoted names into extra columns. CSV quoting and escape configurations must match the source serialization. In the stated problem, listing the full historical directory every run exceeds the ingestion SLA; this choice would not establish the required auto loader file discovery decision.
B:Configure rescued-data handling and route affected rows to a review table. This implement action addresses a different case (auto loader rescued data), where unknown or mismatched fields disappear during the bronze load. The rescued data column can preserve records with unexpected schema content for later remediation. In the stated problem, listing the full historical directory every run exceeds the ingestion SLA; this choice would not establish the required auto loader file discovery decision.
D:Implement checkpoint-aware incremental ingestion and a safe merge downstream. This implement action addresses a different case (retry-safe file loads), where a partially processed input file is loaded twice into Delta. Checkpointed incremental ingestion or stable event-level deduplication prevents repeated effects. In the stated problem, listing the full historical directory every run exceeds the ingestion SLA; this choice would not establish the required auto loader file discovery decision.
Question 144
At Cascade Manufacturing, a notebook uses a third-party package also installed as a cluster library. Architecture constraint: A notebook-scoped installation overrides the expected cluster version. Which design best addresses this requirement?
- Add representative expected DataFrame fixtures and schema assertions
- Refactor business rules as pure DataFrame transform functions
- Choose compatible pinned versions using a tested dependency lock strategy
- Use one managed and repeatable dependency source for the workload
Explanation
Correct Answer
D:Use one managed and repeatable dependency source for the workload. This design addresses a notebook-scoped installation overrides the expected cluster version. The runtime can resolve conflicting versions from different installation scopes. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Add representative expected DataFrame fixtures and schema assertions. This design action addresses a different case (dataframe testing), where tests only assert that the DataFrame is nonempty. DataFrame equality and schema assertions test data behavior and contracts. In the stated problem, a notebook-scoped installation overrides the expected cluster version; this choice would not establish the required library precedence decision.
B:Refactor business rules as pure DataFrame transform functions. This design action addresses a different case (transform composability), where business logic depends on notebook state and widgets. DataFrame.transform supports composable DataFrame-to-DataFrame functions. In the stated problem, a notebook-scoped installation overrides the expected cluster version; this choice would not establish the required library precedence decision.
C:Choose compatible pinned versions using a tested dependency lock strategy. This design action addresses a different case (library version pinning), where a transitive dependency updated to an incompatible version. Explicit compatible dependency versions improve reproducibility across new compute environments. In the stated problem, a notebook-scoped installation overrides the expected cluster version; this choice would not establish the required library precedence decision.
Question 145
At Cascade Manufacturing, a pipeline expectation dashboard shows records failing a rule. A change was deployed to address: Analysts misread a quality count as the number of pipeline task failures. Which test result provides the strongest evidence that the requirement is now met?
- Compare injected bad-row counts to quality metrics and task status
- Ingest a mixed batch and reconcile passed and quarantined counts
- Count retained fact rows and confirm null-key records are represented
- Compare known local times from two regions against the same UTC instant
Explanation
Correct Answer
A:Compare injected bad-row counts to quality metrics and task status. This tests the expected result against analysts misread a quality count as the number of pipeline task failures, not simply whether a job or object exists. Expectation metrics describe data rows and must be interpreted separately from orchestration status.
Incorrect Answers
B:Ingest a mixed batch and reconcile passed and quarantined counts. This validate action addresses a different case (quarantine bad records), where dropping invalid rows silently hides upstream quality failures. Quarantine preserves invalid events separately while allowing valid records through. In the stated problem, analysts misread a quality count as the number of pipeline task failures; this choice would not establish the required quality metric interpretation decision.
C:Count retained fact rows and confirm null-key records are represented. This validate action addresses a different case (join with nullable keys), where rows with null external IDs disappear from an inner join. SQL null equality and join type determine whether unmatched rows survive. In the stated problem, analysts misread a quality count as the number of pipeline task failures; this choice would not establish the required quality metric interpretation decision.
D:Compare known local times from two regions against the same UTC instant. This validate action addresses a different case (timestamp timezone normalization), where events from different regions are ordered incorrectly in a UTC dashboard. Time-zone normalization is required before comparing timestamps from different sources. In the stated problem, analysts misread a quality count as the number of pipeline task failures; this choice would not establish the required quality metric interpretation decision.
Question 146
At Pinecrest Holdings, a partner outside Databricks needs access to one curated table. Observed issue: The partner cannot use an internal Databricks-to-Databricks credential flow. Before applying a fix, which investigation is most directly relevant?
- Inspect recipient credential expiration and access history
- Confirm recipient platform and supported open-sharing connector
- Check whether the external source and connector are supported
- Inspect share contents and approved sharing manifest
Explanation
Correct Answer
B:Confirm recipient platform and supported open-sharing connector. This checks the specific evidence behind the partner cannot use an internal Databricks-to-Databricks credential flow. Open Delta Sharing can support non-Databricks recipients using the sharing protocol. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect recipient credential expiration and access history. This diagnose action addresses a different case (sharing credential rotation), where partners risk losing access or using a leaked long-lived credential. Credential rotation must be coordinated without unnecessarily broadening permissions. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.
C:Check whether the external source and connector are supported. This diagnose action addresses a different case (federated connection), where engineers export the full source database into cloud object storage each hour. Lakehouse Federation can query supported external sources through a catalog connection. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.
D:Inspect share contents and approved sharing manifest. This diagnose action addresses a different case (share catalog scope), where the recipient can discover unintended datasets. A Delta Sharing share should contain only explicitly approved assets. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.
Question 147
At Lakewood Distribution, a reporting dashboard queries a complex aggregate without requiring row-by-row streaming. Observed issue: Each dashboard query recomputes the same expensive aggregation. Before applying a fix, which investigation is most directly relevant?
- Inspect input rate, processed rate and microbatch duration during peaks
- Inspect per-key sequence collisions and upstream log offsets
- Inspect refresh cadence and whether continuous incremental stream processing is required
- Inspect whether a Spark action was issued and review the execution plan
Explanation
Correct Answer
C:Inspect refresh cadence and whether continuous incremental stream processing is required. This checks the specific evidence behind each dashboard query recomputes the same expensive aggregation. A materialized view maintains a precomputed query result through refresh. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect input rate, processed rate and microbatch duration during peaks. This diagnose action addresses a different case (streaming trigger tuning), where microbatch processing time exceeds the configured trigger interval. Shortening a trigger cannot make a saturated single streaming query process overlapping batches faster. In the stated problem, each dashboard query recomputes the same expensive aggregation; this choice would not establish the required materialized view choice decision.
B:Inspect per-key sequence collisions and upstream log offsets. This diagnose action addresses a different case (cdc duplicate sequence), where the processing order for tied source events is nondeterministic. A deterministic sequencing contract needs a unique ordered event position. In the stated problem, each dashboard query recomputes the same expensive aggregation; this choice would not establish the required materialized view choice decision.
D:Inspect whether a Spark action was issued and review the execution plan. This diagnose action addresses a different case (spark lazy evaluation), where engineers assume an intermediate filter executed before any action. Spark transformations are lazy until an action triggers a job. In the stated problem, each dashboard query recomputes the same expensive aggregation; this choice would not establish the required materialized view choice decision.
Question 148
At Harborline Utilities, a daily pipeline must load bronze before parallel silver transformations. A change is required because: Dependent tasks sometimes run before the bronze load finishes. Which implementation action most directly corrects the configuration?
- Add a condition task and connect success and repair paths to its respective outcomes
- Deploy the wheel task with its documented entrypoint and parameters
- Update the task dependency list and deploy a tested wheel or requirements set
- Set silver task depends_on edges pointing to the bronze task
Explanation
Correct Answer
D:Set silver task depends_on edges pointing to the bronze task. This directly changes the feature or logic behind dependent tasks sometimes run before the bronze load finishes. Lakeflow Job task dependencies form a directed acyclic execution graph. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Add a condition task and connect success and repair paths to its respective outcomes. This implement action addresses a different case (conditional task branches), where both the normal and repair tasks run despite a branching requirement. Lakeflow Jobs condition tasks can route downstream execution based on a result. In the stated problem, dependent tasks sometimes run before the bronze load finishes; this choice would not establish the required job task dag decision.
B:Deploy the wheel task with its documented entrypoint and parameters. This implement action addresses a different case (python task entrypoints), where the task exits before setting up its argument parser. A standard Python entrypoint and explicit arguments avoid notebook runtime assumptions. In the stated problem, dependent tasks sometimes run before the bronze load finishes; this choice would not establish the required job task dag decision.
C:Update the task dependency list and deploy a tested wheel or requirements set. This implement action addresses a different case (library version pinning), where a transitive dependency updated to an incompatible version. Explicit compatible dependency versions improve reproducibility across new compute environments. In the stated problem, dependent tasks sometimes run before the bronze load finishes; this choice would not establish the required job task dag decision.
Question 149
Quality assurance at Oak Valley Retail checks a corrected workload where source customer identifiers can be reused following account mergers. The original fault was: Joining facts directly on mutable business IDs links history incorrectly. Which verification is most decisive?
- Reproduce an account merge and verify historical facts retain correct links
- Query facts as of past dates and verify the appropriate historical tier
- Join reports across marts and verify consistent country totals
- Validate distinct customer counts and allocated totals after bridge joins
Explanation
Correct Answer
A:Reproduce an account merge and verify historical facts retain correct links. This tests the expected result against joining facts directly on mutable business IDs links history incorrectly, not simply whether a job or object exists. Stable surrogate keys decouple dimensional identity from changing source identifiers.
Incorrect Answers
B:Query facts as of past dates and verify the appropriate historical tier. This validate action addresses a different case (scd type 2 effective dates), where a single current tier overwrites history in the dimension. SCD Type 2 stores effective start/end intervals for distinct historical dimension versions. In the stated problem, joining facts directly on mutable business IDs links history incorrectly; this choice would not establish the required surrogate dimension keys decision.
C:Join reports across marts and verify consistent country totals. This validate action addresses a different case (conformed dimensions), where the same country appears under conflicting dimension keys across marts. A conformed dimension provides common attribute definitions across fact models. In the stated problem, joining facts directly on mutable business IDs links history incorrectly; this choice would not establish the required surrogate dimension keys decision.
D:Validate distinct customer counts and allocated totals after bridge joins. This validate action addresses a different case (many-to-many bridge), where joining facts directly to the membership table multiplies totals. A bridge table models many-to-many relationships at an explicit membership grain. In the stated problem, joining facts directly on mutable business IDs links history incorrectly; this choice would not establish the required surrogate dimension keys decision.
Question 150
At Riverbend Energy, a logistics partner sends XML with repeated line-item elements. A change is required because: An XML ingest incorrectly flattens multiple line items into one text field. Which implementation action most directly corrects the configuration?
- Configure cloudFiles ingestion into a Delta bronze streaming table
- Configure the supported XML reader with the correct rowTag and schema
- Configure CSV quote/escape and malformed-record capture
- Configure the Kafka stream and persistent checkpoint path
Explanation
Correct Answer
B:Configure the supported XML reader with the correct rowTag and schema. This directly changes the feature or logic behind an XML ingest incorrectly flattens multiple line items into one text field. XML ingestion must specify the repeating row element and preserve nested structure. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Configure cloudFiles ingestion into a Delta bronze streaming table. This implement action addresses a different case (auto loader file discovery), where listing the full historical directory every run exceeds the ingestion SLA. Auto Loader is designed for incremental new-file discovery with persisted state. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.
C:Configure CSV quote/escape and malformed-record capture. This implement action addresses a different case (malformed csv quotes), where an ingestion reader splits quoted names into extra columns. CSV quoting and escape configurations must match the source serialization. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.
D:Configure the Kafka stream and persistent checkpoint path. This implement action addresses a different case (kafka offset durability), where restarting with a fresh checkpoint replays previously ingested messages. A stable streaming checkpoint maintains Kafka offsets and sink progress across restarts. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.