Databricks Certified Data Engineer Professional Practice Test 37
Question 360
Quality assurance at Arbor Medical checks a corrected workload where cI deploys bundles using a developer personal access token. The original fault was: Scheduled production deployment fails when the developer's token expires. Which verification is most decisive?
- Submit a known broken transformation and confirm deployment is blocked
- Repair the task and verify one notification per batch
- Introduce an invalid key and confirm CI prevents deployment
- Rotate the human token and verify CI deployment still authenticates
Explanation
Correct Answer
D:Rotate the human token and verify CI deployment still authenticates. This tests the expected result against scheduled production deployment fails when the developer's token expires, not simply whether a job or object exists. Workload identity or service principal authentication decouples CI from employees.
Incorrect Answers
A:Submit a known broken transformation and confirm deployment is blocked. This validate action addresses a different case (ci quality gates), where the deployment pipeline only checks YAML syntax. CI must run transformation tests and controlled integration checks before deployment. In the stated problem, scheduled production deployment fails when the developer's token expires; this choice would not establish the required api deployment authentication decision.
B:Repair the task and verify one notification per batch. This validate action addresses a different case (idempotent retry repair), where customers receive repeated notifications for the same batch. Task retries need idempotent side effects even when upstream work is reused. In the stated problem, scheduled production deployment fails when the developer's token expires; this choice would not establish the required api deployment authentication decision.
C:Introduce an invalid key and confirm CI prevents deployment. This validate action addresses a different case (bundle validation), where an invalid resource key would be rejected only after production change. Bundle validation detects many configuration problems before deployment. In the stated problem, scheduled production deployment fails when the developer's token expires; this choice would not establish the required api deployment authentication decision.
Question 361
At Riverbend Energy, a four-task Lakeflow Job has one failed downstream task after three predecessors succeeded. The known risk is: Operators rerun the entire job and duplicate completed side effects. After remediation, which ongoing signal best detects recurrence?
- Monitor repeated repairs and remaining downstream failures
- Audit repeated authorization errors and new broad grants
- Monitor spill, OOM and failed-task distribution
- Monitor duplicate delivery IDs and retry attempts
Explanation
Correct Answer
A:Monitor repeated repairs and remaining downstream failures. This watches the failure mode described by operators rerun the entire job and duplicate completed side effects. Repair runs can rerun failed or selected tasks while preserving successful predecessor work under supported semantics. It supplies a regression signal after remediation.
Incorrect Answers
B:Audit repeated authorization errors and new broad grants. This monitor action addresses a different case (job run permission debug), where the run-as principal lacks MODIFY on the target table. Unity Catalog authorization applies to the job execution identity. In the stated problem, operators rerun the entire job and duplicate completed side effects; this choice would not establish the required repair failed tasks decision.
C:Monitor spill, OOM and failed-task distribution. This monitor action addresses a different case (spark ui failure stage), where operators increase driver memory but failures occur on executors. Spark UI task failures and shuffle/spill metrics distinguish worker pressure from driver issues. In the stated problem, operators rerun the entire job and duplicate completed side effects; this choice would not establish the required repair failed tasks decision.
D:Monitor duplicate delivery IDs and retry attempts. This monitor action addresses a different case (idempotent retry repair), where customers receive repeated notifications for the same batch. Task retries need idempotent side effects even when upstream work is reused. In the stated problem, operators rerun the entire job and duplicate completed side effects; this choice would not establish the required repair failed tasks decision.
Question 362
A deployment owner at Silver Ridge Logistics needs to implement a remedy: a silver customer table needs insert and update handling from CDC. The present failure is: The current INSERT-only target contains duplicate customer keys. Which change should be applied?
- Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition
- Use MERGE with matched update and not-matched insert branches
- Quarantine or reconcile duplicate active-key records before publishing silver
- Apply the compatible target schema migration and update the transformation
Explanation
Correct Answer
B:Use MERGE with matched update and not-matched insert branches. This directly changes the feature or logic behind the current INSERT-only target contains duplicate customer keys. A keyed Delta MERGE can update existing rows and insert new keys. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition. This implement action addresses a different case (expectation drop action), where invalid rows currently reach downstream aggregates. A Lakeflow expectation configured to drop invalid rows filters violations from target output. In the stated problem, the current INSERT-only target contains duplicate customer keys; this choice would not establish the required cdc merge semantics decision.
C:Quarantine or reconcile duplicate active-key records before publishing silver. This implement action addresses a different case (uniqueness quality rule), where an upstream replay creates multiple active versions. An explicit uniqueness check identifies duplicate business keys before serving data. In the stated problem, the current INSERT-only target contains duplicate customer keys; this choice would not establish the required cdc merge semantics decision.
D:Apply the compatible target schema migration and update the transformation. This implement action addresses a different case (controlled schema evolution), where the silver writer rejects every new record after the upstream schema change. Schema evolution must be reviewed rather than enabling unrestricted acceptance for every column. In the stated problem, the current INSERT-only target contains duplicate customer keys; this choice would not establish the required cdc merge semantics decision.
Question 363
Operations at Granite Edge Labs owns a workload where mobile events can arrive several hours after their event timestamp. The previously identified issue was: Reports silently discard legitimate late events under an aggressive watermark. Which metric or audit check best detects drift?
- Monitor freshness and streaming progress events
- Monitor new per-key ordering collisions
- Monitor lateness distributions relative to the chosen threshold
- Monitor state rows and dropped late-event counts
Explanation
Correct Answer
C:Monitor lateness distributions relative to the chosen threshold. This watches the failure mode described by reports silently discard legitimate late events under an aggressive watermark. A watermark trades supported lateness for state size and timeliness. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor freshness and streaming progress events. This monitor action addresses a different case (streaming table choice), where periodic full-refresh queries miss required incremental freshness. A streaming table tracks continuously arriving source data incrementally. In the stated problem, reports silently discard legitimate late events under an aggressive watermark; this choice would not establish the required late event semantics decision.
B:Monitor new per-key ordering collisions. This monitor action addresses a different case (cdc duplicate sequence), where the processing order for tied source events is nondeterministic. A deterministic sequencing contract needs a unique ordered event position. In the stated problem, reports silently discard legitimate late events under an aggressive watermark; this choice would not establish the required late event semantics decision.
D:Monitor state rows and dropped late-event counts. This monitor action addresses a different case (event-time watermarking), where old keys remain in state indefinitely without a lateness bound. Watermarks provide a lateness threshold for bounded stateful streaming operations. In the stated problem, reports silently discard legitimate late events under an aggressive watermark; this choice would not establish the required late event semantics decision.
Question 364
At Silver Ridge Logistics, an aggregation workload has one very slow Spark stage. Architecture constraint: Operators only monitor cluster-wide CPU and miss skewed task durations. Which design best addresses this requirement?
- Build structured operational diagnostics from pipeline event records
- Define freshness SLO based on end-to-end observed lag
- Create a historical failure and runtime dashboard by workflow
- Build a tuning plan from the specific skew or spill evidence
Explanation
Correct Answer
D:Build a tuning plan from the specific skew or spill evidence. This design addresses operators only monitor cluster-wide CPU and miss skewed task durations. Spark UI stage metrics reveal shuffle, spill and task skew. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Build structured operational diagnostics from pipeline event records. This design action addresses a different case (pipeline event log), where the Jobs status only reports a high-level failure. Lakeflow pipeline event logs include update-level errors and progress events. In the stated problem, operators only monitor cluster-wide CPU and miss skewed task durations; this choice would not establish the required spark ui shuffle metrics decision.
B:Define freshness SLO based on end-to-end observed lag. This design action addresses a different case (end-to-end sla), where operators monitor compute uptime rather than data freshness. SLA monitoring should measure source event time to published table availability. In the stated problem, operators only monitor cluster-wide CPU and miss skewed task durations; this choice would not establish the required spark ui shuffle metrics decision.
C:Create a historical failure and runtime dashboard by workflow. This design action addresses a different case (job operational history), where operators only see the most recent run in the Jobs UI. Jobs system records support historical analysis of runs and tasks. In the stated problem, operators only monitor cluster-wide CPU and miss skewed task durations; this choice would not establish the required spark ui shuffle metrics decision.
Question 365
At Cascade Manufacturing, a monolithic notebook transformation cannot be tested separately. A change was deployed to address: Business logic depends on notebook state and widgets. Which test result provides the strongest evidence that the requirement is now met?
- Run the same function in a unit test and in a production job
- Deploy the same revision to test and staging using distinct bundle targets
- Confirm unit tests run without credentials and integrations run in an approved test workspace
- Test late events on both sides of the threshold and reconcile counts
Explanation
Correct Answer
A:Run the same function in a unit test and in a production job. This tests the expected result against business logic depends on notebook state and widgets, not simply whether a job or object exists. DataFrame.transform supports composable DataFrame-to-DataFrame functions.
Incorrect Answers
B:Deploy the same revision to test and staging using distinct bundle targets. This validate action addresses a different case (bundle repository layout), where source files and deployment definitions are coupled to one developer workspace. A versioned Declarative Automation Bundle defines code and resource configurations separately from the target workspace. In the stated problem, business logic depends on notebook state and widgets; this choice would not establish the required transform composability decision.
C:Confirm unit tests run without credentials and integrations run in an approved test workspace. This validate action addresses a different case (unit versus integration tests), where small logic regressions are discovered late and costly to debug. Unit tests isolate transformations while integration tests validate external contracts. In the stated problem, business logic depends on notebook state and widgets; this choice would not establish the required transform composability decision.
D:Test late events on both sides of the threshold and reconcile counts. This validate action addresses a different case (event-time watermarking), where old keys remain in state indefinitely without a lateness bound. Watermarks provide a lateness threshold for bounded stateful streaming operations. In the stated problem, business logic depends on notebook state and widgets; this choice would not establish the required transform composability decision.
Question 366
Operations at Clearstone Media owns a workload where a message bus sends Avro-encoded order events with added optional fields. The previously identified issue was: An older consumer cannot deserialize records from a newer producer schema. Which metric or audit check best detects drift?
- Monitor consumer lag and checkpoint health
- Monitor schema-registry incompatibility and decode failures
- Monitor newly arriving file-schema mismatches
- Monitor schema changes and rescued-field counts
Explanation
Correct Answer
B:Monitor schema-registry incompatibility and decode failures. This watches the failure mode described by an older consumer cannot deserialize records from a newer producer schema. Avro writer-reader schema compatibility must be maintained for evolution. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor consumer lag and checkpoint health. This monitor action addresses a different case (kafka offset durability), where restarting with a fresh checkpoint replays previously ingested messages. A stable streaming checkpoint maintains Kafka offsets and sink progress across restarts. In the stated problem, an older consumer cannot deserialize records from a newer producer schema; this choice would not establish the required avro schema evolution decision.
C:Monitor newly arriving file-schema mismatches. This monitor action addresses a different case (parquet schema mismatch), where a merge step fails after a producer changes an integer field to string. Parquet schema compatibility matters even when column names are unchanged. In the stated problem, an older consumer cannot deserialize records from a newer producer schema; this choice would not establish the required avro schema evolution decision.
D:Monitor schema changes and rescued-field counts. This monitor action addresses a different case (nested json ingest), where the initial reader assumes one unchanging flat schema and loses new fields. Explicit schema management and rescued data handling protect evolving semi-structured ingestion. In the stated problem, an older consumer cannot deserialize records from a newer producer schema; this choice would not establish the required avro schema evolution decision.
Question 367
Operations at Oak Valley Retail owns a workload where source customer identifiers can be reused following account mergers. The previously identified issue was: Joining facts directly on mutable business IDs links history incorrectly. Which metric or audit check best detects drift?
- Monitor overlapping intervals and multiple current versions
- Monitor duplicate bridge rows and invalid overlapping memberships
- Monitor orphaned foreign keys and repeated business identifiers
- Monitor fact foreign-key consistency and changing dimension attributes
Explanation
Correct Answer
C:Monitor orphaned foreign keys and repeated business identifiers. This watches the failure mode described by joining facts directly on mutable business IDs links history incorrectly. Stable surrogate keys decouple dimensional identity from changing source identifiers. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor overlapping intervals and multiple current versions. This monitor action addresses a different case (scd type 2 effective dates), where a single current tier overwrites history in the dimension. SCD Type 2 stores effective start/end intervals for distinct historical dimension versions. In the stated problem, joining facts directly on mutable business IDs links history incorrectly; this choice would not establish the required surrogate dimension keys decision.
B:Monitor duplicate bridge rows and invalid overlapping memberships. This monitor action addresses a different case (many-to-many bridge), where joining facts directly to the membership table multiplies totals. A bridge table models many-to-many relationships at an explicit membership grain. In the stated problem, joining facts directly on mutable business IDs links history incorrectly; this choice would not establish the required surrogate dimension keys decision.
D:Monitor fact foreign-key consistency and changing dimension attributes. This monitor action addresses a different case (star schema analytics), where a denormalized mixed-grain table produces redundant attributes and expensive corrections. A star schema links a clear-grain fact to reusable dimensions for consistent analytics. In the stated problem, joining facts directly on mutable business IDs links history incorrectly; this choice would not establish the required surrogate dimension keys decision.
Question 368
At Summit Consumer Goods, a large Delta table receives frequent narrow DELETE operations. A change was deployed to address: Each small delete rewrites large data files and increases write cost. Which test result provides the strongest evidence that the requirement is now met?
- Reconcile rows and compare scan cost against text storage
- Confirm reused stages avoid recomputation with no change in output
- Confirm maintenance activity occurs on appropriate tables without duplicate jobs
- Delete a few rows and confirm snapshot consistency without unnecessary rewrites
Explanation
Correct Answer
D:Delete a few rows and confirm snapshot consistency without unnecessary rewrites. This tests the expected result against each small delete rewrites large data files and increases write cost, not simply whether a job or object exists. Deletion vectors can mark deleted rows without immediate whole-file rewrite when supported.
Incorrect Answers
A:Reconcile rows and compare scan cost against text storage. This validate action addresses a different case (columnar compression), where row-oriented text storage increases scans and transfer volumes. Columnar formats and appropriate encoding improve analytical read efficiency. In the stated problem, each small delete rewrites large data files and increases write cost; this choice would not establish the required deletion vectors decision.
B:Confirm reused stages avoid recomputation with no change in output. This validate action addresses a different case (cache reuse), where repeated recomputation dominates each branch's execution. Persistence can avoid repeated work if reused data fits the memory budget. In the stated problem, each small delete rewrites large data files and increases write cost; this choice would not establish the required deletion vectors decision.
C:Confirm maintenance activity occurs on appropriate tables without duplicate jobs. This validate action addresses a different case (predictive optimization), where operators run blanket manual OPTIMIZE jobs regardless of table use. Predictive optimization can automate supported managed-table maintenance based on workload signals. In the stated problem, each small delete rewrites large data files and increases write cost; this choice would not establish the required deletion vectors decision.
Question 369
A deployment owner at Riverbend Energy needs to implement a remedy: an engineering team owns many internal Delta tables with no need for external file lifecycle control. The present failure is: Each table has custom file cleanup jobs and broad storage permissions. Which change should be applied?
- Create the managed table under the approved catalog and migrate governed data
- Activate the table feature under approved compatibility requirements
- Migrate a compatible task to serverless execution with approved permissions
- Enable predictive optimization for eligible catalog objects under governance
Explanation
Correct Answer
A:Create the managed table under the approved catalog and migrate governed data. This directly changes the feature or logic behind each table has custom file cleanup jobs and broad storage permissions. Unity Catalog managed tables provide platform-managed data placement and maintenance capabilities. It is the concrete configuration for the stated incident.
Incorrect Answers
B:Activate the table feature under approved compatibility requirements. This implement action addresses a different case (deletion vectors), where each small delete rewrites large data files and increases write cost. Deletion vectors can mark deleted rows without immediate whole-file rewrite when supported. In the stated problem, each table has custom file cleanup jobs and broad storage permissions; this choice would not establish the required managed table maintenance decision.
C:Migrate a compatible task to serverless execution with approved permissions. This implement action addresses a different case (serverless job fit), where provisioning overhead exceeds useful task execution time. Supported serverless jobs can reduce infrastructure management and startup overhead. In the stated problem, each table has custom file cleanup jobs and broad storage permissions; this choice would not establish the required managed table maintenance decision.
D:Enable predictive optimization for eligible catalog objects under governance. This implement action addresses a different case (predictive optimization), where operators run blanket manual OPTIMIZE jobs regardless of table use. Predictive optimization can automate supported managed-table maintenance based on workload signals. In the stated problem, each table has custom file cleanup jobs and broad storage permissions; this choice would not establish the required managed table maintenance decision.
Question 370
At Bayview Insurance, a bundle task fails after a runtime upgrade. Observed issue: A package import error appears only on the target compute. Before applying a fix, which investigation is most directly relevant?
- Inspect pipeline event log update IDs and error messages
- Inspect Python version, wheel tags and package resolution logs
- Inspect effective grants on source, target and parent containers for the run-as principal
- Inspect executor stderr and driver logs near the failed stage
Explanation
Correct Answer
B:Inspect Python version, wheel tags and package resolution logs. This checks the specific evidence behind a package import error appears only on the target compute. Runtime-specific wheel compatibility and pinning must be tested in CI. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect pipeline event log update IDs and error messages. This diagnose action addresses a different case (lakeflow event error classes), where the only visible dashboard status is FAILED without a dataset name. Pipeline event logs identify failing update and dataset messages. In the stated problem, a package import error appears only on the target compute; this choice would not establish the required python dependency triage decision.
C:Inspect effective grants on source, target and parent containers for the run-as principal. This diagnose action addresses a different case (job run permission debug), where the run-as principal lacks MODIFY on the target table. Unity Catalog authorization applies to the job execution identity. In the stated problem, a package import error appears only on the target compute; this choice would not establish the required python dependency triage decision.
D:Inspect executor stderr and driver logs near the failed stage. This diagnose action addresses a different case (cluster log diagnostics), where the notebook cell shows only a generic stage failure. Driver and executor logs often contain the exact import or JVM error. In the stated problem, a package import error appears only on the target compute; this choice would not establish the required python dependency triage decision.