Databricks Certified Data Engineer Professional Practice Test 70
Question 690
A deployment owner at Northfield Analytics needs to implement a remedy: a streaming table may accept only rows with nonnull transaction IDs. The present failure is: Invalid rows currently reach downstream aggregates. Which change should be applied?
- Query supported expectation metric records in the event log
- Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition
- Change to a left join and explicitly handle missing dimension fields
- Use MERGE with matched update and not-matched insert branches
Explanation
Correct Answer
B:Add a supported expectation with ON VIOLATION DROP ROW to the dataset definition. This directly changes the feature or logic behind invalid rows currently reach downstream aggregates. A Lakeflow expectation configured to drop invalid rows filters violations from target output. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Query supported expectation metric records in the event log. This implement action addresses a different case (quality metric interpretation), where analysts misread a quality count as the number of pipeline task failures. Expectation metrics describe data rows and must be interpreted separately from orchestration status. In the stated problem, invalid rows currently reach downstream aggregates; this choice would not establish the required expectation drop action decision.
C:Change to a left join and explicitly handle missing dimension fields. This implement action addresses a different case (join with nullable keys), where rows with null external IDs disappear from an inner join. SQL null equality and join type determine whether unmatched rows survive. In the stated problem, invalid rows currently reach downstream aggregates; this choice would not establish the required expectation drop action decision.
D:Use MERGE with matched update and not-matched insert branches. This implement action addresses a different case (cdc merge semantics), where the current INSERT-only target contains duplicate customer keys. A keyed Delta MERGE can update existing rows and insert new keys. In the stated problem, invalid rows currently reach downstream aggregates; this choice would not establish the required expectation drop action decision.
Question 691
At Eastern Star Foods, a customer requests removal of personal data across bronze, silver and gold. Architecture constraint: Deleting a row from only the gold table leaves copies elsewhere. Which design best addresses this requirement?
- Use group-based least-privilege workspace object permissions
- Use a row filter keyed to approved region mapping
- Design an auditable deletion workflow spanning every relevant layer
- Publish a minimal secure view with only approved attributes
Explanation
Correct Answer
C:Design an auditable deletion workflow spanning every relevant layer. This design addresses deleting a row from only the gold table leaves copies elsewhere. A complete delete workflow tracks identifiers and propagation across all governed copies. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Use group-based least-privilege workspace object permissions. This design action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, deleting a row from only the gold table leaves copies elsewhere; this choice would not establish the required data subject request decision.
B:Use a row filter keyed to approved region mapping. This design action addresses a different case (unity catalog row filter), where a shared table currently returns all regions to every reader. Unity Catalog row filters can restrict row visibility through governed policies. In the stated problem, deleting a row from only the gold table leaves copies elsewhere; this choice would not establish the required data subject request decision.
D:Publish a minimal secure view with only approved attributes. This design action addresses a different case (sensitive data suppression), where a broad SELECT still includes unstructured notes. Column suppression and data minimization prevent unnecessary sensitive-field release. In the stated problem, deleting a row from only the gold table leaves copies elsewhere; this choice would not establish the required data subject request decision.
Question 692
At Lakewood Distribution, a reporting dashboard queries a complex aggregate without requiring row-by-row streaming. Architecture constraint: Each dashboard query recomputes the same expensive aggregation. Which design best addresses this requirement?
- Define validated region and catalog parameters for each invocation
- Defer expensive materialization until the required action
- Choose a return type and null-handling contract matching the output
- Choose a materialized view for refreshable precomputed analytics
Explanation
Correct Answer
D:Choose a materialized view for refreshable precomputed analytics. This design addresses each dashboard query recomputes the same expensive aggregation. A materialized view maintains a precomputed query result through refresh. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Define validated region and catalog parameters for each invocation. This design action addresses a different case (job parameters), where code contains hard-coded region IDs and catalog names. Task parameters allow runtime-specific values without changing notebook source. In the stated problem, each dashboard query recomputes the same expensive aggregation; this choice would not establish the required materialized view choice decision.
B:Defer expensive materialization until the required action. This design action addresses a different case (spark lazy evaluation), where engineers assume an intermediate filter executed before any action. Spark transformations are lazy until an action triggers a job. In the stated problem, each dashboard query recomputes the same expensive aggregation; this choice would not establish the required materialized view choice decision.
C:Choose a return type and null-handling contract matching the output. This design action addresses a different case (udf return schema), where returned values cannot be converted to the declared Spark return type. Pandas UDF return values must conform to the declared Spark data type. In the stated problem, each dashboard query recomputes the same expensive aggregation; this choice would not establish the required materialized view choice decision.
Question 693
Quality assurance at Eastern Star Foods checks a corrected workload where an existing unpartitioned table is being migrated to liquid clustering. The original fault was: The proposed maintenance script also runs ZORDER on that clustered table. Which verification is most decisive?
- Execute supported OPTIMIZE on the clustered table successfully
- Confirm maintenance activity occurs on appropriate tables without duplicate jobs
- Benchmark light and heavy runs to verify runtime and cost objectives
- Run small and large test inputs and compare execution strategies
Explanation
Correct Answer
A:Execute supported OPTIMIZE on the clustered table successfully. This tests the expected result against the proposed maintenance script also runs ZORDER on that clustered table, not simply whether a job or object exists. Liquid clustering is incompatible with concurrently applying ZORDER to the same table.
Incorrect Answers
B:Confirm maintenance activity occurs on appropriate tables without duplicate jobs. This validate action addresses a different case (predictive optimization), where operators run blanket manual OPTIMIZE jobs regardless of table use. Predictive optimization can automate supported managed-table maintenance based on workload signals. In the stated problem, the proposed maintenance script also runs ZORDER on that clustered table; this choice would not establish the required clustering versus zorder decision.
C:Benchmark light and heavy runs to verify runtime and cost objectives. This validate action addresses a different case (autoscaling right-size), where a fixed high-worker count wastes resources on quiet days. Supported autoscaling can adapt worker count within configured bounds. In the stated problem, the proposed maintenance script also runs ZORDER on that clustered table; this choice would not establish the required clustering versus zorder decision.
D:Run small and large test inputs and compare execution strategies. This validate action addresses a different case (adaptive execution), where static join strategy performs poorly at varying input volumes. Adaptive Query Execution can adjust supported shuffle partitions and join strategies at runtime. In the stated problem, the proposed maintenance script also runs ZORDER on that clustered table; this choice would not establish the required clustering versus zorder decision.
Question 694
A data engineer at Cascade Manufacturing must troubleshoot this case: a reporting table must always contain at least one row per active business unit. Evidence: The current dashboard has no automated alert for missing units. What should be inspected first to confirm the failure mechanism?
- Inspect query history for SQL text identity, duration and warehouse
- Inspect the unit-coverage query and expected threshold
- Inspect allowed notification-destination types and current run triggers
- Inspect Query Profile operators for scan volume and pruning
Explanation
Correct Answer
B:Inspect the unit-coverage query and expected threshold. This checks the specific evidence behind the current dashboard has no automated alert for missing units. SQL alerts can evaluate a query result against a configured threshold. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect query history for SQL text identity, duration and warehouse. This diagnose action addresses a different case (sql query history), where no baseline is available for the warehouse query execution. Query history/system query records support comparison of execution time and workload. In the stated problem, the current dashboard has no automated alert for missing units; this choice would not establish the required sql alert threshold decision.
C:Inspect allowed notification-destination types and current run triggers. This diagnose action addresses a different case (job webhook destination), where only personal email notifications are configured. Supported job notification destinations can integrate workflow status with an incident system. In the stated problem, the current dashboard has no automated alert for missing units; this choice would not establish the required sql alert threshold decision.
D:Inspect Query Profile operators for scan volume and pruning. This diagnose action addresses a different case (query profile scans), where operators assume the query is always CPU-bound. Query Profile identifies scan, pruning and shuffle behavior for SQL workloads. In the stated problem, the current dashboard has no automated alert for missing units; this choice would not establish the required sql alert threshold decision.
Question 695
At Westridge Research, on-call must learn when a critical batch job fails. A change was deployed to address: Operators manually check the run page every morning. Which test result provides the strongest evidence that the requirement is now met?
- Confirm the profile scans fewer files with correct result rows
- Cause a safe test failure and confirm exactly one incident is created
- Trigger a controlled failure and confirm notification delivery
- Find the expected schema error and confirm it disappears after correction
Explanation
Correct Answer
C:Trigger a controlled failure and confirm notification delivery. This tests the expected result against operators manually check the run page every morning, not simply whether a job or object exists. Lakeflow Jobs support status-triggered notifications for run outcomes.
Incorrect Answers
A:Confirm the profile scans fewer files with correct result rows. This validate action addresses a different case (query profile scans), where operators assume the query is always CPU-bound. Query Profile identifies scan, pruning and shuffle behavior for SQL workloads. In the stated problem, operators manually check the run page every morning; this choice would not establish the required job email notification decision.
B:Cause a safe test failure and confirm exactly one incident is created. This validate action addresses a different case (job webhook destination), where only personal email notifications are configured. Supported job notification destinations can integrate workflow status with an incident system. In the stated problem, operators manually check the run page every morning; this choice would not establish the required job email notification decision.
D:Find the expected schema error and confirm it disappears after correction. This validate action addresses a different case (pipeline event log), where the Jobs status only reports a high-level failure. Lakeflow pipeline event logs include update-level errors and progress events. In the stated problem, operators manually check the run page every morning; this choice would not establish the required job email notification decision.
Question 696
At Arbor Medical, short scheduled ETL tasks waste time starting and idling classic clusters. Observed issue: Provisioning overhead exceeds useful task execution time. Before applying a fix, which investigation is most directly relevant?
- Inspect per-task shuffle read bytes and spilled data
- Inspect task skew and key-frequency histograms
- Inspect repeated lineage and memory availability for the reference data
- Inspect job task support, runtime constraints and cluster startup times
Explanation
Correct Answer
D:Inspect job task support, runtime constraints and cluster startup times. This checks the specific evidence behind provisioning overhead exceeds useful task execution time. Supported serverless jobs can reduce infrastructure management and startup overhead. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect per-task shuffle read bytes and spilled data. This diagnose action addresses a different case (shuffle partition count), where the current partition count is too low for data volume. Shuffle partition sizing affects memory pressure and task parallelism. In the stated problem, provisioning overhead exceeds useful task execution time; this choice would not establish the required serverless job fit decision.
B:Inspect task skew and key-frequency histograms. This diagnose action addresses a different case (skewed joins), where a single join partition spills heavily to disk. Adaptive skew handling or key salting can mitigate a hot join key. In the stated problem, provisioning overhead exceeds useful task execution time; this choice would not establish the required serverless job fit decision.
C:Inspect repeated lineage and memory availability for the reference data. This diagnose action addresses a different case (cache reuse), where repeated recomputation dominates each branch's execution. Persistence can avoid repeated work if reused data fits the memory budget. In the stated problem, provisioning overhead exceeds useful task execution time; this choice would not establish the required serverless job fit decision.
Question 697
A deployment owner at Fairhaven Transport needs to implement a remedy: security wants to limit interactive workspace logins from untrusted networks. The present failure is: A Unity Catalog table grant does not restrict web-console network entry. Which change should be applied?
- Configure the approved network access control for the workspace
- Grant only required read privileges on the external location
- Eliminate overly broad edit access and grant view permission to readers
- Revoke unnecessary production ownership and grant task-specific roles
Explanation
Correct Answer
A:Configure the approved network access control for the workspace. This directly changes the feature or logic behind a Unity Catalog table grant does not restrict web-console network entry. Workspace access controls and data-object privileges address different security layers. It is the concrete configuration for the stated incident.
Incorrect Answers
B:Grant only required read privileges on the external location. This implement action addresses a different case (storage credential scope), where engineers propose a credential allowing access to all company buckets. Unity Catalog storage credentials and external locations can constrain storage access. In the stated problem, a Unity Catalog table grant does not restrict web-console network entry; this choice would not establish the required ip-based workspace restrictions decision.
C:Remove overly broad edit access and grant view permission to readers. This implement action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, a Unity Catalog table grant does not restrict web-console network entry; this choice would not establish the required ip-based workspace restrictions decision.
D:Revoke unnecessary production ownership and grant task-specific roles. This implement action addresses a different case (separation of duties), where all developers currently own production catalogs and can grant access. Separate deploy and governance roles reduce accidental privilege escalation. In the stated problem, a Unity Catalog table grant does not restrict web-console network entry; this choice would not establish the required ip-based workspace restrictions decision.
Question 698
At Harborline Utilities, a shared ETL project uses development and production Lakeflow Jobs. A change was deployed to address: Development job arguments were deployed into the production target. Which test result provides the strongest evidence that the requirement is now met?
- Simulate passing and failing checks and verify only the intended path runs
- Verify production tasks receive the production catalog and input paths
- Run two regional jobs with distinct parameters and compare target outputs
- Reverse input arrival order and verify identical final target state
Explanation
Correct Answer
B:Verify production tasks receive the production catalog and input paths. This tests the expected result against development job arguments were deployed into the production target, not simply whether a job or object exists. Bundle targets can override environment-specific variables and resource settings.
Incorrect Answers
A:Simulate passing and failing checks and verify only the intended path runs. This validate action addresses a different case (conditional task branches), where both the normal and repair tasks run despite a branching requirement. Lakeflow Jobs condition tasks can route downstream execution based on a result. In the stated problem, development job arguments were deployed into the production target; this choice would not establish the required bundle target overrides decision.
C:Run two regional jobs with distinct parameters and compare target outputs. This validate action addresses a different case (job parameters), where code contains hard-coded region IDs and catalog names. Task parameters allow runtime-specific values without changing notebook source. In the stated problem, development job arguments were deployed into the production target; this choice would not establish the required bundle target overrides decision.
D:Reverse input arrival order and verify identical final target state. This validate action addresses a different case (cdc duplicate sequence), where the processing order for tied source events is nondeterministic. A deterministic sequencing contract needs a unique ordered event position. In the stated problem, development job arguments were deployed into the production target; this choice would not establish the required bundle target overrides decision.
Question 699
At Pinecrest Holdings, a foreachBatch sink writes aggregates to an external database. The known risk is: A retry of a completed batch produces duplicate sink rows. After remediation, which ongoing signal best detects recurrence?
- Monitor state rows and dropped late-event counts
- Monitor divergence between tested package and deployed code version
- Monitor duplicate business keys and retry/batch-ID counts
- Track right-side duplicate-key rate across loads
Explanation
Correct Answer
C:Monitor duplicate business keys and retry/batch-ID counts. This watches the failure mode described by a retry of a completed batch produces duplicate sink rows. foreachBatch can execute a batch again and requires idempotent external writes for exactly-once effects. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor state rows and dropped late-event counts. This monitor action addresses a different case (event-time watermarking), where old keys remain in state indefinitely without a lateness bound. Watermarks provide a lateness threshold for bounded stateful streaming operations. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.
B:Monitor divergence between tested package and deployed code version. This monitor action addresses a different case (transform composability), where business logic depends on notebook state and widgets. DataFrame.transform supports composable DataFrame-to-DataFrame functions. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.
D:Track right-side duplicate-key rate across loads. This monitor action addresses a different case (sql join cardinality), where the row count doubles because the right side contains duplicate join keys. A many-to-many join expands cardinality unless the right grain is established. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.
Question 700
Quality assurance at Meadowland Software checks a corrected workload where a dimension table requires exactly one active row per supplier identifier. The original fault was: An upstream replay creates multiple active versions. Which verification is most decisive?
- Shuffle source row order and verify the same account record survives
- Ingest a mixed batch and reconcile passed and quarantined counts
- Read old and new records and verify null semantics for the new field
- Assert zero supplier IDs have more than one active row
Explanation
Correct Answer
D:Assert zero supplier IDs have more than one active row. This tests the expected result against an upstream replay creates multiple active versions, not simply whether a job or object exists. An explicit uniqueness check identifies duplicate business keys before serving data.
Incorrect Answers
A:Shuffle source row order and verify the same account record survives. This validate action addresses a different case (latest-per-key with ties), where two events have the same event time and yield different surviving rows. A deterministic tie-breaker is required in a per-key window ranking. In the stated problem, an upstream replay creates multiple active versions; this choice would not establish the required uniqueness quality rule decision.
B:Ingest a mixed batch and reconcile passed and quarantined counts. This validate action addresses a different case (quarantine bad records), where dropping invalid rows silently hides upstream quality failures. Quarantine preserves invalid events separately while allowing valid records through. In the stated problem, an upstream replay creates multiple active versions; this choice would not establish the required uniqueness quality rule decision.
C:Read old and new records and verify null semantics for the new field. This validate action addresses a different case (controlled schema evolution), where the silver writer rejects every new record after the upstream schema change. Schema evolution must be reviewed rather than enabling unrestricted acceptance for every column. In the stated problem, an upstream replay creates multiple active versions; this choice would not establish the required uniqueness quality rule decision.