Databricks Certified Data Engineer Professional Practice Test 19
Question 180
Quality assurance at Orion Capital checks a corrected workload where a group creates many tables in random external paths. The original fault was: The data platform cannot apply consistent lifecycle governance. Which verification is most decisive?
- Confirm stewards can manage the table while readers retain intended access
- Confirm users can discover metadata but cannot read restricted table rows
- Create a test table and confirm approved analysts can query it
- Confirm managed table storage resolves to the approved location
Explanation
Correct Answer
D:Confirm managed table storage resolves to the approved location. This tests the expected result against the data platform cannot apply consistent lifecycle governance, not simply whether a job or object exists. Managed locations centralize storage ownership for Unity Catalog managed assets.
Incorrect Answers
A:Confirm stewards can manage the table while readers retain intended access. This validate action addresses a different case (ownership reassignment), where new data stewards cannot maintain its grants. Unity Catalog ownership should be assigned to a maintained principal. In the stated problem, the data platform cannot apply consistent lifecycle governance; this choice would not establish the required managed storage ownership decision.
B:Confirm users can discover metadata but cannot read restricted table rows. This validate action addresses a different case (browse metadata privilege), where teams grant SELECT merely to expose object listings. BROWSE enables controlled metadata discovery without granting data read access. In the stated problem, the data platform cannot apply consistent lifecycle governance; this choice would not establish the required managed storage ownership decision.
C:Create a test table and confirm approved analysts can query it. This validate action addresses a different case (schema-level grant inheritance), where analysts must repeatedly request identical SELECT grants for each table. Unity Catalog schema-level privileges can apply to existing and future child objects. In the stated problem, the data platform cannot apply consistent lifecycle governance; this choice would not establish the required managed storage ownership decision.
Question 181
A data engineer at Harborline Utilities must troubleshoot this case: a job repair reruns an external notification task. Evidence: Customers receive repeated notifications for the same batch. What should be inspected first to confirm the failure mechanism?
- Inspect message IDs and prior notification delivery records
- Inspect CLI target resolution and workspace host for the command
- Inspect effective grants on source, target and parent containers for the run-as principal
- Inspect CI stages for tests, bundle validation and approvals
Explanation
Correct Answer
A:Inspect message IDs and prior notification delivery records. This checks the specific evidence behind customers receive repeated notifications for the same batch. Task retries need idempotent side effects even when upstream work is reused. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
B:Inspect CLI target resolution and workspace host for the command. This diagnose action addresses a different case (target-safe deployment), where the wrong workspace is selected because target is implicit. Explicit bundle target selection prevents accidental environment deployment. In the stated problem, customers receive repeated notifications for the same batch; this choice would not establish the required idempotent retry repair decision.
C:Inspect effective grants on source, target and parent containers for the run-as principal. This diagnose action addresses a different case (job run permission debug), where the run-as principal lacks MODIFY on the target table. Unity Catalog authorization applies to the job execution identity. In the stated problem, customers receive repeated notifications for the same batch; this choice would not establish the required idempotent retry repair decision.
D:Inspect CI stages for tests, bundle validation and approvals. This diagnose action addresses a different case (ci quality gates), where the deployment pipeline only checks YAML syntax. CI must run transformation tests and controlled integration checks before deployment. In the stated problem, customers receive repeated notifications for the same batch; this choice would not establish the required idempotent retry repair decision.
Question 182
A data engineer at Clearstone Media must troubleshoot this case: customer emails must be hidden from most analytics users. Evidence: A raw SELECT on an approved table exposes plaintext email. What should be inspected first to confirm the failure mechanism?
- Inspect current workspace network access controls and identity sign-in rules
- Inspect sensitive-column policy and current grants
- Inspect consumer business need and raw table grants
- Inspect ownership and grant-management privileges in production
Explanation
Correct Answer
B:Inspect sensitive-column policy and current grants. This checks the specific evidence behind a raw SELECT on an approved table exposes plaintext email. Unity Catalog column masks can transform sensitive values at read time. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect current workspace network access controls and identity sign-in rules. This diagnose action addresses a different case (ip-based workspace restrictions), where a Unity Catalog table grant does not restrict web-console network entry. Workspace access controls and data-object privileges address different security layers. In the stated problem, a raw SELECT on an approved table exposes plaintext email; this choice would not establish the required column masking decision.
C:Inspect consumer business need and raw table grants. This diagnose action addresses a different case (secure view sharing), where granting SELECT on the raw fact table violates minimization. A governed aggregation view can expose only approved derived information. In the stated problem, a raw SELECT on an approved table exposes plaintext email; this choice would not establish the required column masking decision.
D:Inspect ownership and grant-management privileges in production. This diagnose action addresses a different case (separation of duties), where all developers currently own production catalogs and can grant access. Separate deploy and governance roles reduce accidental privilege escalation. In the stated problem, a raw SELECT on an approved table exposes plaintext email; this choice would not establish the required column masking decision.
Question 183
At Redwood Civic Services, a slow query uses a complex chain of joins and projections. The known risk is: Developers tune executor size without inspecting actual exchanges. After remediation, which ongoing signal best detects recurrence?
- Track unit test and integration test failures separately
- Monitor plan regressions that reintroduce Python evaluation
- Track execution-plan regressions for representative heavy queries
- Monitor Python worker execution time and error rates
Explanation
Correct Answer
C:Track execution-plan regressions for representative heavy queries. This watches the failure mode described by developers tune executor size without inspecting actual exchanges. The physical execution plan exposes exchanges, scan filters and join strategy. It supplies a regression signal after remediation.
Incorrect Answers
A:Track unit test and integration test failures separately. This monitor action addresses a different case (unit versus integration tests), where small logic regressions are discovered late and costly to debug. Unit tests isolate transformations while integration tests validate external contracts. In the stated problem, developers tune executor size without inspecting actual exchanges; this choice would not establish the required spark explain plans decision.
B:Monitor plan regressions that reintroduce Python evaluation. This monitor action addresses a different case (avoid python udf for sql), where the Python UDF creates an avoidable cross-language execution boundary. Built-in Spark SQL functions can execute in optimized expressions without Python UDF overhead. In the stated problem, developers tune executor size without inspecting actual exchanges; this choice would not establish the required spark explain plans decision.
D:Monitor Python worker execution time and error rates. This monitor action addresses a different case (vectorized udf), where row-by-row Python invocation dominates stage runtime. Pandas UDFs process batches through Arrow rather than per-row Python calls. In the stated problem, developers tune executor size without inspecting actual exchanges; this choice would not establish the required spark explain plans decision.
Question 184
At Cedar Brook Commerce, a customer-to-events join has one giant default customer ID. A change is required because: A single join partition spills heavily to disk. Which implementation action most directly corrects the configuration?
- Tune shuffle partitions or adaptive coalescing on the affected workload
- Enable or tune adaptive features without disabling correctness checks
- Add supported partition or clustering predicates to MERGE match conditions
- Tune adaptive query execution or implement controlled key salting
Explanation
Correct Answer
D:Tune adaptive query execution or implement controlled key salting. This directly changes the feature or logic behind a single join partition spills heavily to disk. Adaptive skew handling or key salting can mitigate a hot join key. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Tune shuffle partitions or adaptive coalescing on the affected workload. This implement action addresses a different case (shuffle partition count), where the current partition count is too low for data volume. Shuffle partition sizing affects memory pressure and task parallelism. In the stated problem, a single join partition spills heavily to disk; this choice would not establish the required skewed joins decision.
B:Enable or tune adaptive features without disabling correctness checks. This implement action addresses a different case (adaptive execution), where static join strategy performs poorly at varying input volumes. Adaptive Query Execution can adjust supported shuffle partitions and join strategies at runtime. In the stated problem, a single join partition spills heavily to disk; this choice would not establish the required skewed joins decision.
C:Add supported partition or clustering predicates to MERGE match conditions. This implement action addresses a different case (predicate-limited merge), where the merge scans unrelated historical partitions. Constraining merge matches to relevant partitions can reduce unnecessary target scanning. In the stated problem, a single join partition spills heavily to disk; this choice would not establish the required skewed joins decision.
Question 185
At Pinecrest Holdings, a partner outside Databricks needs access to one curated table. Architecture constraint: The partner cannot use an internal Databricks-to-Databricks credential flow. Which design best addresses this requirement?
- Use an open-sharing recipient with restricted data and secure credential delivery
- Choose D2D sharing for supported Databricks managed recipients
- Define a minimal table-level share membership contract
- Plan rotation with a short overlap and controlled delivery path
Explanation
Correct Answer
A:Use an open-sharing recipient with restricted data and secure credential delivery. This design addresses the partner cannot use an internal Databricks-to-Databricks credential flow. Open Delta Sharing can support non-Databricks recipients using the sharing protocol. It establishes the right contract or architecture before execution changes.
Incorrect Answers
B:Choose D2D sharing for supported Databricks managed recipients. This design action addresses a different case (d2d managed recipient), where a recipient team copies files manually and loses timely updates. Databricks-to-Databricks Delta Sharing supports governed live access without manual copying. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.
C:Define a minimal table-level share membership contract. This design action addresses a different case (share catalog scope), where the recipient can discover unintended datasets. A Delta Sharing share should contain only explicitly approved assets. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.
D:Plan rotation with a short overlap and controlled delivery path. This design action addresses a different case (sharing credential rotation), where partners risk losing access or using a leaked long-lived credential. Credential rotation must be coordinated without unnecessarily broadening permissions. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.
Question 186
At Arbor Medical, a Spark transformation produces incorrect null handling after a refactor. The known risk is: Tests only assert that the DataFrame is nonempty. After remediation, which ongoing signal best detects recurrence?
- Monitor new tie cases that would make ordering nondeterministic
- Track coverage and test failures for changed transformation functions
- Monitor out-of-order changes and CDC application errors
- Monitor plan regressions that reintroduce Python evaluation
Explanation
Correct Answer
B:Track coverage and test failures for changed transformation functions. This watches the failure mode described by tests only assert that the DataFrame is nonempty. DataFrame equality and schema assertions test data behavior and contracts. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor new tie cases that would make ordering nondeterministic. This monitor action addresses a different case (window latest record), where the pipeline picks an arbitrary record instead of the newest sequence. ROW_NUMBER ordered by a deterministic sequence within each key selects the desired latest row. In the stated problem, tests only assert that the DataFrame is nonempty; this choice would not establish the required dataframe testing decision.
C:Monitor out-of-order changes and CDC application errors. This monitor action addresses a different case (auto cdc keys), where applying events as append-only rows leaves outdated active accounts. AUTO CDC or applicable APPLY CHANGES needs stable keys and sequence ordering for stateful upserts. In the stated problem, tests only assert that the DataFrame is nonempty; this choice would not establish the required dataframe testing decision.
D:Monitor plan regressions that reintroduce Python evaluation. This monitor action addresses a different case (avoid python udf for sql), where the Python UDF creates an avoidable cross-language execution boundary. Built-in Spark SQL functions can execute in optimized expressions without Python UDF overhead. In the stated problem, tests only assert that the DataFrame is nonempty; this choice would not establish the required dataframe testing decision.
Question 187
At Oak Valley Retail, a Python UDF fails on certain malformed nested payloads. Architecture constraint: Scattered print statements do not reveal the failing object shape. Which design best addresses this requirement?
- Select a supported wheel build for the execution environment
- Use lower and trim native expressions for string standardization
- Isolate parsing logic so it can be debugged outside distributed execution
- Refactor business rules as pure DataFrame transform functions
Explanation
Correct Answer
C:Isolate parsing logic so it can be debugged outside distributed execution. This design addresses scattered print statements do not reveal the failing object shape. Debugger breakpoints and targeted local repros help inspect code path and values. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Select a supported wheel build for the execution environment. This design action addresses a different case (wheel abi compatibility), where the wheel works on a developer laptop but fails with a binary compatibility error. Compiled extensions must match the target runtime platform and Python ABI. In the stated problem, scattered print statements do not reveal the failing object shape; this choice would not establish the required notebook debugger decision.
B:Use lower and trim native expressions for string standardization. This design action addresses a different case (avoid python udf for sql), where the Python UDF creates an avoidable cross-language execution boundary. Built-in Spark SQL functions can execute in optimized expressions without Python UDF overhead. In the stated problem, scattered print statements do not reveal the failing object shape; this choice would not establish the required notebook debugger decision.
D:Refactor business rules as pure DataFrame transform functions. This design action addresses a different case (transform composability), where business logic depends on notebook state and widgets. DataFrame.transform supports composable DataFrame-to-DataFrame functions. In the stated problem, scattered print statements do not reveal the failing object shape; this choice would not establish the required notebook debugger decision.
Question 188
A deployment owner at Westhaven Services needs to implement a remedy: a new streaming pipeline revision causes wrong customer totals. The present failure is: Deployers have no record of the previous working configuration. Which change should be applied?
- Deploy using an explicit reviewed production target
- Run databricks bundle validate for the intended target in CI
- Connect a Git folder and deploy from an approved commit
- Deploy the previous approved bundle revision with correct target
Explanation
Correct Answer
D:Deploy the previous approved bundle revision with correct target. This directly changes the feature or logic behind deployers have no record of the previous working configuration. Versioned bundle releases enable controlled rollback to a known commit. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Deploy using an explicit reviewed production target. This implement action addresses a different case (target-safe deployment), where the wrong workspace is selected because target is implicit. Explicit bundle target selection prevents accidental environment deployment. In the stated problem, deployers have no record of the previous working configuration; this choice would not establish the required rollback deployment decision.
B:Run databricks bundle validate for the intended target in CI. This implement action addresses a different case (bundle validation), where an invalid resource key would be rejected only after production change. Bundle validation detects many configuration problems before deployment. In the stated problem, deployers have no record of the previous working configuration; this choice would not establish the required rollback deployment decision.
C:Connect a Git folder and deploy from an approved commit. This implement action addresses a different case (git folder collaboration), where there is no controlled code review or rollback history. Databricks Git folders connect workspace development to version control. In the stated problem, deployers have no record of the previous working configuration; this choice would not establish the required rollback deployment decision.
Question 189
At Northfield Analytics, a SQL workload spends substantial CPU executing supported relational operators. Observed issue: An unsupported execution mode limits vectorized engine use. Before applying a fix, which investigation is most directly relevant?
- Inspect compute capabilities and supported physical plan operators
- Inspect executed plan for AQE decisions and stage statistics
- Inspect managed table eligibility and current optimization settings
- Inspect repeated lineage and memory availability for the reference data
Explanation
Correct Answer
A:Inspect compute capabilities and supported physical plan operators. This checks the specific evidence behind an unsupported execution mode limits vectorized engine use. Photon can accelerate supported SQL and DataFrame operators on compatible compute. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
B:Inspect executed plan for AQE decisions and stage statistics. This diagnose action addresses a different case (adaptive execution), where static join strategy performs poorly at varying input volumes. Adaptive Query Execution can adjust supported shuffle partitions and join strategies at runtime. In the stated problem, an unsupported execution mode limits vectorized engine use; this choice would not establish the required photon query execution decision.
C:Inspect managed table eligibility and current optimization settings. This diagnose action addresses a different case (predictive optimization), where operators run blanket manual OPTIMIZE jobs regardless of table use. Predictive optimization can automate supported managed-table maintenance based on workload signals. In the stated problem, an unsupported execution mode limits vectorized engine use; this choice would not establish the required photon query execution decision.
D:Inspect repeated lineage and memory availability for the reference data. This diagnose action addresses a different case (cache reuse), where repeated recomputation dominates each branch's execution. Persistence can avoid repeated work if reused data fits the memory budget. In the stated problem, an unsupported execution mode limits vectorized engine use; this choice would not establish the required photon query execution decision.
Question 190
A deployment owner at Lakewood Distribution needs to implement a remedy: a CI pipeline is about to deploy modified Lakeflow Jobs YAML. The present failure is: An invalid resource key would be rejected only after production change. Which change should be applied?
- Deploy using an explicit reviewed production target
- Run databricks bundle validate for the intended target in CI
- Pin or rebuild incompatible wheels and update the bundle library
- Deploy the previous approved bundle revision with correct target
Explanation
Correct Answer
B:Run databricks bundle validate for the intended target in CI. This directly changes the feature or logic behind an invalid resource key would be rejected only after production change. Bundle validation detects many configuration problems before deployment. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Deploy using an explicit reviewed production target. This implement action addresses a different case (target-safe deployment), where the wrong workspace is selected because target is implicit. Explicit bundle target selection prevents accidental environment deployment. In the stated problem, an invalid resource key would be rejected only after production change; this choice would not establish the required bundle validation decision.
C:Pin or rebuild incompatible wheels and update the bundle library. This implement action addresses a different case (python dependency triage), where a package import error appears only on the target compute. Runtime-specific wheel compatibility and pinning must be tested in CI. In the stated problem, an invalid resource key would be rejected only after production change; this choice would not establish the required bundle validation decision.
D:Deploy the previous approved bundle revision with correct target. This implement action addresses a different case (rollback deployment), where deployers have no record of the previous working configuration. Versioned bundle releases enable controlled rollback to a known commit. In the stated problem, an invalid resource key would be rejected only after production change; this choice would not establish the required bundle validation decision.