Databricks Certified Data Engineer Professional Practice Test 46
Question 450
At Westhaven Services, a daily Delta MERGE updates a tiny set of dates in a massive fact table. Architecture constraint: The merge scans unrelated historical partitions. Which design best addresses this requirement?
- Keep filter predicates aligned with the stored partition column
- Use source-date scope and target predicate pruning without changing match semantics
- Choose supported liquid clustering on commonly filtered device fields
- Choose a supported columnar Delta or Parquet layout for analytics
Explanation
Correct Answer
B:Use source-date scope and target predicate pruning without changing match semantics. This design addresses the merge scans unrelated historical partitions. Constraining merge matches to relevant partitions can reduce unnecessary target scanning. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Keep filter predicates aligned with the stored partition column. This design action addresses a different case (partition pruning), where a query wraps business_date in a transformation and reads many partitions. Direct predicates on partition keys can enable partition pruning. In the stated problem, the merge scans unrelated historical partitions; this choice would not establish the required predicate-limited merge decision.
C:Choose supported liquid clustering on commonly filtered device fields. This design action addresses a different case (liquid clustering cardinality), where partitioning by device would produce many tiny partitions. Liquid clustering organizes data by useful filter keys without high-cardinality partition directories. In the stated problem, the merge scans unrelated historical partitions; this choice would not establish the required predicate-limited merge decision.
D:Choose a supported columnar Delta or Parquet layout for analytics. This design action addresses a different case (columnar compression), where row-oriented text storage increases scans and transfer volumes. Columnar formats and appropriate encoding improve analytical read efficiency. In the stated problem, the merge scans unrelated historical partitions; this choice would not establish the required predicate-limited merge decision.
Question 451
At Northfield Analytics, a production task can read source tables but fails writing its silver output. The known risk is: The run-as principal lacks MODIFY on the target table. After remediation, which ongoing signal best detects recurrence?
- Monitor CI authentication failures and principal grants
- Monitor duplicate delivery IDs and retry attempts
- Audit repeated authorization errors and new broad grants
- Audit deployment target changes and cross-environment drift
Explanation
Correct Answer
C:Audit repeated authorization errors and new broad grants. This watches the failure mode described by the run-as principal lacks MODIFY on the target table. Unity Catalog authorization applies to the job execution identity. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor CI authentication failures and principal grants. This monitor action addresses a different case (api deployment authentication), where scheduled production deployment fails when the developer's token expires. Workload identity or service principal authentication decouples CI from employees. In the stated problem, the run-as principal lacks MODIFY on the target table; this choice would not establish the required job run permission debug decision.
B:Monitor duplicate delivery IDs and retry attempts. This monitor action addresses a different case (idempotent retry repair), where customers receive repeated notifications for the same batch. Task retries need idempotent side effects even when upstream work is reused. In the stated problem, the run-as principal lacks MODIFY on the target table; this choice would not establish the required job run permission debug decision.
D:Audit deployment target changes and cross-environment drift. This monitor action addresses a different case (target-safe deployment), where the wrong workspace is selected because target is implicit. Explicit bundle target selection prevents accidental environment deployment. In the stated problem, the run-as principal lacks MODIFY on the target table; this choice would not establish the required job run permission debug decision.
Question 452
At Fairhaven Transport, a finance team needs daily attribution of Databricks compute charges by workload. The known risk is: Cost reports omit some job and workspace labels. After remediation, which ongoing signal best detects recurrence?
- Alert on freshness lag rather than only job failure
- Monitor alert evaluations and delivery failures
- Alert on unusual sensitive-object access patterns
- Alert on cost spikes and untagged spend
Explanation
Correct Answer
D:Alert on cost spikes and untagged spend. This watches the failure mode described by cost reports omit some job and workspace labels. Databricks system billing tables can expose usage records for cost attribution. It supplies a regression signal after remediation.
Incorrect Answers
A:Alert on freshness lag rather than only job failure. This monitor action addresses a different case (end-to-end sla), where operators monitor compute uptime rather than data freshness. SLA monitoring should measure source event time to published table availability. In the stated problem, cost reports omit some job and workspace labels; this choice would not establish the required billing usage monitoring decision.
B:Monitor alert evaluations and delivery failures. This monitor action addresses a different case (sql alert threshold), where the current dashboard has no automated alert for missing units. SQL alerts can evaluate a query result against a configured threshold. In the stated problem, cost reports omit some job and workspace labels; this choice would not establish the required billing usage monitoring decision.
C:Alert on unusual sensitive-object access patterns. This monitor action addresses a different case (audit log review), where workspace notebook logs do not show the full audited account activity. System audit tables provide security-relevant account events subject to access controls. In the stated problem, cost reports omit some job and workspace labels; this choice would not establish the required billing usage monitoring decision.
Question 453
At Westridge Research, a Python UDF transforms millions of numeric rows with substantial serialization overhead. A change is required because: Row-by-row Python invocation dominates stage runtime. Which implementation action most directly corrects the configuration?
- Replace a scalar Python UDF with supported column expressions or a Pandas UDF
- Include the unique tie-breaker in the supported sequence expression
- Apply watermark before supported stateful aggregation with matching event-time column
- Configure the supported SCD Type 2 flow with keys and sequence
Explanation
Correct Answer
A:Replace a scalar Python UDF with supported column expressions or a Pandas UDF. This directly changes the feature or logic behind row-by-row Python invocation dominates stage runtime. Pandas UDFs process batches through Arrow rather than per-row Python calls. It is the concrete configuration for the stated incident.
Incorrect Answers
B:Include the unique tie-breaker in the supported sequence expression. This implement action addresses a different case (cdc duplicate sequence), where the processing order for tied source events is nondeterministic. A deterministic sequencing contract needs a unique ordered event position. In the stated problem, row-by-row Python invocation dominates stage runtime; this choice would not establish the required vectorized udf decision.
C:Apply watermark before supported stateful aggregation with matching event-time column. This implement action addresses a different case (event-time watermarking), where old keys remain in state indefinitely without a lateness bound. Watermarks provide a lateness threshold for bounded stateful streaming operations. In the stated problem, row-by-row Python invocation dominates stage runtime; this choice would not establish the required vectorized udf decision.
D:Configure the supported SCD Type 2 flow with keys and sequence. This implement action addresses a different case (scd type 2 history), where overwrite-style updates destroy historical state needed for point-in-time reporting. SCD Type 2 records effective history instead of replacing the previous version. In the stated problem, row-by-row Python invocation dominates stage runtime; this choice would not establish the required vectorized udf decision.
Question 454
Quality assurance at Cobalt Biotech checks a corrected workload where a table is partitioned by business_date. The original fault was: A query wraps business_date in a transformation and reads many partitions. Which verification is most decisive?
- Run representative device filters and compare data skipping
- Confirm partition scan count falls with equal query results
- Verify shuffle spill decreases without excessive scheduling overhead
- Compare results and max task time before and after
Explanation
Correct Answer
B:Confirm partition scan count falls with equal query results. This tests the expected result against a query wraps business_date in a transformation and reads many partitions, not simply whether a job or object exists. Direct predicates on partition keys can enable partition pruning.
Incorrect Answers
A:Run representative device filters and compare data skipping. This validate action addresses a different case (liquid clustering cardinality), where partitioning by device would produce many tiny partitions. Liquid clustering organizes data by useful filter keys without high-cardinality partition directories. In the stated problem, a query wraps business_date in a transformation and reads many partitions; this choice would not establish the required partition pruning decision.
C:Verify shuffle spill decreases without excessive scheduling overhead. This validate action addresses a different case (shuffle partition count), where the current partition count is too low for data volume. Shuffle partition sizing affects memory pressure and task parallelism. In the stated problem, a query wraps business_date in a transformation and reads many partitions; this choice would not establish the required partition pruning decision.
D:Compare results and max task time before and after. This validate action addresses a different case (skewed joins), where a single join partition spills heavily to disk. Adaptive skew handling or key salting can mitigate a hot join key. In the stated problem, a query wraps business_date in a transformation and reads many partitions; this choice would not establish the required partition pruning decision.
Question 455
At Cedar Brook Commerce, operators change a production job setting in the UI after deployment. Observed issue: Subsequent CI redeployment unexpectedly overwrites it. Before applying a fix, which investigation is most directly relevant?
- Inspect the bundle manifest schema and validation result
- Inspect CI stages for tests, bundle validation and approvals
- Compare live job settings with the committed bundle specification
- Inspect effective grants on source, target and parent containers for the run-as principal
Explanation
Correct Answer
C:Compare live job settings with the committed bundle specification. This checks the specific evidence behind subsequent CI redeployment unexpectedly overwrites it. Declarative source of truth should include intentional job-resource changes. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect the bundle manifest schema and validation result. This diagnose action addresses a different case (bundle validation), where an invalid resource key would be rejected only after production change. Bundle validation detects many configuration problems before deployment. In the stated problem, subsequent CI redeployment unexpectedly overwrites it; this choice would not establish the required unmanaged resource drift decision.
B:Inspect CI stages for tests, bundle validation and approvals. This diagnose action addresses a different case (ci quality gates), where the deployment pipeline only checks YAML syntax. CI must run transformation tests and controlled integration checks before deployment. In the stated problem, subsequent CI redeployment unexpectedly overwrites it; this choice would not establish the required unmanaged resource drift decision.
D:Inspect effective grants on source, target and parent containers for the run-as principal. This diagnose action addresses a different case (job run permission debug), where the run-as principal lacks MODIFY on the target table. Unity Catalog authorization applies to the job execution identity. In the stated problem, subsequent CI redeployment unexpectedly overwrites it; this choice would not establish the required unmanaged resource drift decision.
Question 456
At Redwood Civic Services, a declarative pipeline needs near-real-time freshness. Observed issue: Scheduled updates are spaced several hours apart. Before applying a fix, which investigation is most directly relevant?
- Inspect event log messages, error class and dataset name around the failed update
- Inspect ingestion timestamps, pipeline refresh times and gold table commit times
- Inspect expectation metric counts in the event log
- Inspect schedule and last successful update timestamps
Explanation
Correct Answer
D:Inspect schedule and last successful update timestamps. This checks the specific evidence behind scheduled updates are spaced several hours apart. Pipeline update frequency determines available end-to-end freshness. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect event log messages, error class and dataset name around the failed update. This diagnose action addresses a different case (pipeline event log), where the Jobs status only reports a high-level failure. Lakeflow pipeline event logs include update-level errors and progress events. In the stated problem, scheduled updates are spaced several hours apart; this choice would not establish the required pipeline update cadence decision.
B:Inspect ingestion timestamps, pipeline refresh times and gold table commit times. This diagnose action addresses a different case (end-to-end sla), where operators monitor compute uptime rather than data freshness. SLA monitoring should measure source event time to published table availability. In the stated problem, scheduled updates are spaced several hours apart; this choice would not establish the required pipeline update cadence decision.
C:Inspect expectation metric counts in the event log. This diagnose action addresses a different case (expectation monitoring), where analysts confuse successful pipeline status with zero rejected records. Quality metric events reveal input and dropped records separately from update status. In the stated problem, scheduled updates are spaced several hours apart; this choice would not establish the required pipeline update cadence decision.
Question 457
Operations at Pinecrest Holdings owns a workload where a compliance export should exclude free-text notes containing PII. The previously identified issue was: A broad SELECT still includes unstructured notes. Which metric or audit check best detects drift?
- Monitor view definition changes and access to raw source
- Audit unauthorized changes to workspace objects
- Audit row filter changes and unexpectedly broad query results
- Audit ownership changes and privilege grants
Explanation
Correct Answer
A:Monitor view definition changes and access to raw source. This watches the failure mode described by a broad SELECT still includes unstructured notes. Column suppression and data minimization prevent unnecessary sensitive-field release. It supplies a regression signal after remediation.
Incorrect Answers
B:Audit unauthorized changes to workspace objects. This monitor action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, a broad SELECT still includes unstructured notes; this choice would not establish the required sensitive data suppression decision.
C:Audit row filter changes and unexpectedly broad query results. This monitor action addresses a different case (unity catalog row filter), where a shared table currently returns all regions to every reader. Unity Catalog row filters can restrict row visibility through governed policies. In the stated problem, a broad SELECT still includes unstructured notes; this choice would not establish the required sensitive data suppression decision.
D:Audit ownership changes and privilege grants. This monitor action addresses a different case (separation of duties), where all developers currently own production catalogs and can grant access. Separate deploy and governance roles reduce accidental privilege escalation. In the stated problem, a broad SELECT still includes unstructured notes; this choice would not establish the required sensitive data suppression decision.
Question 458
At Westridge Research, a sales fact table mixes order-level rows with order-line-level rows. Architecture constraint: Summing revenue double-counts orders with multiple items. Which design best addresses this requirement?
- Model Type 2 records with effective intervals and current-row flag
- Choose one order-line grain for transaction detail metrics
- Create a gold aggregate keyed by day and product with documented measures
- Model one order-line fact with product, store and date dimensions
Explanation
Correct Answer
B:Choose one order-line grain for transaction detail metrics. This design addresses summing revenue double-counts orders with multiple items. A dimensional fact must have one clearly documented row grain. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Model Type 2 records with effective intervals and current-row flag. This design action addresses a different case (scd type 2 effective dates), where a single current tier overwrites history in the dimension. SCD Type 2 stores effective start/end intervals for distinct historical dimension versions. In the stated problem, summing revenue double-counts orders with multiple items; this choice would not establish the required fact table grain decision.
C:Create a gold aggregate keyed by day and product with documented measures. This design action addresses a different case (aggregate gold model), where interactive queries scan billions of detail rows for identical totals. A summarized gold table can serve a stable reporting grain efficiently. In the stated problem, summing revenue double-counts orders with multiple items; this choice would not establish the required fact table grain decision.
D:Model one order-line fact with product, store and date dimensions. This design action addresses a different case (star schema analytics), where a denormalized mixed-grain table produces redundant attributes and expensive corrections. A star schema links a clear-grain fact to reusable dimensions for consistent analytics. In the stated problem, summing revenue double-counts orders with multiple items; this choice would not establish the required fact table grain decision.
Question 459
At Bayview Insurance, a production workflow runs as the notebook author's personal identity. A change is required because: An employee departure causes permissions failures during scheduled runs. Which implementation action most directly corrects the configuration?
- Set explicit target overrides and deploy using the approved production target
- Define a streaming table reading the source through a supported streaming connector
- Assign the approved run-as identity and reconfigure required grants
- Restore the approved checkpoint path in the query deployment
Explanation
Correct Answer
C:Assign the approved run-as identity and reconfigure required grants. This directly changes the feature or logic behind an employee departure causes permissions failures during scheduled runs. A stable least-privilege service principal can provide durable job identity. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Set explicit target overrides and deploy using the approved production target. This implement action addresses a different case (bundle target overrides), where development job arguments were deployed into the production target. Bundle targets can override environment-specific variables and resource settings. In the stated problem, an employee departure causes permissions failures during scheduled runs; this choice would not establish the required job run-as identity decision.
B:Define a streaming table reading the source through a supported streaming connector. This implement action addresses a different case (streaming table choice), where periodic full-refresh queries miss required incremental freshness. A streaming table tracks continuously arriving source data incrementally. In the stated problem, an employee departure causes permissions failures during scheduled runs; this choice would not establish the required job run-as identity decision.
D:Restore the approved checkpoint path in the query deployment. This implement action addresses a different case (streaming checkpoint identity), where a newly selected checkpoint location makes the stream reread historical input. Streaming checkpoint metadata records progress and state for restart continuity. In the stated problem, an employee departure causes permissions failures during scheduled runs; this choice would not establish the required job run-as identity decision.
Question 460
Quality assurance at Lakewood Distribution checks a corrected workload where a new workspace team needs read-only access to a vendor landing bucket. The original fault was: Engineers propose a credential allowing access to all company buckets. Which verification is most decisive?
- Query as two regional users and confirm different allowed row sets
- Confirm deployment works while a developer cannot grant themselves table access
- Confirm reader cannot edit but approved maintainer can deploy
- Confirm allowed source reads succeed and unrelated prefixes are denied
Explanation
Correct Answer
D:Confirm allowed source reads succeed and unrelated prefixes are denied. This tests the expected result against engineers propose a credential allowing access to all company buckets, not simply whether a job or object exists. Unity Catalog storage credentials and external locations can constrain storage access.
Incorrect Answers
A:Query as two regional users and confirm different allowed row sets. This validate action addresses a different case (unity catalog row filter), where a shared table currently returns all regions to every reader. Unity Catalog row filters can restrict row visibility through governed policies. In the stated problem, engineers propose a credential allowing access to all company buckets; this choice would not establish the required storage credential scope decision.
B:Confirm deployment works while a developer cannot grant themselves table access. This validate action addresses a different case (separation of duties), where all developers currently own production catalogs and can grant access. Separate deploy and governance roles reduce accidental privilege escalation. In the stated problem, engineers propose a credential allowing access to all company buckets; this choice would not establish the required storage credential scope decision.
C:Confirm reader cannot edit but approved maintainer can deploy. This validate action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, engineers propose a credential allowing access to all company buckets; this choice would not establish the required storage credential scope decision.