Databricks Certified Data Engineer Associate Practice Test 60
Question 590
Consider the artifact_metadata_updates_metro workload at Regional museum alliance: artifact_metadata_updates_metro delta target must apply corrected invoice amounts from an incremental feed. Inspection found that the source has two updates for the same invoice_id in one batch. The governed outcome must ensure that target must have one latest value per invoice after applying ordered changes. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Identify stable merge keys and deduplicate multiple updates in the batch for artifact_metadata_updates_metro
- Use Delta MERGE with a deduplicated source on invoice_id and matched update logic
- MERGE with multiple unfiltered source matches per target key at Regional museum alliance
- Test MERGE on a small target and rerun an identical source batch
Explanation
Correct Answer
A:Identify stable merge keys and deduplicate multiple updates in the batch for artifact_metadata_updates_metro directly investigates the evidence (the source has two updates for the same invoice_id in one batch) and distinguishes possible causes of artifact_metadata_updates_metro delta target must apply corrected invoice amounts from an incremental feed. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: target must have one latest value per invoice after applying ordered changes.
Incorrect Answers
B:Use Delta MERGE with a deduplicated source on invoice_id and matched update logic is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the artifact_metadata_updates_metro scenario, the observed evidence is that the source has two updates for the same invoice_id in one batch; the requirement remains: target must have one latest value per invoice after applying ordered changes.
C:Ambiguous multiple source matches can make updates nondeterministic or fail. The proposed action (MERGE with multiple unfiltered source matches per target key at Regional museum alliance) does not meet the requirement that target must have one latest value per invoice after applying ordered changes, nor does it constitute the required pre-change investigation.
D:Test MERGE on a small target and rerun an identical source batch may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that the source has two updates for the same invoice_id in one batch. This matters here because consider the artifact_metadata_updates_metro workload at Regional museum alliance: artifact_metadata_updates_metro….
Question 591
The clinical_appointment_reports_east pipeline at Regional hospital network has this failure: clinical_appointment_reports_east job runs hourly although only one file arrives most days. The team observed that execution history shows repeated empty runs with compute startup cost. Business requirement: process each new delivered file promptly without idle polling. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What should the engineer inspect to establish the failure mechanism?
- Use a supported file-arrival trigger for the eligible storage location
- Inspect file arrival pattern, supported trigger source and job startup overhead for clinical_appointment_reports_east
- Enable trigger in staging and test a new object arrival
- Run a full pipeline every minute even if no files arrive at Regional hospital network
Explanation
Correct Answer
B:Inspect file arrival pattern, supported trigger source and job startup overhead for clinical_appointment_reports_east directly investigates the evidence (execution history shows repeated empty runs with compute startup cost) and distinguishes possible causes of clinical_appointment_reports_east job runs hourly although only one file arrives most days. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: process each new delivered file promptly without idle polling.
Incorrect Answers
A:Use a supported file-arrival trigger for the eligible storage location is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the clinical_appointment_reports_east scenario, the observed evidence is that execution history shows repeated empty runs with compute startup cost; the requirement remains: process each new delivered file promptly without idle polling.
C:Enable trigger in staging and test a new object arrival may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that execution history shows repeated empty runs with compute startup cost.
D:Repetitive empty executions waste compute and cost. The proposed action (Run a full pipeline every minute even if no files arrive at Regional hospital network) does not meet the requirement that process each new delivered file promptly without idle polling, nor does it constitute the required pre-change investigation.
Question 592
Consider the pipeline_flow_metrics_west workload at Public water district: pipeline_flow_metrics_west Delta table suffers frequent filter-pattern changes and costly partition rewrites. Inspection found that workload history alternates date and customer search predicates across teams. The governed outcome must ensure that layout optimization must adapt without a rigid proliferating partition tree. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. How should the team verify the original fault has been addressed?
- Create thousands of tiny static partitions for every possible predicate at Public water district
- Enable clustering on a pilot target and run supported maintenance operations
- Compare query scan size and update overhead for representative filter sets
- Profile common filters, table size and clustering support for pipeline_flow_metrics_west
Explanation
Correct Answer
C:Compare query scan size and update overhead for representative filter sets directly tests the behavior that should change following the correction. With the observed evidence (workload history alternates date and customer search predicates across teams), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that layout optimization must adapt without a rigid proliferating partition tree.
Incorrect Answers
A:Overpartitioning drives metadata overhead and inflexible layout. The proposed action (Create thousands of tiny static partitions for every possible predicate at Public water district) does not meet the requirement that layout optimization must adapt without a rigid proliferating partition tree, nor does it constitute the post-change acceptance test.
B:Enable clustering on a pilot target and run supported maintenance operations may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that workload history alternates date and customer search predicates across teams. This matters here because consider the pipeline_flow_metrics_west workload at Public water district: pipeline_flow_metrics_west Delta table….
D:Profile common filters, table size and clustering support for pipeline_flow_metrics_west could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The pipeline_flow_metrics_west workflow still requires that layout optimization must adapt without a rigid proliferating partition tree.
Question 593
A data engineer at Research clinical trial group is responsible for consent-related data changes. The broadcast small dimension issue is that joining consent_related_data_changes_delta fact records to a tiny reference table produces a large shuffle. The supporting evidence is that Spark UI shows both inputs shuffled despite one lookup occupying a few megabytes. The engineer must ensure that join output must retain key equality and correct multiplicity. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What is the safest migration or deployment sequence?
- CrossJoin the entire fact and reference datasets then filter after collection at Research clinical trial group
- Compare join counts and stage shuffle bytes with and without the hint
- Broadcast the eligible small reference DataFrame when appropriate to avoid a large shuffle
- Trial a broadcast hint with representative fact cardinality and verify output
Explanation
Correct Answer
D:Trial a broadcast hint with representative fact cardinality and verify output provides a bounded way to introduce the approved solution into consent_related_data_changes_delta and detect an early mismatch. The existing signal is Spark UI shows both inputs shuffled despite one lookup occupying a few megabytes; a staged change protects the service requirement (join output must retain key equality and correct multiplicity) and makes rollback practical.
Incorrect Answers
A:A cartesian expansion wastes resources and risks mis-joins. The proposed action (CrossJoin the entire fact and reference datasets then filter after collection at Research clinical trial group) does not meet the requirement that join output must retain key equality and correct multiplicity, nor does it constitute the controlled implementation step.
B:Compare join counts and stage shuffle bytes with and without the hint is a one-time acceptance check. It does not supply the controlled implementation step for consent_related_data_changes_delta; a passing test cannot substitute for the specific task now required.
C:Broadcast the eligible small reference DataFrame when appropriate to avoid a large shuffle is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the consent_related_data_changes_delta scenario, the observed evidence is that Spark UI shows both inputs shuffled despite one lookup occupying a few megabytes; the requirement remains: join output must retain key equality and correct multiplicity.
Question 594
At Coastal freight alliance, the shipment manifests workflow depends on shipment_manifests_central. Engineers report that the shipment_manifests_central SQL aggregation runs slowly on eligible compute despite a stable dataset; the evidence is that query profiling identifies CPU-bound SQL operations with little external I/O. They require that performance changes must be measured at equivalent compute and data volume. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. Which platform or PySpark choice best fits the stated semantics?
- Evaluate Photon-capable compute for supported SQL and DataFrame workloads
- Track query latency and cost-per-run after moving shipment_manifests_central
- Assume Photon eliminates every network, skew and access-permission problem at Coastal freight alliance
- Baseline SQL plan, engine configuration and resource use for shipment_manifests_central
Explanation
Correct Answer
A:Evaluate Photon-capable compute for supported SQL and DataFrame workloads is the intended durable approach for shipment_manifests_central: it addresses the verified problem (the shipment_manifests_central SQL aggregation runs slowly on eligible compute despite a stable dataset) while preserving the required behavior (performance changes must be measured at equivalent compute and data volume). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
B:Track query latency and cost-per-run after moving shipment_manifests_central would help detect future recurrence, but metrics alone do not deliver the target approach. The shipment_manifests_central requirement is that performance changes must be measured at equivalent compute and data volume.
C:An execution engine cannot cure unrelated input or authorization constraints. The proposed action (Assume Photon eliminates every network, skew and access-permission problem at Coastal freight alliance) does not meet the requirement that performance changes must be measured at equivalent compute and data volume, nor does it constitute the target approach.
D:Baseline SQL plan, engine configuration and resource use for shipment_manifests_central could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The shipment_manifests_central workflow still requires that performance changes must be measured at equivalent compute and data volume.
Question 595
At Air quality nonprofit, the Auto Loader schema evolution workflow handles pollution measurements. The problem is that a new optional field appears in pollution_measurements_central JSON files and the ingestion job starts rejecting rows. Evidence: failure details identify a field absent from the previously inferred schema. Constraint: new attributes must be captured without silently corrupting existing columns. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which monitoring approach will detect newly emerging defects?
- Ingest old and new records and verify preserved fields and captured additions
- Configure supported Auto Loader schema evolution with an explicit schema location and review process
- Monitor schema changes, rescued columns and parsing failures for pollution_measurements_central
- Silently drop every unexpected field and label ingestion complete at Air quality nonprofit
Explanation
Correct Answer
C:Monitor schema changes, rescued columns and parsing failures for pollution_measurements_central creates recurring visibility into the failure mode (a new optional field appears in pollution_measurements_central JSON files and the ingestion job starts rejecting rows). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while new attributes must be captured without silently corrupting existing columns.
Incorrect Answers
A:Ingest old and new records and verify preserved fields and captured additions is a one-time acceptance check. It does not supply continuing operational detection for pollution_measurements_central; a passing test cannot substitute for the specific task now required.
B:Configure supported Auto Loader schema evolution with an explicit schema location and review process is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the pollution_measurements_central scenario, the observed evidence is that failure details identify a field absent from the previously inferred schema; the requirement remains: new attributes must be captured without silently corrupting existing columns.
D:Dropping added fields hides source changes and loses information. The proposed action (Silently drop every unexpected field and label ingestion complete at Air quality nonprofit) does not meet the requirement that new attributes must be captured without silently corrupting existing columns, nor does it constitute continuing operational detection.
Question 596
At Maritime safety agency, the gold streaming table workflow handles vessel position events. The problem is that vessel_position_events_central gold consumers need fresh incremental data from an append-only event feed. Evidence: the source produces new events continuously while downstream reports lag by hours. Constraint: materialized output must incorporate new events with recoverable processing state. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which change can be introduced with minimal production risk?
- Append new source events and verify exactly the required incremental result
- Build a governed streaming table for continuous or triggered incremental append processing
- Truncate and fully rebuild the target after every incoming event at Maritime safety agency
- Deploy a streaming table with tested recovery checkpoints and refresh configuration
Explanation
Correct Answer
D:Deploy a streaming table with tested recovery checkpoints and refresh configuration provides a bounded way to introduce the approved solution into vessel_position_events_central and detect an early mismatch. The existing signal is the source produces new events continuously while downstream reports lag by hours; a staged change protects the service requirement (materialized output must incorporate new events with recoverable processing state) and makes rollback practical.
Incorrect Answers
A:Append new source events and verify exactly the required incremental result is a one-time acceptance check. It does not supply the controlled implementation step for vessel_position_events_central; a passing test cannot substitute for the specific task now required.
B:Build a governed streaming table for continuous or triggered incremental append processing is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the vessel_position_events_central scenario, the observed evidence is that the source produces new events continuously while downstream reports lag by hours; the requirement remains: materialized output must incorporate new events with recoverable processing state.
C:Full rebuilds are expensive and defeat incremental freshness goals. The proposed action (Truncate and fully rebuild the target after every incoming event at Maritime safety agency) does not meet the requirement that materialized output must incorporate new events with recoverable processing state, nor does it constitute the controlled implementation step.
Question 597
At National farm supplier, the conditional flag when otherwise workflow handles harvest purchase orders. The problem is that harvest_purchase_orders_metro rows have statuses requiring a three-way risk category. Evidence: a simple boolean expression sends null status to a high-risk category. Constraint: business rules distinguish active, cancelled and unknown statuses. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which investigation is the strongest first step?
- Use when/otherwise with explicit conditions and an unknown default
- Treat null as an active subscription without evidence at National farm supplier
- Deploy a deterministic mapping table and validate the input states
- Enumerate source states and null handling for the harvest_purchase_orders_metro flag
Explanation
Correct Answer
D:Enumerate source states and null handling for the harvest_purchase_orders_metro flag directly investigates the evidence (a simple boolean expression sends null status to a high-risk category) and distinguishes possible causes of harvest_purchase_orders_metro rows have statuses requiring a three-way risk category. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: business rules distinguish active, cancelled and unknown statuses.
Incorrect Answers
A:Use when/otherwise with explicit conditions and an unknown default is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the harvest_purchase_orders_metro scenario, the observed evidence is that a simple boolean expression sends null status to a high-risk category; the requirement remains: business rules distinguish active, cancelled and unknown statuses.
B:Missing status cannot be assumed to mean active. The proposed action (Treat null as an active subscription without evidence at National farm supplier) does not meet the requirement that business rules distinguish active, cancelled and unknown statuses, nor does it constitute the required pre-change investigation.
C:Deploy a deterministic mapping table and validate the input states may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that a simple boolean expression sends null status to a high-risk category. Here, the relevant situation is: the conditional flag when otherwise workflow handles harvest purchase orders. The problem is that….
Question 598
The dispatch_and_delivery_status_highland pipeline at Food delivery franchise has this failure: dispatch_and_delivery_status_highland sort aggregation spills gigabytes of intermediates to disk. The team observed that stage metrics show spilled bytes and GC pressure on executors. Business requirement: the job should improve without losing aggregation accuracy. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What should the engineer inspect to establish the failure mechanism?
- Trial an adjusted executor configuration on identical data volume
- Compare per-task memory, partition sizes and spill metrics for dispatch_and_delivery_status_highland
- Disable logs and assume spilled data was never written at Food delivery franchise
- Right-size memory or repartition workload based on measured per-task pressure
Explanation
Correct Answer
B:Compare per-task memory, partition sizes and spill metrics for dispatch_and_delivery_status_highland directly investigates the evidence (stage metrics show spilled bytes and GC pressure on executors) and distinguishes possible causes of dispatch_and_delivery_status_highland sort aggregation spills gigabytes of intermediates to disk. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: the job should improve without losing aggregation accuracy.
Incorrect Answers
A:Trial an adjusted executor configuration on identical data volume may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that stage metrics show spilled bytes and GC pressure on executors. This matters here because the dispatch_and_delivery_status_highland pipeline at Food delivery franchise has this failure:….
C:Missing telemetry cannot correct memory pressure. The proposed action (Disable logs and assume spilled data was never written at Food delivery franchise) does not meet the requirement that the job should improve without losing aggregation accuracy, nor does it constitute the required pre-change investigation.
D:Right-size memory or repartition workload based on measured per-task pressure is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the dispatch_and_delivery_status_highland scenario, the observed evidence is that stage metrics show spilled bytes and GC pressure on executors; the requirement remains: the job should improve without losing aggregation accuracy.
Question 599
A data engineer at Airport facilities operator is responsible for maintenance work orders. The Silver null cleaning issue is that maintenance_work_orders_north bronze records include null business IDs that break downstream joins. The supporting evidence is that profiling identifies a small fraction of missing identifiers plus valid optional fields. The engineer must ensure that silver must reject missing required keys but preserve optional null attributes. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What is the safest migration or deployment sequence?
- Verify no null required IDs remain and legitimate optional-null records survive
- Create a tested silver transform and quarantine invalid rows before promoting
- Filter or quarantine null required IDs while applying explicit nullable rules to optional columns
- Call fillna on every field with zero regardless of semantic type at Airport facilities operator
Explanation
Correct Answer
B:Create a tested silver transform and quarantine invalid rows before promoting provides a bounded way to introduce the approved solution into maintenance_work_orders_north and detect an early mismatch. The existing signal is profiling identifies a small fraction of missing identifiers plus valid optional fields; a staged change protects the service requirement (silver must reject missing required keys but preserve optional null attributes) and makes rollback practical.
Incorrect Answers
A:Verify no null required IDs remain and legitimate optional-null records survive is a one-time acceptance check. It does not supply the controlled implementation step for maintenance_work_orders_north; a passing test cannot substitute for the specific task now required.
C:Filter or quarantine null required IDs while applying explicit nullable rules to optional columns is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the maintenance_work_orders_north scenario, the observed evidence is that profiling identifies a small fraction of missing identifiers plus valid optional fields; the requirement remains: silver must reject missing required keys but preserve optional null attributes.
D:Blanket replacement changes data meaning and can corrupt types. The proposed action (Call fillna on every field with zero regardless of semantic type at Airport facilities operator) does not meet the requirement that silver must reject missing required keys but preserve optional null attributes, nor does it constitute the controlled implementation step.
Question 600
The claims_payment_events_delta pipeline at Regional insurance broker has this failure: claims_payment_events_delta dashboard repeatedly scans years of silver transactions for daily rollups. The team observed that SQL history shows expensive repeated aggregation on an otherwise stable reporting query. Business requirement: analysts need low-latency aggregates refreshed on a defined cadence. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What should be done next to deploy the approved solution?
- Configure a representative refresh schedule and reconcile the initial build
- Publish a materialized view for precomputed frequently queried aggregates
- Verify refresh applies new silver rows and queries return the expected totals
- Create a regular view and promise it stores its results between queries at Regional insurance broker
Explanation
Correct Answer
A:Configure a representative refresh schedule and reconcile the initial build provides a bounded way to introduce the approved solution into claims_payment_events_delta and detect an early mismatch. The existing signal is SQL history shows expensive repeated aggregation on an otherwise stable reporting query; a staged change protects the service requirement (analysts need low-latency aggregates refreshed on a defined cadence) and makes rollback practical.
Incorrect Answers
B:Publish a materialized view for precomputed frequently queried aggregates is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the claims_payment_events_delta scenario, the observed evidence is that SQL history shows expensive repeated aggregation on an otherwise stable reporting query; the requirement remains: analysts need low-latency aggregates refreshed on a defined cadence.
C:Verify refresh applies new silver rows and queries return the expected totals is a one-time acceptance check. It does not supply the controlled implementation step for claims_payment_events_delta; a passing test cannot substitute for the specific task now required.
D:Ordinary views do not persist precomputed results. The proposed action (Create a regular view and promise it stores its results between queries at Regional insurance broker) does not meet the requirement that analysts need low-latency aggregates refreshed on a defined cadence, nor does it constitute the controlled implementation step.