Databricks Certified Data Engineer Associate Practice Test 59
Question 580
At Regional sporting venue, the Lakeflow Connect managed connector workflow handles admission scan events. The problem is that a admission_scan_events_metro enterprise SaaS table requires recurring ingestion but the team maintains a brittle custom API crawler. Evidence: API tokens expire and source field names change without ingestion alerts. Constraint: the source must arrive in governed Delta tables with manageable credential rotation. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. What governed target implementation should the engineer design?
- Monitor connector lag, source authentication errors and schema drift for admission_scan_events_metro
- Prefer a supported Lakeflow Connect managed connector with managed ingestion state
- Copy API secrets into a widely shared notebook and paste full responses manually at Regional sporting venue
- Assess managed connector source support, permissions and refresh semantics for admission_scan_events_metro
Explanation
Correct Answer
B:Prefer a supported Lakeflow Connect managed connector with managed ingestion state is the intended durable approach for admission_scan_events_metro: it addresses the verified problem (a admission_scan_events_metro enterprise SaaS table requires recurring ingestion but the team maintains a brittle custom API crawler) while preserving the required behavior (the source must arrive in governed Delta tables with manageable credential rotation). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
A:Monitor connector lag, source authentication errors and schema drift for admission_scan_events_metro would help detect future recurrence, but metrics alone do not deliver the target approach. The admission_scan_events_metro requirement is that the source must arrive in governed Delta tables with manageable credential rotation.
C:Shared secrets and manual copies do not provide reliable governed ingestion. The proposed action (Copy API secrets into a widely shared notebook and paste full responses manually at Regional sporting venue) does not meet the requirement that the source must arrive in governed Delta tables with manageable credential rotation, nor does it constitute the target approach.
D:Assess managed connector source support, permissions and refresh semantics for admission_scan_events_metro could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The admission_scan_events_metro workflow still requires that the source must arrive in governed Delta tables with manageable credential rotation.
Question 581
At National farm supplier, the harvest purchase orders workflow depends on harvest_purchase_orders_west. Engineers report that harvest_purchase_orders_west silver records pass ingestion despite negative billed quantities; the evidence is that validation sample identifies impossible values in otherwise well-formed JSON. They require that bad records need a defined reject/quarantine policy without hiding valid data. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which controlled implementation step is appropriate now?
- Delete invalid rows silently and report the entire batch as error-free at National farm supplier
- Define explicit data-quality expectations with documented action for invalid records
- Roll out expectation checks and a quarantine audit before enforcing production policy
- Inject invalid rows and verify correct drop, fail or quarantine behavior
Explanation
Correct Answer
C:Roll out expectation checks and a quarantine audit before enforcing production policy provides a bounded way to introduce the approved solution into harvest_purchase_orders_west and detect an early mismatch. The existing signal is validation sample identifies impossible values in otherwise well-formed JSON; a staged change protects the service requirement (bad records need a defined reject/quarantine policy without hiding valid data) and makes rollback practical.
Incorrect Answers
A:Silent loss hides quality issues and masks upstream defects. The proposed action (Delete invalid rows silently and report the entire batch as error-free at National farm supplier) does not meet the requirement that bad records need a defined reject/quarantine policy without hiding valid data, nor does it constitute the controlled implementation step.
B:Define explicit data-quality expectations with documented action for invalid records is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the harvest_purchase_orders_west scenario, the observed evidence is that validation sample identifies impossible values in otherwise well-formed JSON; the requirement remains: bad records need a defined reject/quarantine policy without hiding valid data.
D:Inject invalid rows and verify correct drop, fail or quarantine behavior is a one-time acceptance check. It does not supply the controlled implementation step for harvest_purchase_orders_west; a passing test cannot substitute for the specific task now required.
Question 582
A data engineer at Air quality nonprofit is responsible for pollution measurements. The Delta transaction isolation issue is that two concurrent writers update pollution_measurements_west and a reader sometimes sees inconsistent exported copies. The supporting evidence is that the write log shows independent files were replaced while an ETL job was reading them. The engineer must ensure that readers require consistent committed versions without pausing ingest. The first acceptance run passed. The current task is to detect future regressions and changes over time. What repeatable measure detects this failure after go-live?
- Allow direct writers to replace arbitrary Parquet files while readers scan them at Air quality nonprofit
- Track failed commits, conflict retries and table-history anomalies for pollution_measurements_west
- Read pollution_measurements_west during overlapping test writes and verify a consistent committed snapshot
- Store pollution_measurements_west as a Delta table and rely on atomic transaction-log commits
Explanation
Correct Answer
B:Track failed commits, conflict retries and table-history anomalies for pollution_measurements_west creates recurring visibility into the failure mode (two concurrent writers update pollution_measurements_west and a reader sometimes sees inconsistent exported copies). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while readers require consistent committed versions without pausing ingest.
Incorrect Answers
A:Uncoordinated file replacement cannot provide delta transaction guarantees. The proposed action (Allow direct writers to replace arbitrary Parquet files while readers scan them at Air quality nonprofit) does not meet the requirement that readers require consistent committed versions without pausing ingest, nor does it constitute continuing operational detection.
C:Read pollution_measurements_west during overlapping test writes and verify a consistent committed snapshot is a one-time acceptance check. It does not supply continuing operational detection for pollution_measurements_west; a passing test cannot substitute for the specific task now required.
D:Store pollution_measurements_west as a Delta table and rely on atomic transaction-log commits is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the pollution_measurements_west scenario, the observed evidence is that the write log shows independent files were replaced while an ETL job was reading them; the requirement remains: readers require consistent committed versions without pausing ingest.
Question 583
Consider the product_sales_transactions_east workload at Online bicycle retailer: product_sales_transactions_east job duration doubles after a code release but task status is green. Inspection found that run history shows a persistent trend from 12 to 30 minutes. The governed outcome must ensure that operations must distinguish performance regressions from random short spikes. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Add duration alerts and annotate the release time on job history
- Establish baseline runtime thresholds and stage-level investigation triggers
- Rely on green status alone and ignore duration changes at Online bicycle retailer
- Compare run-history duration across identical product_sales_transactions_east input sizes and revisions
Explanation
Correct Answer
D:Compare run-history duration across identical product_sales_transactions_east input sizes and revisions directly investigates the evidence (run history shows a persistent trend from 12 to 30 minutes) and distinguishes possible causes of product_sales_transactions_east job duration doubles after a code release but task status is green. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: operations must distinguish performance regressions from random short spikes.
Incorrect Answers
A:Add duration alerts and annotate the release time on job history may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that run history shows a persistent trend from 12 to 30 minutes. This matters here because consider the product_sales_transactions_east workload at Online bicycle retailer: product_sales_transactions_east….
B:Establish baseline runtime thresholds and stage-level investigation triggers is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the product_sales_transactions_east scenario, the observed evidence is that run history shows a persistent trend from 12 to 30 minutes; the requirement remains: operations must distinguish performance regressions from random short spikes.
C:Successful completion does not imply healthy latency. The proposed action (Rely on green status alone and ignore duration changes at Online bicycle retailer) does not meet the requirement that operations must distinguish performance regressions from random short spikes, nor does it constitute the required pre-change investigation.
Question 584
The vessel_position_events_central pipeline at Maritime safety agency has this failure: vessel_position_events_central gold consumers need fresh incremental data from an append-only event feed. The team observed that the source produces new events continuously while downstream reports lag by hours. Business requirement: materialized output must incorporate new events with recoverable processing state. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. What test most directly proves the requirement is satisfied?
- Append new source events and verify exactly the required incremental result
- Truncate and fully rebuild the target after every incoming event at Maritime safety agency
- Deploy a streaming table with tested recovery checkpoints and refresh configuration
- Measure source update shape, processing latency and offset semantics for vessel_position_events_central
Explanation
Correct Answer
A:Append new source events and verify exactly the required incremental result directly tests the behavior that should change after the correction. With the observed evidence (the source produces new events continuously while downstream reports lag by hours), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that materialized output must incorporate new events with recoverable processing state.
Incorrect Answers
B:Full rebuilds are expensive and defeat incremental freshness goals. The proposed action (Truncate and fully rebuild the target after every incoming event at Maritime safety agency) does not meet the requirement that materialized output must incorporate new events with recoverable processing state, nor does it constitute the post-change acceptance test.
C:Deploy a streaming table with tested recovery checkpoints and refresh configuration may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that the source produces new events continuously while downstream reports lag by hours.
D:Measure source update shape, processing latency and offset semantics for vessel_position_events_central could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The vessel_position_events_central workflow still requires that materialized output must incorporate new events with recoverable processing state.
Question 585
Consider the customer_order_changes_west workload at Online apparel cooperative: customer_order_changes_west test run overwrites production tables because both deployment targets share one catalog. Inspection found that resolved bundle targets point to the same UC prefix. The governed outcome must ensure that development tests must not mutate production data. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which staged rollout best preserves the existing workload?
- Deploy a nonproduction copy with unique resource namespace and service identity
- Run destructive test inputs and verify production table remains unchanged
- Use identical production destinations and hope developers avoid dangerous input at Online apparel cooperative
- Segregate dev/test/prod catalog settings using bundle target overrides
Explanation
Correct Answer
A:Deploy a nonproduction copy with unique resource namespace and service identity provides a bounded way to introduce the approved solution into customer_order_changes_west and detect an early mismatch. The existing signal is resolved bundle targets point to the same UC prefix; a staged change protects the service requirement (development tests must not mutate production data) and makes rollback practical.
Incorrect Answers
B:Run destructive test inputs and verify production table remains unchanged is a one-time acceptance check. It does not supply the controlled implementation step for customer_order_changes_west; a passing test cannot substitute for the specific task now required.
C:Shared output targets defeat environment isolation. The proposed action (Use identical production destinations and hope developers avoid dangerous input at Online apparel cooperative) does not meet the requirement that development tests must not mutate production data, nor does it constitute the controlled implementation step.
D:Segregate dev/test/prod catalog settings using bundle target overrides is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the customer_order_changes_west scenario, the observed evidence is that resolved bundle targets point to the same UC prefix; the requirement remains: development tests must not mutate production data.
Question 586
A data engineer at Regional hospital network is responsible for clinical appointment reports. The shuffle partition oversplitting issue is that clinical_appointment_reports_rural job has hundreds of thousands of very small tasks on a modest daily dataset. The supporting evidence is that Spark UI shows scheduling overhead dominates work time. The engineer must ensure that the adjustment must keep exact output and avoid poor parallelism. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. What evidence establishes that correct output is maintained?
- Set shuffle partitions to an arbitrary enormous value for all workloads at Regional hospital network
- Test bounded shuffle partition adjustment in staging and capture the plan
- Measure input sizes, task overhead and shuffle partition count for clinical_appointment_reports_rural
- Compare results and runtime at multiple partition settings
Explanation
Correct Answer
D:Compare results and runtime at multiple partition settings directly tests the behavior that should change after the correction. With the observed evidence (Spark UI shows scheduling overhead dominates work time), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that the adjustment must keep exact output and avoid poor parallelism.
Incorrect Answers
A:More tasks can increase scheduling overhead rather than speed work. The proposed action (Set shuffle partitions to an arbitrary enormous value for all workloads at Regional hospital network) does not meet the requirement that the adjustment must keep exact output and avoid poor parallelism, nor does it constitute the post-change acceptance test.
B:Test bounded shuffle partition adjustment in staging and capture the plan may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that Spark UI shows scheduling overhead dominates work time.
C:Measure input sizes, task overhead and shuffle partition count for clinical_appointment_reports_rural could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The clinical_appointment_reports_rural workflow still requires that the adjustment must keep exact output and avoid poor parallelism.
Question 587
The shipment_manifests_metro pipeline at Coastal freight alliance has this failure: shipment_manifests_metro CI reports successful deployment even though its Lakeflow job has never executed. The team observed that the build log only shows resource creation, no job-run evidence. Business requirement: release must demonstrate a working data pipeline, not just valid YAML. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. What lasting solution should be selected for the verified conditions?
- Monitor smoke-test failures and drift between tested and deployed revisions
- Accept a successful YAML validation as proof the ETL output is correct at Coastal freight alliance
- Add a post-deployment smoke test and acceptance assertions to the CI pipeline
- Check whether bundle validation, deployment and execution steps are separated for shipment_manifests_metro
Explanation
Correct Answer
C:Add a post-deployment smoke test and acceptance assertions to the CI pipeline is the intended durable approach for shipment_manifests_metro: it addresses the verified problem (shipment_manifests_metro CI reports successful deployment even though its Lakeflow job has never executed) while preserving the required behavior (release must demonstrate a working data pipeline, not just valid YAML). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
A:Monitor smoke-test failures and drift between tested and deployed revisions would help detect future recurrence, but metrics alone do not deliver the target approach. The shipment_manifests_metro requirement is that release must demonstrate a working data pipeline, not just valid YAML.
B:Syntax validation cannot establish runtime data correctness. The proposed action (Accept a successful YAML validation as proof the ETL output is correct at Coastal freight alliance) does not meet the requirement that release must demonstrate a working data pipeline, not just valid YAML, nor does it constitute the target approach.
D:Check whether bundle validation, deployment and execution steps are separated for shipment_manifests_metro could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The shipment_manifests_metro workflow still requires that release must demonstrate a working data pipeline, not just valid YAML.
Question 588
The shipment_manifests_urban pipeline at Coastal freight alliance has this failure: shipment_manifests_urban orders disappear from silver after a dimension join. The team observed that audit shows some orders have a missing matching customer row. Business requirement: all fact orders must remain visible even if a dimension lookup fails. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What should the engineer inspect to establish the failure mechanism?
- Replace the fact join with inner join and discard nonmatches at Coastal freight alliance
- Use a left join from facts to customer dimension and flag unmatched rows
- Count unmatched keys and examine join predicates for shipment_manifests_urban
- Introduce the corrected join with data-quality quarantine for missing attributes
Explanation
Correct Answer
C:Count unmatched keys and examine join predicates for shipment_manifests_urban directly investigates the evidence (audit shows some orders have a missing matching customer row) and distinguishes possible causes of shipment_manifests_urban orders disappear from silver after a dimension join. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: all fact orders must remain visible even if a dimension lookup fails.
Incorrect Answers
A:An inner join suppresses facts when the right-side lookup is absent. The proposed action (Replace the fact join with inner join and discard nonmatches at Coastal freight alliance) does not meet the requirement that all fact orders must remain visible even if a dimension lookup fails, nor does it constitute the required pre-change investigation.
B:Use a left join from facts to customer dimension and flag unmatched rows is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the shipment_manifests_urban scenario, the observed evidence is that audit shows some orders have a missing matching customer row; the requirement remains: all fact orders must remain visible even if a dimension lookup fails.
D:Introduce the corrected join with data-quality quarantine for missing attributes may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that audit shows some orders have a missing matching customer row. This matters here because the shipment_manifests_urban pipeline at Coastal freight alliance has this failure: shipment_manifests_urban orders….
Question 589
Consider the ticket_transactions_coastal workload at Independent cinema group: ticket_transactions_coastal quality validation flags an empty input but the pipeline always publishes gold. Inspection found that the job DAG lacks an if/else decision based on validation outcome. The governed outcome must ensure that empty invalid batches must skip publication while still reporting status. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which continuing signal best reveals recurrence or drift?
- Use a Lakeflow condition task to route pass and fail paths explicitly
- Always run publish regardless of validator outcome at Independent cinema group
- Test valid and invalid input and verify only the intended branch executes
- Monitor branch outcomes and unexpected publish-after-failure events for ticket_transactions_coastal
Explanation
Correct Answer
D:Monitor branch outcomes and unexpected publish-after-failure events for ticket_transactions_coastal creates recurring visibility into the failure mode (ticket_transactions_coastal quality validation flags an empty input but the pipeline always publishes gold). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while empty invalid batches must skip publication while still reporting status.
Incorrect Answers
A:Use a Lakeflow condition task to route pass and fail paths explicitly is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the ticket_transactions_coastal scenario, the observed evidence is that the job DAG lacks an if/else decision based on validation outcome; the requirement remains: empty invalid batches must skip publication while still reporting status.
B:Unconditional publication violates the quality gate. The proposed action (Always run publish regardless of validator outcome at Independent cinema group) does not meet the requirement that empty invalid batches must skip publication while still reporting status, nor does it constitute continuing operational detection.
C:Test valid and invalid input and verify only the intended branch executes is a one-time acceptance check. It does not supply continuing operational detection for ticket_transactions_coastal; a passing test cannot substitute for the specific task now required.
Question 590
Consider the artifact_metadata_updates_metro workload at Regional museum alliance: artifact_metadata_updates_metro delta target must apply corrected invoice amounts from an incremental feed. Inspection found that the source has two updates for the same invoice_id in one batch. The governed outcome must ensure that target must have one latest value per invoice after applying ordered changes. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Identify stable merge keys and deduplicate multiple updates in the batch for artifact_metadata_updates_metro
- Use Delta MERGE with a deduplicated source on invoice_id and matched update logic
- MERGE with multiple unfiltered source matches per target key at Regional museum alliance
- Test MERGE on a small target and rerun an identical source batch
Explanation
Correct Answer
A:Identify stable merge keys and deduplicate multiple updates in the batch for artifact_metadata_updates_metro directly investigates the evidence (the source has two updates for the same invoice_id in one batch) and distinguishes possible causes of artifact_metadata_updates_metro delta target must apply corrected invoice amounts from an incremental feed. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: target must have one latest value per invoice after applying ordered changes.
Incorrect Answers
B:Use Delta MERGE with a deduplicated source on invoice_id and matched update logic is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the artifact_metadata_updates_metro scenario, the observed evidence is that the source has two updates for the same invoice_id in one batch; the requirement remains: target must have one latest value per invoice after applying ordered changes.
C:Ambiguous multiple source matches can make updates nondeterministic or fail. The proposed action (MERGE with multiple unfiltered source matches per target key at Regional museum alliance) does not meet the requirement that target must have one latest value per invoice after applying ordered changes, nor does it constitute the required pre-change investigation.
D:Test MERGE on a small target and rerun an identical source batch may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that the source has two updates for the same invoice_id in one batch. This matters here because consider the artifact_metadata_updates_metro workload at Regional museum alliance: artifact_metadata_updates_metro….