Databricks Certified Data Engineer Associate Practice Test 98
Question 970
The farm_loan_transaction_history_south pipeline at Agricultural credit union has this failure: farm_loan_transaction_history_south notebook hardcodes a development catalog and writes there during production runs. The team observed that job run settings differ from values embedded in the notebook code. Business requirement: the same notebook must respect runtime environment configuration. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What should the engineer inspect to establish the failure mechanism?
- Remove hardcoded target paths and test parameterized notebook invocation
- Pass task parameters through supported notebook widgets or job parameters
- Edit the source notebook by hand every time the job changes environment at Agricultural credit union
- Identify job parameter keys and where farm_loan_transaction_history_south notebook resolves the catalog
Explanation
Correct Answer
D:Identify job parameter keys and where farm_loan_transaction_history_south notebook resolves the catalog directly investigates the evidence (job run settings differ from values embedded in the notebook code) and distinguishes possible causes of farm_loan_transaction_history_south notebook hardcodes a development catalog and writes there during production runs. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: the same notebook must respect runtime environment configuration.
Incorrect Answers
A:Remove hardcoded target paths and test parameterized notebook invocation may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that job run settings differ from values embedded in the notebook code. This matters here because the farm_loan_transaction_history_south pipeline at Agricultural credit union has this failure:….
B:Pass task parameters through supported notebook widgets or job parameters is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the farm_loan_transaction_history_south scenario, the observed evidence is that job run settings differ from values embedded in the notebook code; the requirement remains: the same notebook must respect runtime environment configuration.
C:Manual code edits are not reproducible parameterization. The proposed action (Edit the source notebook by hand every time the job changes environment at Agricultural credit union) does not meet the requirement that the same notebook must respect runtime environment configuration, nor does it constitute the required pre-change investigation.
Question 971
Consider the farm_loan_transaction_history_east workload at Agricultural credit union: the farm_loan_transaction_history_east landing zone has millions of objects and directory discovery is becoming expensive. Inspection found that ingestion latency rises as source listing grows while only a few files arrive. The governed outcome must ensure that incremental detection must scale without repeatedly listing the whole prefix. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. Which data-engineering approach resolves the documented weakness?
- Recursively list the entire object-store prefix every second at Agricultural credit union
- Select supported managed file events or notification mode when appropriate
- Compare arrival rates, discovery costs and event-notification prerequisites for farm_loan_transaction_history_east
- Monitor notification lag, missed discoveries and fallbacks for farm_loan_transaction_history_east
Explanation
Correct Answer
B:Select supported managed file events or notification mode when appropriate is the intended durable approach for farm_loan_transaction_history_east: it addresses the verified problem (the farm_loan_transaction_history_east landing zone has millions of objects and directory discovery is becoming expensive) while preserving the required behavior (incremental detection must scale without repeatedly listing the whole prefix). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
A:Repeated full listings increase costs and discovery latency. The proposed action (Recursively list the entire object-store prefix every second at Agricultural credit union) does not meet the requirement that incremental detection must scale without repeatedly listing the whole prefix, nor does it constitute the target approach.
C:Compare arrival rates, discovery costs and event-notification prerequisites for farm_loan_transaction_history_east could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The farm_loan_transaction_history_east workflow still requires that incremental detection must scale without repeatedly listing the whole prefix.
D:Monitor notification lag, missed discoveries and fallbacks for farm_loan_transaction_history_east would help detect future recurrence, but metrics alone do not deliver the target approach. The farm_loan_transaction_history_east requirement is that incremental detection must scale without repeatedly listing the whole prefix.
Question 972
The claims_payment_events_central pipeline at Regional insurance broker has this failure: claims_payment_events_central daily DataFrames have the same field names in different column orders. The team observed that union positional matching puts phone numbers into postal-code fields. Business requirement: combined data must align values by field name rather than column position. The first acceptance run passed. The current task is to detect future regressions and changes over time. What should be tracked as an operational early warning?
- Use positional union without checking source column order at Regional insurance broker
- Monitor new and incompatible columns in claims_payment_events_central ingestion
- Use unionByName with controlled missing-column behavior as needed
- Compare samples with reordered columns to verify identity-preserving alignment
Explanation
Correct Answer
B:Monitor new and incompatible columns in claims_payment_events_central ingestion creates recurring visibility into the failure mode (claims_payment_events_central daily DataFrames have the same field names in different column orders). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while combined data must align values by field name rather than column position.
Incorrect Answers
A:Position-based union can silently misalign values. The proposed action (Use positional union without checking source column order at Regional insurance broker) does not meet the requirement that combined data must align values by field name rather than column position, nor does it constitute continuing operational detection.
C:Use unionByName with controlled missing-column behavior as needed is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the claims_payment_events_central scenario, the observed evidence is that union positional matching puts phone numbers into postal-code fields; the requirement remains: combined data must align values by field name rather than column position.
D:Compare samples with reordered columns to verify identity-preserving alignment is a one-time acceptance check. It does not supply continuing operational detection for claims_payment_events_central; a passing test cannot substitute for the specific task now required.
Question 973
At Public weather agency, the station observation files workflow depends on station_observation_files_east. Engineers report that station_observation_files_east Delta queries slow after the filter distribution changes; the evidence is that query profile shows increased file scanning despite healthy worker utilization. They require that performance optimization must adapt to predicate patterns. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which controlled implementation step is appropriate now?
- Add a new physical partition for every possible filter value at Public weather agency
- Compare representative query scan metrics and results before and after
- Recluster a pilot table and measure read/write behavior
- Choose suitable liquid clustering keys and supported optimization maintenance
Explanation
Correct Answer
C:Recluster a pilot table and measure read/write behavior provides a bounded way to introduce the approved solution into station_observation_files_east and detect an early mismatch. The existing signal is query profile shows increased file scanning despite healthy worker utilization; a staged change protects the service requirement (performance optimization must adapt to predicate patterns) and makes rollback practical.
Incorrect Answers
A:Extreme partitioning creates metadata overhead. The proposed action (Add a new physical partition for every possible filter value at Public weather agency) does not meet the requirement that performance optimization must adapt to predicate patterns, nor does it constitute the controlled implementation step.
B:Compare representative query scan metrics and results before and after is a one-time acceptance check. It does not supply the controlled implementation step for station_observation_files_east; a passing test cannot substitute for the specific task now required.
D:Choose suitable liquid clustering keys and supported optimization maintenance is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the station_observation_files_east scenario, the observed evidence is that query profile shows increased file scanning despite healthy worker utilization; the requirement remains: performance optimization must adapt to predicate patterns.
Question 974
The water_quality_readings_coastal pipeline at Aquaculture farming group has this failure: water_quality_readings_coastal failed halfway through a historical backfill, and a rerun doubles completed partitions. The team observed that run log records successfully committed early dates before failure. Business requirement: recovery must replay only incomplete work or be provably idempotent. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What should the engineer inspect to establish the failure mechanism?
- Append the entire historical dataset on every restart at Aquaculture farming group
- Use partition-aware idempotent writes and recoverable backfill parameters
- Resume the backfill on a tested subset without changing good partitions
- List committed partitions, failed tasks and source batch identifiers for water_quality_readings_coastal
Explanation
Correct Answer
D:List committed partitions, failed tasks and source batch identifiers for water_quality_readings_coastal directly investigates the evidence (run log records successfully committed early dates before failure) and distinguishes possible causes of water_quality_readings_coastal failed halfway through a historical backfill, and a rerun doubles completed partitions. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: recovery must replay only incomplete work or be provably idempotent.
Incorrect Answers
A:Unconditional append duplicates earlier successful partitions. The proposed action (Append the entire historical dataset on every restart at Aquaculture farming group) does not meet the requirement that recovery must replay only incomplete work or be provably idempotent, nor does it constitute the required pre-change investigation.
B:Use partition-aware idempotent writes and recoverable backfill parameters is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the water_quality_readings_coastal scenario, the observed evidence is that run log records successfully committed early dates before failure; the requirement remains: recovery must replay only incomplete work or be provably idempotent.
C:Resume the backfill on a tested subset without changing good partitions may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that run log records successfully committed early dates before failure.
Question 975
Consider the clinical_appointment_reports_island workload at Regional hospital network: clinical_appointment_reports_island failed halfway through a historical backfill, and a rerun doubles completed partitions. Inspection found that run log records successfully committed early dates before failure. The governed outcome must ensure that recovery must replay only incomplete work or be provably idempotent. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Resume the backfill on a tested subset without changing good partitions
- Use partition-aware idempotent writes and recoverable backfill parameters
- List committed partitions, failed tasks and source batch identifiers for clinical_appointment_reports_island
- Append the entire historical dataset on every restart at Regional hospital network
Explanation
Correct Answer
C:List committed partitions, failed tasks and source batch identifiers for clinical_appointment_reports_island directly investigates the evidence (run log records successfully committed early dates before failure) and distinguishes possible causes of clinical_appointment_reports_island failed halfway through a historical backfill, and a rerun doubles completed partitions. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: recovery must replay only incomplete work or be provably idempotent.
Incorrect Answers
A:Resume the backfill on a tested subset without changing good partitions may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that run log records successfully committed early dates before failure. This matters here because consider the clinical_appointment_reports_island workload at Regional hospital network:….
B:Use partition-aware idempotent writes and recoverable backfill parameters is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the clinical_appointment_reports_island scenario, the observed evidence is that run log records successfully committed early dates before failure; the requirement remains: recovery must replay only incomplete work or be provably idempotent.
D:Unconditional append duplicates earlier successful partitions. The proposed action (Append the entire historical dataset on every restart at Regional hospital network) does not meet the requirement that recovery must replay only incomplete work or be provably idempotent, nor does it constitute the required pre-change investigation.
Question 976
Consider the trade_settlement_files_north workload at Wholesale produce exchange: trade_settlement_files_north source timestamps arrive as mixed strings and local offsets. Inspection found that a daily fact report groups the same instant into different dates. The governed outcome must ensure that event time must use a documented timezone and typed timestamp convention. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Inspect sample offsets and source timestamp formats for trade_settlement_files_north
- Parse timestamps to a common typed convention before time-window aggregation
- Compare time strings lexicographically and ignore offsets at Wholesale produce exchange
- Run a pilot cast with invalid values quarantined and documented time policy
Explanation
Correct Answer
A:Inspect sample offsets and source timestamp formats for trade_settlement_files_north directly investigates the evidence (a daily fact report groups the same instant into different dates) and distinguishes possible causes of trade_settlement_files_north source timestamps arrive as mixed strings and local offsets. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: event time must use a documented timezone and typed timestamp convention.
Incorrect Answers
B:Parse timestamps to a common typed convention before time-window aggregation is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the trade_settlement_files_north scenario, the observed evidence is that a daily fact report groups the same instant into different dates; the requirement remains: event time must use a documented timezone and typed timestamp convention.
C:String ordering and timezone ambiguity are not robust event-time modeling. The proposed action (Compare time strings lexicographically and ignore offsets at Wholesale produce exchange) does not meet the requirement that event time must use a documented timezone and typed timestamp convention, nor does it constitute the required pre-change investigation.
D:Run a pilot cast with invalid values quarantined and documented time policy may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that a daily fact report groups the same instant into different dates. This matters here because consider the trade_settlement_files_north workload at Wholesale produce exchange: trade_settlement_files_north….
Question 977
A data engineer at Environmental inspection office is responsible for sampling field notes. The managed versus external lifecycle issue is that sampling_field_notes_north reporting team plans to drop a UC-managed table but needs its data files retained. The supporting evidence is that table ownership review shows storage lifecycle controlled by Unity Catalog. The engineer must ensure that a cleanup must not accidentally delete retained source data. The first acceptance run passed. The current task is to detect future regressions and changes over time. What repeatable measure detects this failure after go-live?
- Choose managed or external table lifecycle deliberately for the retention requirement
- Assume managed and external tables always have identical drop semantics at Environmental inspection office
- Drop a disposable test table and verify actual data-lifecycle behavior
- Monitor orphaned external paths and accidental managed-table deletion for sampling_field_notes_north
Explanation
Correct Answer
D:Monitor orphaned external paths and accidental managed-table deletion for sampling_field_notes_north creates recurring visibility into the failure mode (sampling_field_notes_north reporting team plans to drop a UC-managed table but needs its data files retained). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while a cleanup must not accidentally delete retained source data.
Incorrect Answers
A:Choose managed or external table lifecycle deliberately for the retention requirement is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the sampling_field_notes_north scenario, the observed evidence is that table ownership review shows storage lifecycle controlled by Unity Catalog; the requirement remains: a cleanup must not accidentally delete retained source data.
B:Uc table type affects ownership and managed data lifecycle. The proposed action (Assume managed and external tables always have identical drop semantics at Environmental inspection office) does not meet the requirement that a cleanup must not accidentally delete retained source data, nor does it constitute continuing operational detection.
C:Drop a disposable test table and verify actual data-lifecycle behavior is a one-time acceptance check. It does not supply continuing operational detection for sampling_field_notes_north; a passing test cannot substitute for the specific task now required.
Question 978
The admission_scan_events_metro pipeline at Regional sporting venue has this failure: a admission_scan_events_metro enterprise SaaS table requires recurring ingestion but the team maintains a brittle custom API crawler. The team observed that API tokens expire and source field names change without ingestion alerts. Business requirement: the source must arrive in governed Delta tables with manageable credential rotation. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What should be done next to deploy the approved solution?
- Copy API secrets into a widely shared notebook and paste full responses manually at Regional sporting venue
- Update a source row and verify correct incremental destination behavior
- Configure source credentials and destination catalog for a pilot sync
- Prefer a supported Lakeflow Connect managed connector with managed ingestion state
Explanation
Correct Answer
C:Configure source credentials and destination catalog for a pilot sync provides a bounded way to introduce the approved solution into admission_scan_events_metro and detect an early mismatch. The existing signal is API tokens expire and source field names change without ingestion alerts; a staged change protects the service requirement (the source must arrive in governed Delta tables with manageable credential rotation) and makes rollback practical.
Incorrect Answers
A:Shared secrets and manual copies do not provide reliable governed ingestion. The proposed action (Copy API secrets into a widely shared notebook and paste full responses manually at Regional sporting venue) does not meet the requirement that the source must arrive in governed Delta tables with manageable credential rotation, nor does it constitute the controlled implementation step.
B:Update a source row and verify correct incremental destination behavior is a one-time acceptance check. It does not supply the controlled implementation step for admission_scan_events_metro; a passing test cannot substitute for the specific task now required.
D:Prefer a supported Lakeflow Connect managed connector with managed ingestion state is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the admission_scan_events_metro scenario, the observed evidence is that API tokens expire and source field names change without ingestion alerts; the requirement remains: the source must arrive in governed Delta tables with manageable credential rotation.
Question 979
The pollution_measurements_central pipeline at Air quality nonprofit has this failure: pollution_measurements_central SQL filter misses transactions where the nullable flag was never populated. The team observed that the predicate uses flag = NULL and always yields unknown. Business requirement: the analysis must distinguish missing from explicitly false flag values. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What should be done next to deploy the approved solution?
- Use IS NULL for missing flags and IS NOT NULL for populated fields
- Test null, false and true rows against the intended filter
- Use flag = NULL and assume it matches missing values at Air quality nonprofit
- Replace equality-to-null filters with tested null-aware predicates
Explanation
Correct Answer
D:Replace equality-to-null filters with tested null-aware predicates provides a bounded way to introduce the approved solution into pollution_measurements_central and detect an early mismatch. The existing signal is the predicate uses flag = NULL and always yields unknown; a staged change protects the service requirement (the analysis must distinguish missing from explicitly false flag values) and makes rollback practical.
Incorrect Answers
A:Use IS NULL for missing flags and IS NOT NULL for populated fields is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the pollution_measurements_central scenario, the observed evidence is that the predicate uses flag = NULL and always yields unknown; the requirement remains: the analysis must distinguish missing from explicitly false flag values.
B:Test null, false and true rows against the intended filter is a one-time acceptance check. It does not supply the controlled implementation step for pollution_measurements_central; a passing test cannot substitute for the specific task now required.
C:Sql equality against null is not true. The proposed action (Use flag = NULL and assume it matches missing values at Air quality nonprofit) does not meet the requirement that the analysis must distinguish missing from explicitly false flag values, nor does it constitute the controlled implementation step.
Question 980
Consider the monthly_pension_movements_rural workload at Public pension trustee: the monthly_pension_movements_rural CSV feed has unrecognized optional columns and the ingest silently discards them. Inspection found that raw samples show new attributes that do not appear in the bronze output. The governed outcome must ensure that unknown values must remain auditable even before mapping is approved. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Roll out controlled schema rules and record each new rescued field
- Retain unexpected fields in a rescued-data column or quarantined bronze path
- Overwrite source files after dropping all unrecognized fields at Public pension trustee
- Compare incoming headers with inferred schema and rescued fields for monthly_pension_movements_rural
Explanation
Correct Answer
D:Compare incoming headers with inferred schema and rescued fields for monthly_pension_movements_rural directly investigates the evidence (raw samples show new attributes that do not appear in the bronze output) and distinguishes possible causes of the monthly_pension_movements_rural CSV feed has unrecognized optional columns and the ingest silently discards them. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: unknown values must remain auditable even before mapping is approved.
Incorrect Answers
A:Roll out controlled schema rules and record each new rescued field may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that raw samples show new attributes that do not appear in the bronze output. This matters here because consider the monthly_pension_movements_rural workload at Public pension trustee: the monthly_pension_movements_rural….
B:Retain unexpected fields in a rescued-data column or quarantined bronze path is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the monthly_pension_movements_rural scenario, the observed evidence is that raw samples show new attributes that do not appear in the bronze output; the requirement remains: unknown values must remain auditable even before mapping is approved.
C:Discarded raw fields cannot be reconstructed during audit. The proposed action (Overwrite source files after dropping all unrecognized fields at Public pension trustee) does not meet the requirement that unknown values must remain auditable even before mapping is approved, nor does it constitute the required pre-change investigation.