Databricks Certified Data Engineer Associate Practice Test 43
Question 420
At Regional hospital network, the shuffle partition oversplitting workflow handles clinical appointment reports. The problem is that clinical_appointment_reports_rural job has hundreds of thousands of very small tasks on a modest daily dataset. Evidence: Spark UI shows scheduling overhead dominates work time. Constraint: the adjustment must keep exact output and avoid poor parallelism. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. What governed target implementation should the engineer design?
- Measure input sizes, task overhead and shuffle partition count for clinical_appointment_reports_rural
- Reduce excessive partitions or enable adaptive coalescing as appropriate
- Monitor tiny-task fraction, scheduling latency and executor utilization for clinical_appointment_reports_rural
- Set shuffle partitions to an arbitrary enormous value for all workloads at Regional hospital network
Explanation
Correct Answer
B:Reduce excessive partitions or enable adaptive coalescing as appropriate is the intended durable approach for clinical_appointment_reports_rural: it addresses the verified problem (clinical_appointment_reports_rural job has hundreds of thousands of very small tasks on a modest daily dataset) while preserving the required behavior (the adjustment must keep exact output and avoid poor parallelism). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
A:Measure input sizes, task overhead and shuffle partition count for clinical_appointment_reports_rural could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The clinical_appointment_reports_rural workflow still requires that the adjustment must keep exact output and avoid poor parallelism.
C:Monitor tiny-task fraction, scheduling latency and executor utilization for clinical_appointment_reports_rural would help detect future recurrence, but metrics alone do not deliver the target approach. The clinical_appointment_reports_rural requirement is that the adjustment must keep exact output and avoid poor parallelism.
D:More tasks can increase scheduling overhead rather than speed work. The proposed action (Set shuffle partitions to an arbitrary enormous value for all workloads at Regional hospital network) does not meet the requirement that the adjustment must keep exact output and avoid poor parallelism, nor does it constitute the target approach.
Question 421
A data engineer at Public weather agency is responsible for station observation files. The Lakeflow Connect standard connector issue is that a station_observation_files_highland source has a compatible connector but existing custom scripts have divergent retries. The supporting evidence is that logs show different authentication and pagination paths across team-owned scripts. The engineer must ensure that ingestion must be standardized under governed destinations. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What is the safest migration or deployment sequence?
- Configure the connector and validate target schema and schedule in staging
- Let every notebook invent a new unscheduled direct source sync at Public weather agency
- Use the supported Lakeflow Connect standard connector with scoped source access
- Compare loaded rows and source bookmarks before switching production
Explanation
Correct Answer
A:Configure the connector and validate target schema and schedule in staging provides a bounded way to introduce the approved solution into station_observation_files_highland and detect an early mismatch. The existing signal is logs show different authentication and pagination paths across team-owned scripts; a staged change protects the service requirement (ingestion must be standardized under governed destinations) and makes rollback practical.
Incorrect Answers
B:Uncoordinated custom pulls are less governed and harder to recover. The proposed action (Let every notebook invent a new unscheduled direct source sync at Public weather agency) does not meet the requirement that ingestion must be standardized under governed destinations, nor does it constitute the controlled implementation step.
C:Use the supported Lakeflow Connect standard connector with scoped source access is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the station_observation_files_highland scenario, the observed evidence is that logs show different authentication and pagination paths across team-owned scripts; the requirement remains: ingestion must be standardized under governed destinations.
D:Compare loaded rows and source bookmarks before switching production is a one-time acceptance check. It does not supply the controlled implementation step for station_observation_files_highland; a passing test cannot substitute for the specific task now required.
Question 422
The maintenance_work_orders_central pipeline at Airport facilities operator has this failure: the maintenance_work_orders_central relational source emits inserts and updates but the lakehouse target retains multiple current versions. The team observed that change records carry primary keys and update sequence numbers. Business requirement: the destination must represent the latest source state without losing required history. The first acceptance run passed. The current task is to detect future regressions and changes over time. What should be tracked as an operational early warning?
- Apply multiple out-of-order changes and verify deterministic final state
- Use supported CDC apply logic with keys, ordering and defined delete handling
- Monitor CDC lag, duplicate keys and discarded out-of-order updates for maintenance_work_orders_central
- Append every change as a new current row without keys at Airport facilities operator
Explanation
Correct Answer
C:Monitor CDC lag, duplicate keys and discarded out-of-order updates for maintenance_work_orders_central creates recurring visibility into the failure mode (the maintenance_work_orders_central relational source emits inserts and updates but the lakehouse target retains multiple current versions). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while the destination must represent the latest source state without losing required history.
Incorrect Answers
A:Apply multiple out-of-order changes and verify deterministic final state is a one-time acceptance check. It does not supply continuing operational detection for maintenance_work_orders_central; a passing test cannot substitute for the specific task now required.
B:Use supported CDC apply logic with keys, ordering and defined delete handling is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the maintenance_work_orders_central scenario, the observed evidence is that change records carry primary keys and update sequence numbers; the requirement remains: the destination must represent the latest source state without losing required history.
D:An append-only current-state table does not resolve updates or deletes. The proposed action (Append every change as a new current row without keys at Airport facilities operator) does not meet the requirement that the destination must represent the latest source state without losing required history, nor does it constitute continuing operational detection.
Question 423
A data engineer at E-commerce payments provider is responsible for payment authorization records. The cluster startup failure issue is that payment_authorization_records_east job fails before any Spark task begins. The supporting evidence is that cluster event log shows an unavailable instance type in the selected region. The engineer must ensure that the diagnostic must distinguish compute provisioning failure from ETL code defects. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. What evidence establishes that correct output is maintained?
- Rewrite successful SQL transformations without investigating provisioning at E-commerce payments provider
- Test alternate eligible compute provisioning and controlled job retry
- Start a fresh cluster then execute one representative task successfully
- Inspect cluster event log, quotas, network and cloud capacity for payment_authorization_records_east
Explanation
Correct Answer
C:Start a fresh cluster then execute one representative task successfully directly tests the behavior that should change after the correction. With the observed evidence (cluster event log shows an unavailable instance type in the selected region), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that the diagnostic must distinguish compute provisioning failure from ETL code defects.
Incorrect Answers
A:Query changes cannot fix unavailable compute capacity. The proposed action (Rewrite successful SQL transformations without investigating provisioning at E-commerce payments provider) does not meet the requirement that the diagnostic must distinguish compute provisioning failure from ETL code defects, nor does it constitute the post-change acceptance test.
B:Test alternate eligible compute provisioning and controlled job retry may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that cluster event log shows an unavailable instance type in the selected region.
D:Inspect cluster event log, quotas, network and cloud capacity for payment_authorization_records_east could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The payment_authorization_records_east workflow still requires that the diagnostic must distinguish compute provisioning failure from ETL code defects.
Question 424
At Airport facilities operator, the Silver null cleaning workflow handles maintenance work orders. The problem is that maintenance_work_orders_north bronze records include null business IDs that break downstream joins. Evidence: profiling identifies a small fraction of missing identifiers plus valid optional fields. Constraint: silver must reject missing required keys but preserve optional null attributes. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which investigation is the strongest first step?
- Create a tested silver transform and quarantine invalid rows before promoting
- Filter or quarantine null required IDs while applying explicit nullable rules to optional columns
- Call fillna on every field with zero regardless of semantic type at Airport facilities operator
- Profile required versus nullable fields and sample failed maintenance_work_orders_north records
Explanation
Correct Answer
D:Profile required versus nullable fields and sample failed maintenance_work_orders_north records directly investigates the evidence (profiling identifies a small fraction of missing identifiers plus valid optional fields) and distinguishes possible causes of maintenance_work_orders_north bronze records include null business IDs that break downstream joins. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: silver must reject missing required keys but preserve optional null attributes.
Incorrect Answers
A:Create a tested silver transform and quarantine invalid rows before promoting may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that profiling identifies a small fraction of missing identifiers plus valid optional fields. Here, the relevant situation is: the Silver null cleaning workflow handles maintenance work orders. The problem is that maintenance_work_orders_north….
B:Filter or quarantine null required IDs while applying explicit nullable rules to optional columns is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the maintenance_work_orders_north scenario, the observed evidence is that profiling identifies a small fraction of missing identifiers plus valid optional fields; the requirement remains: silver must reject missing required keys but preserve optional null attributes.
C:Blanket replacement changes data meaning and can corrupt types. The proposed action (Call fillna on every field with zero regardless of semantic type at Airport facilities operator) does not meet the requirement that silver must reject missing required keys but preserve optional null attributes, nor does it constitute the required pre-change investigation.
Question 425
Consider the water_quality_readings_rural workload at Aquaculture farming group: the water_quality_readings_rural landing zone has millions of objects and directory discovery is becoming expensive. Inspection found that ingestion latency rises as source listing grows while only a few files arrive. The governed outcome must ensure that incremental detection must scale without repeatedly listing the whole prefix. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Recursively list the entire object-store prefix every second at Aquaculture farming group
- Enable managed file-event discovery on a pilot source and retain checkpoint state
- Compare arrival rates, discovery costs and event-notification prerequisites for water_quality_readings_rural
- Select supported managed file events or notification mode when appropriate
Explanation
Correct Answer
C:Compare arrival rates, discovery costs and event-notification prerequisites for water_quality_readings_rural directly investigates the evidence (ingestion latency rises as source listing grows while only a few files arrive) and distinguishes possible causes of the water_quality_readings_rural landing zone has millions of objects and directory discovery is becoming expensive. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: incremental detection must scale without repeatedly listing the whole prefix.
Incorrect Answers
A:Repeated full listings increase costs and discovery latency. The proposed action (Recursively list the entire object-store prefix every second at Aquaculture farming group) does not meet the requirement that incremental detection must scale without repeatedly listing the whole prefix, nor does it constitute the required pre-change investigation.
B:Enable managed file-event discovery on a pilot source and retain checkpoint state may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that ingestion latency rises as source listing grows while only a few files arrive.
D:Select supported managed file events or notification mode when appropriate is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the water_quality_readings_rural scenario, the observed evidence is that ingestion latency rises as source listing grows while only a few files arrive; the requirement remains: incremental detection must scale without repeatedly listing the whole prefix.
Question 426
Consider the dispatch_and_delivery_status_south workload at Food delivery franchise: dispatch_and_delivery_status_south managed table has manual OPTIMIZE jobs colliding with platform-managed maintenance. Inspection found that table maintenance history shows overlapping compaction attempts. The governed outcome must ensure that the maintenance policy must avoid redundant cost and scheduling conflict. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which evidence-gathering action should precede an architectural change?
- Use predictive optimization where supported and retire redundant manual jobs
- Disable the overlapping manual maintenance after verifying managed policy
- Inspect table eligibility, predictive optimization configuration and maintenance history for dispatch_and_delivery_status_south
- Manually VACUUM all data files before retention safeguards are checked at Food delivery franchise
Explanation
Correct Answer
C:Inspect table eligibility, predictive optimization configuration and maintenance history for dispatch_and_delivery_status_south directly investigates the evidence (table maintenance history shows overlapping compaction attempts) and distinguishes possible causes of dispatch_and_delivery_status_south managed table has manual OPTIMIZE jobs colliding with platform-managed maintenance. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: the maintenance policy must avoid redundant cost and scheduling conflict.
Incorrect Answers
A:Use predictive optimization where supported and retire redundant manual jobs is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the dispatch_and_delivery_status_south scenario, the observed evidence is that table maintenance history shows overlapping compaction attempts; the requirement remains: the maintenance policy must avoid redundant cost and scheduling conflict.
B:Disable the overlapping manual maintenance after verifying managed policy may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that table maintenance history shows overlapping compaction attempts. This matters here because consider the dispatch_and_delivery_status_south workload at Food delivery franchise:….
D:Unguarded vacuum can remove files needed for recovery. The proposed action (Manually VACUUM all data files before retention safeguards are checked at Food delivery franchise) does not meet the requirement that the maintenance policy must avoid redundant cost and scheduling conflict, nor does it constitute the required pre-change investigation.
Question 427
At Manufacturing quality network, the factory defect records workflow depends on factory_defect_records_urban. Engineers report that factory_defect_records_urban scheduled ingestion succeeds under one employee but fails under automation identity; the evidence is that UC audit shows only the employee has the required table rights. They require that the job must operate under a managed nonhuman least-privilege principal. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which pre-change diagnostic will resolve the uncertainty?
- Grant the required scoped privileges to an approved service principal
- Switch the job identity and verify its input and output access
- Compare job run-as identity and catalog, schema and table grants for factory_defect_records_urban
- Share a workspace administrator's personal access token with all job owners at Manufacturing quality network
Explanation
Correct Answer
C:Compare job run-as identity and catalog, schema and table grants for factory_defect_records_urban directly investigates the evidence (UC audit shows only the employee has the required table rights) and distinguishes possible causes of factory_defect_records_urban scheduled ingestion succeeds under one employee but fails under automation identity. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: the job must operate under a managed nonhuman least-privilege principal.
Incorrect Answers
A:Grant the required scoped privileges to an approved service principal is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the factory_defect_records_urban scenario, the observed evidence is that UC audit shows only the employee has the required table rights; the requirement remains: the job must operate under a managed nonhuman least-privilege principal.
B:Switch the job identity and verify its input and output access may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that UC audit shows only the employee has the required table rights. Here, the relevant situation is: the factory defect records workflow depends on factory_defect_records_urban. Engineers report that….
D:Shared personal tokens are fragile and overprivileged. The proposed action (Share a workspace administrator's personal access token with all job owners at Manufacturing quality network) does not meet the requirement that the job must operate under a managed nonhuman least-privilege principal, nor does it constitute the required pre-change investigation.
Question 428
The viewer_activity_data_coastal pipeline at Subscription video network has this failure: a viewer_activity_data_coastal source has a compatible connector but existing custom scripts have divergent retries. The team observed that logs show different authentication and pagination paths across team-owned scripts. Business requirement: ingestion must be standardized under governed destinations. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What should the engineer inspect to establish the failure mechanism?
- Identify connector coverage, source permissions and supported incremental behavior for viewer_activity_data_coastal
- Use the supported Lakeflow Connect standard connector with scoped source access
- Configure the connector and validate target schema and schedule in staging
- Let every notebook invent a new unscheduled direct source sync at Subscription video network
Explanation
Correct Answer
A:Identify connector coverage, source permissions and supported incremental behavior for viewer_activity_data_coastal directly investigates the evidence (logs show different authentication and pagination paths across team-owned scripts) and distinguishes possible causes of a viewer_activity_data_coastal source has a compatible connector but existing custom scripts have divergent retries. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: ingestion must be standardized under governed destinations.
Incorrect Answers
B:Use the supported Lakeflow Connect standard connector with scoped source access is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the viewer_activity_data_coastal scenario, the observed evidence is that logs show different authentication and pagination paths across team-owned scripts; the requirement remains: ingestion must be standardized under governed destinations.
C:Configure the connector and validate target schema and schedule in staging may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that logs show different authentication and pagination paths across team-owned scripts.
D:Uncoordinated custom pulls are less governed and harder to recover. The proposed action (Let every notebook invent a new unscheduled direct source sync at Subscription video network) does not meet the requirement that ingestion must be standardized under governed destinations, nor does it constitute the required pre-change investigation.
Question 429
At Consumer finance platform, the loan repayment events workflow depends on loan_repayment_events_island. Engineers report that loan_repayment_events_island overnight export fires one hour early after daylight-saving changes; the evidence is that run history shows schedule configured in a different timezone than business cutoff. They require that the export must complete relative to the local reporting calendar. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which controlled implementation step is appropriate now?
- Change the job schedule in a test environment and review next fire times
- Assume all local timezones have the same UTC offset year-round at Consumer finance platform
- Verify triggers across DST transition dates against business calendar
- Configure Lakeflow scheduled trigger using an explicit timezone and cutoff
Explanation
Correct Answer
A:Change the job schedule in a test environment and review next fire times provides a bounded way to introduce the approved solution into loan_repayment_events_island and detect an early mismatch. The existing signal is run history shows schedule configured in a different timezone than business cutoff; a staged change protects the service requirement (the export must complete relative to the local reporting calendar) and makes rollback practical.
Incorrect Answers
B:Timezones with dst can shift relative to utc. The proposed action (Assume all local timezones have the same UTC offset year-round at Consumer finance platform) does not meet the requirement that the export must complete relative to the local reporting calendar, nor does it constitute the controlled implementation step.
C:Verify triggers across DST transition dates against business calendar is a one-time acceptance check. It does not supply the controlled implementation step for loan_repayment_events_island; a passing test cannot substitute for the specific task now required.
D:Configure Lakeflow scheduled trigger using an explicit timezone and cutoff is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the loan_repayment_events_island scenario, the observed evidence is that run history shows schedule configured in a different timezone than business cutoff; the requirement remains: the export must complete relative to the local reporting calendar.
Question 430
A data engineer at Regional telecom operator is responsible for subscriber provisioning logs. The drift versus bundle source issue is that subscriber_provisioning_logs_north production job UI was manually edited after bundle deployment. The supporting evidence is that resolved job settings differ from the versioned bundle manifest. The engineer must ensure that releases must remain traceable to approved Git changes. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What analysis is necessary before deciding on remediation?
- Treat the versioned bundle as the source of truth and prohibit unreviewed manual drift
- Compare actual resource definitions with deployed bundle target and source commit for subscriber_provisioning_logs_north
- Update the source by copying random current production settings without review at Regional telecom operator
- Reapply the approved bundle after controlled review of production-only changes
Explanation
Correct Answer
B:Compare actual resource definitions with deployed bundle target and source commit for subscriber_provisioning_logs_north directly investigates the evidence (resolved job settings differ from the versioned bundle manifest) and distinguishes possible causes of subscriber_provisioning_logs_north production job UI was manually edited after bundle deployment. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: releases must remain traceable to approved Git changes.
Incorrect Answers
A:Treat the versioned bundle as the source of truth and prohibit unreviewed manual drift is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the subscriber_provisioning_logs_north scenario, the observed evidence is that resolved job settings differ from the versioned bundle manifest; the requirement remains: releases must remain traceable to approved Git changes.
C:Unreviewed drift can be promoted into every environment. The proposed action (Update the source by copying random current production settings without review at Regional telecom operator) does not meet the requirement that releases must remain traceable to approved Git changes, nor does it constitute the required pre-change investigation.
D:Reapply the approved bundle after controlled review of production-only changes may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that resolved job settings differ from the versioned bundle manifest.