Databricks Certified Data Engineer Associate Practice Test 81
Question 800
At Regional telecom operator, the Unity Catalog lineage workflow handles subscriber provisioning logs. The problem is that subscriber_provisioning_logs_metro Gold dashboard shows wrong revenue and owners cannot trace the upstream dataset. Evidence: lineage view is missing a newly added silver transformation. Constraint: impact analysis needs upstream and downstream dependency visibility. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which change can be introduced with minimal production risk?
- Rely on file naming conventions as proof of full lineage at Regional telecom operator
- Instrument or update the supported job dependency record before release
- Use Unity Catalog lineage and supported pipeline metadata to trace data dependencies
- Compare inferred lineage to an actual source-to-gold test transformation
Explanation
Correct Answer
B:Instrument or update the supported job dependency record before release provides a bounded way to introduce the approved solution into subscriber_provisioning_logs_metro and detect an early mismatch. The existing signal is lineage view is missing a newly added silver transformation; a staged change protects the service requirement (impact analysis needs upstream and downstream dependency visibility) and makes rollback practical.
Incorrect Answers
A:Names alone cannot prove transformations and dependencies. The proposed action (Rely on file naming conventions as proof of full lineage at Regional telecom operator) does not meet the requirement that impact analysis needs upstream and downstream dependency visibility, nor does it constitute the controlled implementation step.
C:Use Unity Catalog lineage and supported pipeline metadata to trace data dependencies is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the subscriber_provisioning_logs_metro scenario, the observed evidence is that lineage view is missing a newly added silver transformation; the requirement remains: impact analysis needs upstream and downstream dependency visibility.
D:Compare inferred lineage to an actual source-to-gold test transformation is a one-time acceptance check. It does not supply the controlled implementation step for subscriber_provisioning_logs_metro; a passing test cannot substitute for the specific task now required.
Question 801
A data engineer at Cloud gaming cooperative is responsible for match-session events. The bundle environment promotion isolation issue is that match_session_events_urban test run overwrites production tables because both deployment targets share one catalog. The supporting evidence is that resolved bundle targets point to the same UC prefix. The engineer must ensure that development tests must not mutate production data. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. What evidence establishes that correct output is maintained?
- Use identical production destinations and hope developers avoid dangerous input at Cloud gaming cooperative
- Compare target resolution, workspace URL and catalog bindings for match_session_events_urban
- Run destructive test inputs and verify production table remains unchanged
- Deploy a nonproduction copy with unique resource namespace and service identity
Explanation
Correct Answer
C:Run destructive test inputs and verify production table remains unchanged directly tests the behavior that should change following the correction. With the observed evidence (resolved bundle targets point to the same UC prefix), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that development tests must not mutate production data.
Incorrect Answers
A:Shared output targets defeat environment isolation. The proposed action (Use identical production destinations and hope developers avoid dangerous input at Cloud gaming cooperative) does not meet the requirement that development tests must not mutate production data, nor does it constitute the post-change acceptance test.
B:Compare target resolution, workspace URL and catalog bindings for match_session_events_urban could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The match_session_events_urban workflow still requires that development tests must not mutate production data.
D:Deploy a nonproduction copy with unique resource namespace and service identity may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that resolved bundle targets point to the same UC prefix. This matters here because a data engineer at Cloud gaming cooperative is responsible for match-session events. The bundle environment….
Question 802
At Wholesale produce exchange, the trade settlement files workflow depends on trade_settlement_files_metro. Engineers report that an interactive notebook cluster runs trade_settlement_files_metro every minute even when analysts are offline; the evidence is that billing shows idle all-purpose compute for a deterministic scheduled transformation. They require that the pipeline needs reproducible cost-controlled scheduled runs. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which ongoing alert or metric is most relevant?
- Choose suitable job or serverless compute for the scheduled trade_settlement_files_metro task rather than an always-on interactive cluster
- Watch idle compute costs and missed trade_settlement_files_metro job triggers
- Verify scheduled execution succeeds and compute releases after the run
- Resize the interactive cluster to its maximum size and keep it on overnight at Wholesale produce exchange
Explanation
Correct Answer
B:Watch idle compute costs and missed trade_settlement_files_metro job triggers creates recurring visibility into the failure mode (an interactive notebook cluster runs trade_settlement_files_metro every minute even when analysts are offline). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while the pipeline needs reproducible cost-controlled scheduled runs.
Incorrect Answers
A:Choose suitable job or serverless compute for the scheduled trade_settlement_files_metro task rather than an always-on interactive cluster is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the trade_settlement_files_metro scenario, the observed evidence is that billing shows idle all-purpose compute for a deterministic scheduled transformation; the requirement remains: the pipeline needs reproducible cost-controlled scheduled runs.
C:Verify scheduled execution succeeds and compute releases after the run is a one-time acceptance check. It does not supply continuing operational detection for trade_settlement_files_metro; a passing test cannot substitute for the specific task now required.
D:Larger always-on interactive compute increases waste without fixing scheduling. The proposed action (Resize the interactive cluster to its maximum size and keep it on overnight at Wholesale produce exchange) does not meet the requirement that the pipeline needs reproducible cost-controlled scheduled runs, nor does it constitute continuing operational detection.
Question 803
The admission_scan_events_central pipeline at Regional sporting venue has this failure: admission_scan_events_central final export shows success even though an optional upstream data audit failed. The team observed that the team treats final task status as whole-pipeline correctness. Business requirement: operational alerts must reflect required DAG path failures. The first acceptance run passed. The current task is to detect future regressions and changes over time. What should be tracked as an operational early warning?
- Monitor only the last task while ignoring failed required upstream nodes at Regional sporting venue
- Monitor repeated task skips, failure rate and false-green executions for admission_scan_events_central
- Simulate a required task failure and verify publish stops and alert fires
- Set success criteria and notifications on required tasks and aggregate job outcome
Explanation
Correct Answer
B:Monitor repeated task skips, failure rate and false-green executions for admission_scan_events_central creates recurring visibility into the failure mode (admission_scan_events_central final export shows success even though an optional upstream data audit failed). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while operational alerts must reflect required DAG path failures.
Incorrect Answers
A:Last-task success does not prove full dag correctness. The proposed action (Monitor only the last task while ignoring failed required upstream nodes at Regional sporting venue) does not meet the requirement that operational alerts must reflect required DAG path failures, nor does it constitute continuing operational detection.
C:Simulate a required task failure and verify publish stops and alert fires is a one-time acceptance check. It does not supply continuing operational detection for admission_scan_events_central; a passing test cannot substitute for the specific task now required.
D:Set success criteria and notifications on required tasks and aggregate job outcome is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the admission_scan_events_central scenario, the observed evidence is that the team treats final task status as whole-pipeline correctness; the requirement remains: operational alerts must reflect required DAG path failures.
Question 804
At Maritime safety agency, the vessel position events workflow depends on vessel_position_events_delta. Engineers report that vessel_position_events_delta tracks successive device configurations and needs the most recent per device; the evidence is that the raw table contains equal device_id rows with varying event timestamps. They require that exactly one latest row per device must be selected using deterministic tie-breaks. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which ongoing alert or metric is most relevant?
- Call dropDuplicates(device_id) without an explicit ordering rule at Maritime safety agency
- Insert tied timestamps and verify deterministic version selection
- Monitor tied-key anomalies and late configuration records for vessel_position_events_delta
- Apply row_number over partitionBy(device_id) orderBy(timestamp desc, version desc)
Explanation
Correct Answer
C:Monitor tied-key anomalies and late configuration records for vessel_position_events_delta creates recurring visibility into the failure mode (vessel_position_events_delta tracks successive device configurations and needs the most recent per device). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while exactly one latest row per device must be selected using deterministic tie-breaks.
Incorrect Answers
A:An unordered dedup does not guarantee selection of the newest record. The proposed action (Call dropDuplicates(device_id) without an explicit ordering rule at Maritime safety agency) does not meet the requirement that exactly one latest row per device must be selected using deterministic tie-breaks, nor does it constitute continuing operational detection.
B:Insert tied timestamps and verify deterministic version selection is a one-time acceptance check. It does not supply continuing operational detection for vessel_position_events_delta; a passing test cannot substitute for the specific task now required.
D:Apply row_number over partitionBy(device_id) orderBy(timestamp desc, version desc) is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the vessel_position_events_delta scenario, the observed evidence is that the raw table contains equal device_id rows with varying event timestamps; the requirement remains: exactly one latest row per device must be selected using deterministic tie-breaks.
Question 805
A data engineer at Regional hospital network is responsible for clinical appointment reports. The executor OOM issue is that clinical_appointment_reports_island transformation intermittently fails with executor out-of-memory. The supporting evidence is that Spark UI shows large skewed partitions and executor lost errors. The engineer must ensure that the job must process full input safely without excluding valid rows. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What analysis is necessary before deciding on remediation?
- Pilot repartitioning or right-sizing against the failing data window
- Choose balanced partitioning and appropriate executor memory for the workload
- Filter out the largest partition without business approval at Regional hospital network
- Inspect task partition sizes, execution plan and memory overhead for clinical_appointment_reports_island
Explanation
Correct Answer
D:Inspect task partition sizes, execution plan and memory overhead for clinical_appointment_reports_island directly investigates the evidence (Spark UI shows large skewed partitions and executor lost errors) and distinguishes possible causes of clinical_appointment_reports_island transformation intermittently fails with executor out-of-memory. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: the job must process full input safely without excluding valid rows.
Incorrect Answers
A:Pilot repartitioning or right-sizing against the failing data window may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that Spark UI shows large skewed partitions and executor lost errors. This matters here because a data engineer at Regional hospital network is responsible for clinical appointment reports. The executor OOM issue….
B:Choose balanced partitioning and appropriate executor memory for the workload is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the clinical_appointment_reports_island scenario, the observed evidence is that Spark UI shows large skewed partitions and executor lost errors; the requirement remains: the job must process full input safely without excluding valid rows.
C:Dropping data to avoid oom violates completeness. The proposed action (Filter out the largest partition without business approval at Regional hospital network) does not meet the requirement that the job must process full input safely without excluding valid rows, nor does it constitute the required pre-change investigation.
Question 806
At Municipal waste service, the Auto Loader checkpoint workflow handles collection telemetry. The problem is that a streaming file reader for collection_telemetry_island reprocesses historical data after every job restart. Evidence: the streaming query creates a new checkpoint directory on each run. Constraint: a restart must resume discovered files instead of replaying all existing input. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. Which test should pass before the new flow is accepted?
- Delete checkpoint state as part of each successful load at Municipal waste service
- Inspect checkpoint continuity, discovery progress and input file identifiers
- Restart after a micro-batch and confirm no duplicate file ingestion
- Deploy the stream with a durable checkpoint and restart the same logical query
Explanation
Correct Answer
C:Restart after a micro-batch and confirm no duplicate file ingestion directly tests the behavior that should change after the correction. With the observed evidence (the streaming query creates a new checkpoint directory on each run), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that a restart must resume discovered files instead of replaying all existing input. Here, the relevant situation is: the Auto Loader checkpoint workflow handles collection telemetry. The problem is that a streaming file reader for….
Incorrect Answers
A:Deleting checkpoint progress can cause full replay. The proposed action (Delete checkpoint state as part of each successful load at Municipal waste service) does not meet the requirement that a restart must resume discovered files instead of replaying all existing input, nor does it constitute the post-change acceptance test.
B:Inspect checkpoint continuity, discovery progress and input file identifiers could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The collection_telemetry_island workflow still requires that a restart must resume discovered files instead of replaying all existing input.
D:Deploy the stream with a durable checkpoint and restart the same logical query may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that the streaming query creates a new checkpoint directory on each run. Here, the relevant situation is: the Auto Loader checkpoint workflow handles collection telemetry. The problem is that a streaming file reader for….
Question 807
At Public weather agency, the Lakeflow Connect standard connector workflow handles station observation files. The problem is that a station_observation_files_highland source has a compatible connector but existing custom scripts have divergent retries. Evidence: logs show different authentication and pagination paths across team-owned scripts. Constraint: ingestion must be standardized under governed destinations. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which investigation is the strongest first step?
- Configure the connector and validate target schema and schedule in staging
- Let every notebook invent a new unscheduled direct source sync at Public weather agency
- Identify connector coverage, source permissions and supported incremental behavior for station_observation_files_highland
- Use the supported Lakeflow Connect standard connector with scoped source access
Explanation
Correct Answer
C:Identify connector coverage, source permissions and supported incremental behavior for station_observation_files_highland directly investigates the evidence (logs show different authentication and pagination paths across team-owned scripts) and distinguishes possible causes of a station_observation_files_highland source has a compatible connector but existing custom scripts have divergent retries. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: ingestion must be standardized under governed destinations.
Incorrect Answers
A:Configure the connector and validate target schema and schedule in staging may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that logs show different authentication and pagination paths across team-owned scripts. Here, the relevant situation is: the Lakeflow Connect standard connector workflow handles station observation files. The problem is that a….
B:Uncoordinated custom pulls are less governed and harder to recover. The proposed action (Let every notebook invent a new unscheduled direct source sync at Public weather agency) does not meet the requirement that ingestion must be standardized under governed destinations, nor does it constitute the required pre-change investigation.
D:Use the supported Lakeflow Connect standard connector with scoped source access is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the station_observation_files_highland scenario, the observed evidence is that logs show different authentication and pagination paths across team-owned scripts; the requirement remains: ingestion must be standardized under governed destinations.
Question 808
Consider the experiment_records_north workload at University research consortium: experiment_records_north dataset is shared with a third party that does not run Databricks. Inspection found that recipient cannot authenticate to a Databricks workspace but supports the open protocol. The governed outcome must ensure that the partner needs supported read-only interoperability. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. How should the team verify the original fault has been addressed?
- Issue recipient configuration using supported sharing workflow and test access
- Verify recipient client protocol and data-access obligations for experiment_records_north
- Read shared sample data through the external client and validate scope
- Require every external organization to obtain provider workspace administrator access at University research consortium
Explanation
Correct Answer
C:Read shared sample data through the external client and validate scope directly tests the behavior that should change following the correction. With the observed evidence (recipient cannot authenticate to a Databricks workspace but supports the open protocol), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that the partner needs supported read-only interoperability.
Incorrect Answers
A:Issue recipient configuration using supported sharing workflow and test access may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that recipient cannot authenticate to a Databricks workspace but supports the open protocol. This matters here because consider the experiment_records_north workload at University research consortium: experiment_records_north dataset….
B:Verify recipient client protocol and data-access obligations for experiment_records_north could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The experiment_records_north workflow still requires that the partner needs supported read-only interoperability.
D:Workspace administration is not needed for open read-only sharing. The proposed action (Require every external organization to obtain provider workspace administrator access at University research consortium) does not meet the requirement that the partner needs supported read-only interoperability, nor does it constitute the post-change acceptance test.
Question 809
At Consumer finance platform, the For each fanout task workflow handles loan repayment events. The problem is that loan_repayment_events_east pipeline runs one fixed notebook for every region using copied task definitions. Evidence: job configuration repeats the same notebook 40 times with only region parameters changed. Constraint: the regional workload must run repeatably with bounded parallelism. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which investigation is the strongest first step?
- Pilot the loop on two regions with per-iteration parameters and retry policy
- Inspect parameter list, per-region failures and concurrency limits for loan_repayment_events_east
- Use a Lakeflow For each task over region inputs with controlled concurrency
- Create one unbounded loop that ignores iteration failures at Consumer finance platform
Explanation
Correct Answer
B:Inspect parameter list, per-region failures and concurrency limits for loan_repayment_events_east directly investigates the evidence (job configuration repeats the same notebook 40 times with only region parameters changed) and distinguishes possible causes of loan_repayment_events_east pipeline runs one fixed notebook for every region using copied task definitions. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: the regional workload must run repeatably with bounded parallelism.
Incorrect Answers
A:Pilot the loop on two regions with per-iteration parameters and retry policy may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that job configuration repeats the same notebook 40 times with only region parameters changed. Here, the relevant situation is: the For each fanout task workflow handles loan repayment events. The problem is that loan_repayment_events_east….
C:Use a Lakeflow For each task over region inputs with controlled concurrency is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the loan_repayment_events_east scenario, the observed evidence is that job configuration repeats the same notebook 40 times with only region parameters changed; the requirement remains: the regional workload must run repeatably with bounded parallelism.
D:Unbounded uncontrolled iteration can overwhelm compute and hide errors. The proposed action (Create one unbounded loop that ignores iteration failures at Consumer finance platform) does not meet the requirement that the regional workload must run repeatably with bounded parallelism, nor does it constitute the required pre-change investigation.
Question 810
A data engineer at Public weather agency is responsible for station observation files. The skewed join key issue is that station_observation_files_rural join hangs on one task while almost every task completes quickly. The supporting evidence is that Spark UI reveals one partition contains most rows for a hot customer key. The engineer must ensure that optimization must preserve all correct joined rows. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What is the safest migration or deployment sequence?
- Address skew with AQE skew handling or justified key-salting strategy
- Test skew-aware join on representative hot and cold key samples
- Verify exact join output and reduced straggler duration
- Drop the most frequent business key to balance partitions at Public weather agency
Explanation
Correct Answer
B:Test skew-aware join on representative hot and cold key samples provides a bounded way to introduce the approved solution into station_observation_files_rural and detect an early mismatch. The existing signal is Spark UI reveals one partition contains most rows for a hot customer key; a staged change protects the service requirement (optimization must preserve all correct joined rows) and makes rollback practical.
Incorrect Answers
A:Address skew with AQE skew handling or justified key-salting strategy is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the station_observation_files_rural scenario, the observed evidence is that Spark UI reveals one partition contains most rows for a hot customer key; the requirement remains: optimization must preserve all correct joined rows.
C:Verify exact join output and reduced straggler duration is a one-time acceptance check. It does not supply the controlled implementation step for station_observation_files_rural; a passing test cannot substitute for the specific task now required.
D:Dropping a hot key loses legitimate data. The proposed action (Drop the most frequent business key to balance partitions at Public weather agency) does not meet the requirement that optimization must preserve all correct joined rows, nor does it constitute the controlled implementation step.