Databricks Certified Data Engineer Associate Practice Test 85
Question 840
At Air quality nonprofit, the Photon and execution engine workflow handles pollution measurements. The problem is that the pollution_measurements_north SQL aggregation runs slowly on eligible compute despite a stable dataset. Evidence: query profiling identifies CPU-bound SQL operations with little external I/O. Constraint: performance changes must be measured at equivalent compute and data volume. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which investigation is the strongest first step?
- Enable the appropriate engine on a pilot compute configuration and compare plans
- Assume Photon eliminates every network, skew and access-permission problem at Air quality nonprofit
- Baseline SQL plan, engine configuration and resource use for pollution_measurements_north
- Evaluate Photon-capable compute for supported SQL and DataFrame workloads
Explanation
Correct Answer
C:Baseline SQL plan, engine configuration and resource use for pollution_measurements_north directly investigates the evidence (query profiling identifies CPU-bound SQL operations with little external I/O) and distinguishes possible causes of the pollution_measurements_north SQL aggregation runs slowly on eligible compute despite a stable dataset. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: performance changes must be measured at equivalent compute and data volume.
Incorrect Answers
A:Enable the appropriate engine on a pilot compute configuration and compare plans may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that query profiling identifies CPU-bound SQL operations with little external I/O. Here, the relevant situation is: the Photon and execution engine workflow handles pollution measurements. The problem is that the….
B:An execution engine cannot cure unrelated input or authorization constraints. The proposed action (Assume Photon eliminates every network, skew and access-permission problem at Air quality nonprofit) does not meet the requirement that performance changes must be measured at equivalent compute and data volume, nor does it constitute the required pre-change investigation.
D:Evaluate Photon-capable compute for supported SQL and DataFrame workloads is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the pollution_measurements_north scenario, the observed evidence is that query profiling identifies CPU-bound SQL operations with little external I/O; the requirement remains: performance changes must be measured at equivalent compute and data volume.
Question 841
A data engineer at City library association is responsible for borrower circulation events. The JDBC watermark extraction issue is that the borrower_circulation_events_south SQL source is polled hourly but every run extracts its entire ten-year table. The supporting evidence is that source query logs show full scans despite an indexed updated_at column. The engineer must ensure that new and changed rows must load with a replay-safe high-water mark. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. Which durable Databricks control best addresses the requirement?
- Verify update timestamp semantics, primary keys and late-arrival window for borrower_circulation_events_south
- Monitor watermark gaps, source query cost and late records for borrower_circulation_events_south
- Use bounded incremental JDBC reads with a persisted watermark and merge keys
- Advance the stored watermark before persisting the extracted batch at City library association
Explanation
Correct Answer
C:Use bounded incremental JDBC reads with a persisted watermark and merge keys is the intended durable approach for borrower_circulation_events_south: it addresses the verified problem (the borrower_circulation_events_south SQL source is polled hourly but every run extracts its entire ten-year table) while preserving the required behavior (new and changed rows must load with a replay-safe high-water mark). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
A:Verify update timestamp semantics, primary keys and late-arrival window for borrower_circulation_events_south could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The borrower_circulation_events_south workflow still requires that new and changed rows must load with a replay-safe high-water mark.
B:Monitor watermark gaps, source query cost and late records for borrower_circulation_events_south would help detect future recurrence, but metrics alone do not deliver the target approach. The borrower_circulation_events_south requirement is that new and changed rows must load with a replay-safe high-water mark.
D:Premature watermark advancement can lose rows on failure. The proposed action (Advance the stored watermark before persisting the extracted batch at City library association) does not meet the requirement that new and changed rows must load with a replay-safe high-water mark, nor does it constitute the target approach.
Question 842
A data engineer at Community energy cooperative is responsible for smart meter readings. The shuffle partition tuning issue is that smart_meter_readings_delta aggregation spends most time scheduling thousands of tiny shuffle tasks. The supporting evidence is that Spark UI shows very low per-task data size and excessive scheduling overhead. The engineer must ensure that performance changes must be benchmarked against the same output. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. What analysis is necessary before deciding on remediation?
- Increase shuffle partitions without measuring task size or skew at Community energy cooperative
- Inspect shuffle output size, task duration and partition setting for smart_meter_readings_delta
- Tune spark.sql.shuffle.partitions or AQE behavior based on measured shuffle workload
- Apply a bounded change in staging and compare resource cost
Explanation
Correct Answer
B:Inspect shuffle output size, task duration and partition setting for smart_meter_readings_delta directly investigates the evidence (Spark UI shows very low per-task data size and excessive scheduling overhead) and distinguishes possible causes of smart_meter_readings_delta aggregation spends most time scheduling thousands of tiny shuffle tasks. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: performance changes must be benchmarked against the same output.
Incorrect Answers
A:Arbitrary partition expansion can increase overhead. The proposed action (Increase shuffle partitions without measuring task size or skew at Community energy cooperative) does not meet the requirement that performance changes must be benchmarked against the same output, nor does it constitute the required pre-change investigation.
C:Tune spark.sql.shuffle.partitions or AQE behavior based on measured shuffle workload is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the smart_meter_readings_delta scenario, the observed evidence is that Spark UI shows very low per-task data size and excessive scheduling overhead; the requirement remains: performance changes must be benchmarked against the same output.
D:Apply a bounded change in staging and compare resource cost may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that Spark UI shows very low per-task data size and excessive scheduling overhead. This matters here because a data engineer at Community energy cooperative is responsible for smart meter readings. The shuffle partition….
Question 843
The land_cover_observations_island pipeline at Satellite imagery startup has this failure: a land_cover_observations_island source has a compatible connector but existing custom scripts have divergent retries. The team observed that logs show different authentication and pagination paths across team-owned scripts. Business requirement: ingestion must be standardized under governed destinations. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. What test most directly proves the requirement is satisfied?
- Configure the connector and validate target schema and schedule in staging
- Identify connector coverage, source permissions and supported incremental behavior for land_cover_observations_island
- Compare loaded rows and source bookmarks before switching production
- Let every notebook invent a new unscheduled direct source sync at Satellite imagery startup
Explanation
Correct Answer
C:Compare loaded rows and source bookmarks before switching production directly tests the behavior that should change after the correction. With the observed evidence (logs show different authentication and pagination paths across team-owned scripts), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that ingestion must be standardized under governed destinations. This matters here because the land_cover_observations_island pipeline at Satellite imagery startup has this failure: a….
Incorrect Answers
A:Configure the connector and validate target schema and schedule in staging may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that logs show different authentication and pagination paths across team-owned scripts. This matters here because the land_cover_observations_island pipeline at Satellite imagery startup has this failure: a….
B:Identify connector coverage, source permissions and supported incremental behavior for land_cover_observations_island could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The land_cover_observations_island workflow still requires that ingestion must be standardized under governed destinations.
D:Uncoordinated custom pulls are less governed and harder to recover. The proposed action (Let every notebook invent a new unscheduled direct source sync at Satellite imagery startup) does not meet the requirement that ingestion must be standardized under governed destinations, nor does it constitute the post-change acceptance test.
Question 844
At Air quality nonprofit, the Delta transaction isolation workflow handles pollution measurements. The problem is that two concurrent writers update pollution_measurements_west and a reader sometimes sees inconsistent exported copies. Evidence: the write log shows independent files were replaced while an ETL job was reading them. Constraint: readers require consistent committed versions without pausing ingest. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which change can be introduced with minimal production risk?
- Store pollution_measurements_west as a Delta table and rely on atomic transaction-log commits
- Allow direct writers to replace arbitrary Parquet files while readers scan them at Air quality nonprofit
- Read pollution_measurements_west during overlapping test writes and verify a consistent committed snapshot
- Convert the staging copy and cut over readers only after commit and rollback tests
Explanation
Correct Answer
D:Convert the staging copy and cut over readers only after commit and rollback tests provides a bounded way to introduce the approved solution into pollution_measurements_west and detect an early mismatch. The existing signal is the write log shows independent files were replaced while an ETL job was reading them; a staged change protects the service requirement (readers require consistent committed versions without pausing ingest) and makes rollback practical.
Incorrect Answers
A:Store pollution_measurements_west as a Delta table and rely on atomic transaction-log commits is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the pollution_measurements_west scenario, the observed evidence is that the write log shows independent files were replaced while an ETL job was reading them; the requirement remains: readers require consistent committed versions without pausing ingest.
B:Uncoordinated file replacement cannot provide delta transaction guarantees. The proposed action (Allow direct writers to replace arbitrary Parquet files while readers scan them at Air quality nonprofit) does not meet the requirement that readers require consistent committed versions without pausing ingest, nor does it constitute the controlled implementation step.
C:Read pollution_measurements_west during overlapping test writes and verify a consistent committed snapshot is a one-time acceptance check. It does not supply the controlled implementation step for pollution_measurements_west; a passing test cannot substitute for the specific task now required.
Question 845
Consider the fare_validation_events_rural workload at Public transport authority: fare_validation_events_rural gold report launches before silver validation finishes. Inspection found that a task graph shows report and validation as parallel children of ingestion. The governed outcome must ensure that downstream publishing must not start until all required upstream checks succeed. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which continuing signal best reveals recurrence or drift?
- Declare an explicit dependency from gold publishing to successful validation tasks
- Pause the validator and verify publish does not start early
- Monitor dependency skips, failures and unexpected concurrent starts for fare_validation_events_rural
- Schedule every task at the same clock time and hope upstream processing completes at Public transport authority
Explanation
Correct Answer
C:Monitor dependency skips, failures and unexpected concurrent starts for fare_validation_events_rural creates recurring visibility into the failure mode (fare_validation_events_rural gold report launches before silver validation finishes). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while downstream publishing must not start until all required upstream checks succeed.
Incorrect Answers
A:Declare an explicit dependency from gold publishing to successful validation tasks is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the fare_validation_events_rural scenario, the observed evidence is that a task graph shows report and validation as parallel children of ingestion; the requirement remains: downstream publishing must not start until all required upstream checks succeed.
B:Pause the validator and verify publish does not start early is a one-time acceptance check. It does not supply continuing operational detection for fare_validation_events_rural; a passing test cannot substitute for the specific task now required.
D:Clock coincidence does not enforce dependency correctness. The proposed action (Schedule every task at the same clock time and hope upstream processing completes at Public transport authority) does not meet the requirement that downstream publishing must not start until all required upstream checks succeed, nor does it constitute continuing operational detection.
Question 846
Consider the smart_meter_readings_central workload at Community energy cooperative: smart_meter_readings_central analysts have SELECT on a table but get a permission error at query time. Inspection found that access review shows missing USE CATALOG on its parent catalog. The governed outcome must ensure that users must read only the authorized governed object. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which continuing signal best reveals recurrence or drift?
- Grant required parent USE CATALOG and USE SCHEMA along with table SELECT
- Monitor failed parent-path access and excessive grants for smart_meter_readings_central
- Query the table as an analyst and confirm sibling data remains denied
- Grant global metastore administration to fix one table query at Community energy cooperative
Explanation
Correct Answer
B:Monitor failed parent-path access and excessive grants for smart_meter_readings_central creates recurring visibility into the failure mode (smart_meter_readings_central analysts have SELECT on a table but get a permission error at query time). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while users must read only the authorized governed object.
Incorrect Answers
A:Grant required parent USE CATALOG and USE SCHEMA along with table SELECT is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the smart_meter_readings_central scenario, the observed evidence is that access review shows missing USE CATALOG on its parent catalog; the requirement remains: users must read only the authorized governed object.
C:Query the table as an analyst and confirm sibling data remains denied is a one-time acceptance check. It does not supply continuing operational detection for smart_meter_readings_central; a passing test cannot substitute for the specific task now required.
D:Global admin rights exceed simple read access needs. The proposed action (Grant global metastore administration to fix one table query at Community energy cooperative) does not meet the requirement that users must read only the authorized governed object, nor does it constitute continuing operational detection.
Question 847
Consider the factory_defect_records_urban workload at Manufacturing quality network: factory_defect_records_urban silver records pass ingestion despite negative billed quantities. Inspection found that validation sample identifies impossible values in otherwise well-formed JSON. The governed outcome must ensure that bad records need a defined reject/quarantine policy without hiding valid data. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. How should the team verify the original fault has been addressed?
- Inject invalid rows and verify correct drop, fail or quarantine behavior
- Delete invalid rows silently and report the entire batch as error-free at Manufacturing quality network
- Quantify invalid predicate occurrences and source impact for factory_defect_records_urban
- Roll out expectation checks and a quarantine audit before enforcing production policy
Explanation
Correct Answer
A:Inject invalid rows and verify correct drop, fail or quarantine behavior directly tests the behavior that should change following the correction. With the observed evidence (validation sample identifies impossible values in otherwise well-formed JSON), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that bad records need a defined reject/quarantine policy without hiding valid data.
Incorrect Answers
B:Silent loss hides quality issues and masks upstream defects. The proposed action (Delete invalid rows silently and report the entire batch as error-free at Manufacturing quality network) does not meet the requirement that bad records need a defined reject/quarantine policy without hiding valid data, nor does it constitute the post-change acceptance test.
C:Quantify invalid predicate occurrences and source impact for factory_defect_records_urban could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The factory_defect_records_urban workflow still requires that bad records need a defined reject/quarantine policy without hiding valid data.
D:Roll out expectation checks and a quarantine audit before enforcing production policy may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that validation sample identifies impossible values in otherwise well-formed JSON. This matters here because consider the factory_defect_records_urban workload at Manufacturing quality network: factory_defect_records_urban….
Question 848
Consider the water_quality_readings_island workload at Aquaculture farming group: the water_quality_readings_island import stores files in personal workspace paths inaccessible to scheduled service principals. Inspection found that a job run fails to read a path that works under one engineer's user session. The governed outcome must ensure that source files must be reachable through governed identity and stable cloud storage. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. Which data-engineering approach resolves the documented weakness?
- Embed a personal access token and home-directory file path in production code at Aquaculture farming group
- Land files in a permitted external or managed Unity Catalog volume
- Inspect storage location, Unity Catalog volume grants and job run-as permissions for water_quality_readings_island
- Monitor missing-volume grants and inaccessible source paths for water_quality_readings_island
Explanation
Correct Answer
B:Land files in a permitted external or managed Unity Catalog volume is the intended durable approach for water_quality_readings_island: it addresses the verified problem (the water_quality_readings_island import stores files in personal workspace paths inaccessible to scheduled service principals) while preserving the required behavior (source files must be reachable through governed identity and stable cloud storage). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
A:Personal paths and identities do not provide durable service access. The proposed action (Embed a personal access token and home-directory file path in production code at Aquaculture farming group) does not meet the requirement that source files must be reachable through governed identity and stable cloud storage, nor does it constitute the target approach.
C:Inspect storage location, Unity Catalog volume grants and job run-as permissions for water_quality_readings_island could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The water_quality_readings_island workflow still requires that source files must be reachable through governed identity and stable cloud storage.
D:Monitor missing-volume grants and inaccessible source paths for water_quality_readings_island would help detect future recurrence, but metrics alone do not deliver the target approach. The water_quality_readings_island requirement is that source files must be reachable through governed identity and stable cloud storage.
Question 849
At Public water district, the liquid clustering for dynamic filters workflow handles pipeline flow metrics. The problem is that pipeline_flow_metrics_west Delta table suffers frequent filter-pattern changes and costly partition rewrites. Evidence: workload history alternates date and customer search predicates across teams. Constraint: layout optimization must adapt without a rigid proliferating partition tree. The team has not selected a remedy. It first needs evidence that distinguishes the actual dependency or data fault. Which investigation is the strongest first step?
- Consider Liquid Clustering on appropriate columns with supported table features
- Create thousands of tiny static partitions for every possible predicate at Public water district
- Enable clustering on a pilot target and run supported maintenance operations
- Profile common filters, table size and clustering support for pipeline_flow_metrics_west
Explanation
Correct Answer
D:Profile common filters, table size and clustering support for pipeline_flow_metrics_west directly investigates the evidence (workload history alternates date and customer search predicates across teams) and distinguishes possible causes of pipeline_flow_metrics_west Delta table suffers frequent filter-pattern changes and costly partition rewrites. It establishes what must be corrected without assuming a fix before verifying the stated business requirement: layout optimization must adapt without a rigid proliferating partition tree.
Incorrect Answers
A:Consider Liquid Clustering on appropriate columns with supported table features is a plausible lasting design once chosen, but it does not perform the required pre-change investigation at this stage. In the pipeline_flow_metrics_west scenario, the observed evidence is that workload history alternates date and customer search predicates across teams; the requirement remains: layout optimization must adapt without a rigid proliferating partition tree.
B:Overpartitioning drives metadata overhead and inflexible layout. The proposed action (Create thousands of tiny static partitions for every possible predicate at Public water district) does not meet the requirement that layout optimization must adapt without a rigid proliferating partition tree, nor does it constitute the required pre-change investigation.
C:Enable clustering on a pilot target and run supported maintenance operations may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the required pre-change investigation, especially given the evidence that workload history alternates date and customer search predicates across teams. Here, the relevant situation is: the liquid clustering for dynamic filters workflow handles pipeline flow metrics. The problem is that….
Question 850
The render_job_telemetry_metro pipeline at Media postproduction team has this failure: render_job_telemetry_metro aggregation spends most time scheduling thousands of tiny shuffle tasks. The team observed that Spark UI shows very low per-task data size and excessive scheduling overhead. Business requirement: performance changes must be benchmarked against the same output. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. What should be done next to deploy the approved solution?
- Increase shuffle partitions without measuring task size or skew at Media postproduction team
- Tune spark.sql.shuffle.partitions or AQE behavior based on measured shuffle workload
- Apply a bounded change in staging and compare resource cost
- Rerun identical aggregation and verify result parity and faster completion
Explanation
Correct Answer
C:Apply a bounded change in staging and compare resource cost provides a bounded way to introduce the approved solution into render_job_telemetry_metro and detect an early mismatch. The existing signal is Spark UI shows very low per-task data size and excessive scheduling overhead; a staged change protects the service requirement (performance changes must be benchmarked against the same output) and makes rollback practical.
Incorrect Answers
A:Arbitrary partition expansion can increase overhead. The proposed action (Increase shuffle partitions without measuring task size or skew at Media postproduction team) does not meet the requirement that performance changes must be benchmarked against the same output, nor does it constitute the controlled implementation step.
B:Tune spark.sql.shuffle.partitions or AQE behavior based on measured shuffle workload is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the render_job_telemetry_metro scenario, the observed evidence is that Spark UI shows very low per-task data size and excessive scheduling overhead; the requirement remains: performance changes must be benchmarked against the same output.
D:Rerun identical aggregation and verify result parity and faster completion is a one-time acceptance check. It does not supply the controlled implementation step for render_job_telemetry_metro; a passing test cannot substitute for the specific task now required.