Databricks Certified Data Engineer Associate Practice Test 87
Question 860
At Coastal freight alliance, the Auto Loader schema evolution workflow handles shipment manifests. The problem is that a new optional field appears in shipment_manifests_west JSON files and the ingestion job starts rejecting rows. Evidence: failure details identify a field absent from the previously inferred schema. Constraint: new attributes must be captured without silently corrupting existing columns. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. What governed target implementation should the engineer design?
- Configure supported Auto Loader schema evolution with an explicit schema location and review process
- Inspect schema inference, evolution mode and rescued data behavior on shipment_manifests_west
- Monitor schema changes, rescued columns and parsing failures for shipment_manifests_west
- Silently drop every unexpected field and label ingestion complete at Coastal freight alliance
Explanation
Correct Answer
A:Configure supported Auto Loader schema evolution with an explicit schema location and review process is the intended durable approach for shipment_manifests_west: it addresses the verified problem (a new optional field appears in shipment_manifests_west JSON files and the ingestion job starts rejecting rows) while preserving the required behavior (new attributes must be captured without silently corrupting existing columns). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
B:Inspect schema inference, evolution mode and rescued data behavior on shipment_manifests_west could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The shipment_manifests_west workflow still requires that new attributes must be captured without silently corrupting existing columns.
C:Monitor schema changes, rescued columns and parsing failures for shipment_manifests_west would help detect future recurrence, but metrics alone do not deliver the target approach. The shipment_manifests_west requirement is that new attributes must be captured without silently corrupting existing columns.
D:Dropping added fields hides source changes and loses information. The proposed action (Silently drop every unexpected field and label ingestion complete at Coastal freight alliance) does not meet the requirement that new attributes must be captured without silently corrupting existing columns, nor does it constitute the target approach.
Question 861
The subscriber_provisioning_logs_metro pipeline at Regional telecom operator has this failure: subscriber_provisioning_logs_metro Gold dashboard shows wrong revenue and owners cannot trace the upstream dataset. The team observed that lineage view is missing a newly added silver transformation. Business requirement: impact analysis needs upstream and downstream dependency visibility. The new approach is deployed. The current task is to prove correct output and authorized behavior, not just successful installation. What test most directly proves the requirement is satisfied?
- Rely on file naming conventions as proof of full lineage at Regional telecom operator
- Instrument or update the supported job dependency record before release
- Compare inferred lineage to an actual source-to-gold test transformation
- Inspect lineage graph and actual query/job references for subscriber_provisioning_logs_metro
Explanation
Correct Answer
C:Compare inferred lineage to an actual source-to-gold test transformation directly tests the behavior that should change following the correction. With the observed evidence (lineage view is missing a newly added silver transformation), the acceptance result must demonstrate actual data or permission correctness, while retaining the requirement that impact analysis needs upstream and downstream dependency visibility.
Incorrect Answers
A:Names alone cannot prove transformations and dependencies. The proposed action (Rely on file naming conventions as proof of full lineage at Regional telecom operator) does not meet the requirement that impact analysis needs upstream and downstream dependency visibility, nor does it constitute the post-change acceptance test.
B:Instrument or update the supported job dependency record before release may introduce a correction, but the question does not ask for installation or migration. A rollout alone cannot replace the post-change acceptance test, especially given the evidence that lineage view is missing a newly added silver transformation. This matters here because the subscriber_provisioning_logs_metro pipeline at Regional telecom operator has this failure:….
D:Inspect lineage graph and actual query/job references for subscriber_provisioning_logs_metro could clarify the original fault, but the available evidence already establishes its scope; it is not the post-change acceptance test now requested. The subscriber_provisioning_logs_metro workflow still requires that impact analysis needs upstream and downstream dependency visibility.
Question 862
At City library association, the JDBC watermark extraction workflow handles borrower circulation events. The problem is that the borrower_circulation_events_south SQL source is polled hourly but every run extracts its entire ten-year table. Evidence: source query logs show full scans despite an indexed updated_at column. Constraint: new and changed rows must load with a replay-safe high-water mark. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which monitoring approach will detect newly emerging defects?
- Advance the stored watermark before persisting the extracted batch at City library association
- Monitor watermark gaps, source query cost and late records for borrower_circulation_events_south
- Modify an old row and verify incremental fetch plus merge applies its update
- Use bounded incremental JDBC reads with a persisted watermark and merge keys
Explanation
Correct Answer
B:Monitor watermark gaps, source query cost and late records for borrower_circulation_events_south creates recurring visibility into the failure mode (the borrower_circulation_events_south SQL source is polled hourly but every run extracts its entire ten-year table). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while new and changed rows must load with a replay-safe high-water mark.
Incorrect Answers
A:Premature watermark advancement can lose rows on failure. The proposed action (Advance the stored watermark before persisting the extracted batch at City library association) does not meet the requirement that new and changed rows must load with a replay-safe high-water mark, nor does it constitute continuing operational detection.
C:Modify an old row and verify incremental fetch plus merge applies its update is a one-time acceptance check. It does not supply continuing operational detection for borrower_circulation_events_south; a passing test cannot substitute for the specific task now required.
D:Use bounded incremental JDBC reads with a persisted watermark and merge keys is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the borrower_circulation_events_south scenario, the observed evidence is that source query logs show full scans despite an indexed updated_at column; the requirement remains: new and changed rows must load with a replay-safe high-water mark.
Question 863
At Public water district, the pipeline flow metrics workflow depends on pipeline_flow_metrics_metro. Engineers report that pipeline_flow_metrics_metro scheduled ingestion succeeds under one employee but fails under automation identity; the evidence is that UC audit shows only the employee has the required table rights. They require that the job must operate under a managed nonhuman least-privilege principal. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which controlled implementation step is appropriate now?
- Run the pipeline end-to-end using the service principal and deny unrelated catalogs
- Switch the job identity and verify its input and output access
- Share a workspace administrator's personal access token with all job owners at Public water district
- Grant the required scoped privileges to an approved service principal
Explanation
Correct Answer
B:Switch the job identity and verify its input and output access provides a bounded way to introduce the approved solution into pipeline_flow_metrics_metro and detect an early mismatch. The existing signal is UC audit shows only the employee has the required table rights; a staged change protects the service requirement (the job must operate under a managed nonhuman least-privilege principal) and makes rollback practical.
Incorrect Answers
A:Run the pipeline end-to-end using the service principal and deny unrelated catalogs is a one-time acceptance check. It does not supply the controlled implementation step for pipeline_flow_metrics_metro; a passing test cannot substitute for the specific task now required.
C:Shared personal tokens are fragile and overprivileged. The proposed action (Share a workspace administrator's personal access token with all job owners at Public water district) does not meet the requirement that the job must operate under a managed nonhuman least-privilege principal, nor does it constitute the controlled implementation step.
D:Grant the required scoped privileges to an approved service principal is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the pipeline_flow_metrics_metro scenario, the observed evidence is that UC audit shows only the employee has the required table rights; the requirement remains: the job must operate under a managed nonhuman least-privilege principal.
Question 864
Consider the treatment_billing_events_delta workload at Veterinary clinic association: a treatment_billing_events_delta enterprise SaaS table requires recurring ingestion but the team maintains a brittle custom API crawler. Inspection found that API tokens expire and source field names change without ingestion alerts. The governed outcome must ensure that the source must arrive in governed Delta tables with manageable credential rotation. The failure and requirements have been diagnosed; the team must select the lasting pipeline or governance approach. Which data-engineering approach resolves the documented weakness?
- Assess managed connector source support, permissions and refresh semantics for treatment_billing_events_delta
- Copy API secrets into a widely shared notebook and paste full responses manually at Veterinary clinic association
- Prefer a supported Lakeflow Connect managed connector with managed ingestion state
- Monitor connector lag, source authentication errors and schema drift for treatment_billing_events_delta
Explanation
Correct Answer
C:Prefer a supported Lakeflow Connect managed connector with managed ingestion state is the intended durable approach for treatment_billing_events_delta: it addresses the verified problem (a treatment_billing_events_delta enterprise SaaS table requires recurring ingestion but the team maintains a brittle custom API crawler) while preserving the required behavior (the source must arrive in governed Delta tables with manageable credential rotation). A diagnostic or monitoring step alone would not create this target-state behavior.
Incorrect Answers
A:Assess managed connector source support, permissions and refresh semantics for treatment_billing_events_delta could clarify the original fault, but the available evidence already establishes its scope; it is not the target approach now requested. The treatment_billing_events_delta workflow still requires that the source must arrive in governed Delta tables with manageable credential rotation.
B:Shared secrets and manual copies do not provide reliable governed ingestion. The proposed action (Copy API secrets into a widely shared notebook and paste full responses manually at Veterinary clinic association) does not meet the requirement that the source must arrive in governed Delta tables with manageable credential rotation, nor does it constitute the target approach.
D:Monitor connector lag, source authentication errors and schema drift for treatment_billing_events_delta would help detect future recurrence, but metrics alone do not deliver the target approach. The treatment_billing_events_delta requirement is that the source must arrive in governed Delta tables with manageable credential rotation.
Question 865
At Air quality nonprofit, the pollution measurements workflow depends on pollution_measurements_urban. Engineers report that pollution_measurements_urban CI reports successful deployment even though its Lakeflow job has never executed; the evidence is that the build log only shows resource creation, no job-run evidence. They require that release must demonstrate a working data pipeline, not just valid YAML. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which controlled implementation step is appropriate now?
- Add a post-deployment smoke test and acceptance assertions to the CI pipeline
- Accept a successful YAML validation as proof the ETL output is correct at Air quality nonprofit
- Execute a scoped smoke run after bundle deployment and capture its ID
- Verify outputs, run status and affected catalog resources after deployment
Explanation
Correct Answer
C:Execute a scoped smoke run after bundle deployment and capture its ID provides a bounded way to introduce the approved solution into pollution_measurements_urban and detect an early mismatch. The existing signal is the build log only shows resource creation, no job-run evidence; a staged change protects the service requirement (release must demonstrate a working data pipeline, not just valid YAML) and makes rollback practical.
Incorrect Answers
A:Add a post-deployment smoke test and acceptance assertions to the CI pipeline is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the pollution_measurements_urban scenario, the observed evidence is that the build log only shows resource creation, no job-run evidence; the requirement remains: release must demonstrate a working data pipeline, not just valid YAML.
B:Syntax validation cannot establish runtime data correctness. The proposed action (Accept a successful YAML validation as proof the ETL output is correct at Air quality nonprofit) does not meet the requirement that release must demonstrate a working data pipeline, not just valid YAML, nor does it constitute the controlled implementation step.
D:Verify outputs, run status and affected catalog resources after deployment is a one-time acceptance check. It does not supply the controlled implementation step for pollution_measurements_urban; a passing test cannot substitute for the specific task now required.
Question 866
At Environmental inspection office, the union versus union all workflow handles sampling field notes. The problem is that sampling_field_notes_west combines regional extracts but normal UNION removes rows that are legitimately identical across different source batches. Evidence: sample data includes repeated values where batch lineage distinguishes observations. Constraint: the target must preserve all source rows for audit. The target approach is approved. The current task is to introduce it safely with bounded scope and rollback. Which change can be introduced with minimal production risk?
- Verify repeated identical source values remain in distinct batch records
- Use UNION DISTINCT and assume it preserves every source row at Environmental inspection office
- Publish a union-all transformation with explicit schema mapping and batch tags
- Use UNION ALL when duplicates must be retained and add a batch identifier
Explanation
Correct Answer
C:Publish a union-all transformation with explicit schema mapping and batch tags provides a bounded way to introduce the approved solution into sampling_field_notes_west and detect an early mismatch. The existing signal is sample data includes repeated values where batch lineage distinguishes observations; a staged change protects the service requirement (the target must preserve all source rows for audit) and makes rollback practical.
Incorrect Answers
A:Verify repeated identical source values remain in distinct batch records is a one-time acceptance check. It does not supply the controlled implementation step for sampling_field_notes_west; a passing test cannot substitute for the specific task now required.
B:Set union removes duplicates and can lose genuine repeated observations. The proposed action (Use UNION DISTINCT and assume it preserves every source row at Environmental inspection office) does not meet the requirement that the target must preserve all source rows for audit, nor does it constitute the controlled implementation step.
D:Use UNION ALL when duplicates must be retained and add a batch identifier is a plausible lasting design once chosen, but it does not perform the controlled implementation step at this stage. In the sampling_field_notes_west scenario, the observed evidence is that sample data includes repeated values where batch lineage distinguishes observations; the requirement remains: the target must preserve all source rows for audit.
Question 867
At Retail pharmacy network, the inventory adjustments workflow depends on inventory_adjustments_island. Engineers report that inventory_adjustments_island high-frequency triggers create overlapping job runs that overwrite the same gold partition; the evidence is that run history shows two active instances of an identical pipeline. They require that concurrent processing must respect destination write semantics. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which ongoing alert or metric is most relevant?
- Set maximum concurrent runs or isolation controls appropriate to the shared target
- Monitor queued and skipped runs with destination conflicts for inventory_adjustments_island
- Raise maximum concurrent runs without coordinating shared output writes at Retail pharmacy network
- Send overlapping triggers and confirm bounded jobs and consistent output
Explanation
Correct Answer
B:Monitor queued and skipped runs with destination conflicts for inventory_adjustments_island creates recurring visibility into the failure mode (inventory_adjustments_island high-frequency triggers create overlapping job runs that overwrite the same gold partition). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while concurrent processing must respect destination write semantics.
Incorrect Answers
A:Set maximum concurrent runs or isolation controls appropriate to the shared target is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the inventory_adjustments_island scenario, the observed evidence is that run history shows two active instances of an identical pipeline; the requirement remains: concurrent processing must respect destination write semantics.
C:Overlapping unsafe writers can corrupt business results or cause failures. The proposed action (Raise maximum concurrent runs without coordinating shared output writes at Retail pharmacy network) does not meet the requirement that concurrent processing must respect destination write semantics, nor does it constitute continuing operational detection.
D:Send overlapping triggers and confirm bounded jobs and consistent output is a one-time acceptance check. It does not supply continuing operational detection for inventory_adjustments_island; a passing test cannot substitute for the specific task now required.
Question 868
Consider the loan_repayment_events_south workload at Consumer finance platform: a nightly load of loan_repayment_events_south repeats files from yesterday and developers fear duplicate inserts. Inspection found that the same object-storage folder is scanned on each scheduled run. The governed outcome must ensure that only new source files should be appended without reprocessing prior paths. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which continuing signal best reveals recurrence or drift?
- Rerun the command with unchanged files and verify row counts do not double
- Track new-file arrivals, load errors and unexpected source-file modifications for loan_repayment_events_south
- Use incremental idempotent COPY INTO against the governed Delta target
- Run unconditional INSERT SELECT over the complete folder on every retry at Consumer finance platform
Explanation
Correct Answer
B:Track new-file arrivals, load errors and unexpected source-file modifications for loan_repayment_events_south creates recurring visibility into the failure mode (a nightly load of loan_repayment_events_south repeats files from yesterday and developers fear duplicate inserts). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while only new source files should be appended without reprocessing prior paths.
Incorrect Answers
A:Rerun the command with unchanged files and verify row counts do not double is a one-time acceptance check. It does not supply continuing operational detection for loan_repayment_events_south; a passing test cannot substitute for the specific task now required.
C:Use incremental idempotent COPY INTO against the governed Delta target is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the loan_repayment_events_south scenario, the observed evidence is that the same object-storage folder is scanned on each scheduled run; the requirement remains: only new source files should be appended without reprocessing prior paths.
D:A blind append can duplicate previously processed records. The proposed action (Run unconditional INSERT SELECT over the complete folder on every retry at Consumer finance platform) does not meet the requirement that only new source files should be appended without reprocessing prior paths, nor does it constitute continuing operational detection.
Question 869
At University research consortium, the experiment records workflow depends on experiment_records_north. Engineers report that experiment_records_north dataset is shared with a third party that does not run Databricks; the evidence is that recipient cannot authenticate to a Databricks workspace but supports the open protocol. They require that the partner needs supported read-only interoperability. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which ongoing alert or metric is most relevant?
- Read shared sample data through the external client and validate scope
- Require every external organization to obtain provider workspace administrator access at University research consortium
- Choose open Delta Sharing recipient method for eligible non-Databricks consumers
- Monitor recipient token expiry and access logs for experiment_records_north
Explanation
Correct Answer
D:Monitor recipient token expiry and access logs for experiment_records_north creates recurring visibility into the failure mode (experiment_records_north dataset is shared with a third party that does not run Databricks). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while the partner needs supported read-only interoperability.
Incorrect Answers
A:Read shared sample data through the external client and validate scope is a one-time acceptance check. It does not supply continuing operational detection for experiment_records_north; a passing test cannot substitute for the specific task now required.
B:Workspace administration is not needed for open read-only sharing. The proposed action (Require every external organization to obtain provider workspace administrator access at University research consortium) does not meet the requirement that the partner needs supported read-only interoperability, nor does it constitute continuing operational detection.
C:Choose open Delta Sharing recipient method for eligible non-Databricks consumers is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the experiment_records_north scenario, the observed evidence is that recipient cannot authenticate to a Databricks workspace but supports the open protocol; the requirement remains: the partner needs supported read-only interoperability.
Question 870
Consider the treatment_billing_events_delta workload at Veterinary clinic association: treatment_billing_events_delta provider needs to offer a governed read-only subset to an external partner. Inspection found that partner does not have workspace access but needs fresh authorized data. The governed outcome must ensure that sharing must not distribute uncontrolled file copies or edit privileges. The first acceptance run passed. The current task is to detect future regressions and changes over time. Which continuing signal best reveals recurrence or drift?
- Use Delta Sharing with a configured provider share and appropriate recipient credentials
- Monitor share access, recipient lifecycle and unexpected additions for treatment_billing_events_delta
- Email raw Delta storage credentials and allow partner writes at Veterinary clinic association
- Query as the recipient and verify only intended datasets are visible
Explanation
Correct Answer
B:Monitor share access, recipient lifecycle and unexpected additions for treatment_billing_events_delta creates recurring visibility into the failure mode (treatment_billing_events_delta provider needs to offer a governed read-only subset to an external partner). Unlike a single successful test, the measure can reveal drift, lag, or new failures after later source and deployment changes while sharing must not distribute uncontrolled file copies or edit privileges.
Incorrect Answers
A:Use Delta Sharing with a configured provider share and appropriate recipient credentials is a plausible lasting design once chosen, but it does not perform continuing operational detection at this stage. In the treatment_billing_events_delta scenario, the observed evidence is that partner does not have workspace access but needs fresh authorized data; the requirement remains: sharing must not distribute uncontrolled file copies or edit privileges.
C:Sharing object-storage secrets bypasses governed read-only access. The proposed action (Email raw Delta storage credentials and allow partner writes at Veterinary clinic association) does not meet the requirement that sharing must not distribute uncontrolled file copies or edit privileges, nor does it constitute continuing operational detection.
D:Query as the recipient and verify only intended datasets are visible is a one-time acceptance check. It does not supply continuing operational detection for treatment_billing_events_delta; a passing test cannot substitute for the specific task now required.