Databricks Certified Data Engineer Professional Practice Test 67
Question 660
Operations at Summit Consumer Goods owns a workload where a nightly compliance job is owned by an individual with broad administrator permissions. The previously identified issue was: The individual's credentials create unnecessary privilege and lifecycle risk. Which metric or audit check best detects drift?
- Monitor attempts to change or bypass column masks
- Audit row filter changes and unexpectedly broad query results
- Monitor view definition changes and access to raw source
- Audit privilege escalation attempts and run-as changes
Explanation
Correct Answer
D:Audit privilege escalation attempts and run-as changes. This watches the failure mode described by the individual's credentials create unnecessary privilege and lifecycle risk. Production automation should use least-privilege service principals for stable identity. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor attempts to change or bypass column masks. This monitor action addresses a different case (column masking), where a raw SELECT on an approved table exposes plaintext email. Unity Catalog column masks can transform sensitive values at read time. In the stated problem, the individual's credentials create unnecessary privilege and lifecycle risk; this choice would not establish the required service principal access decision.
B:Audit row filter changes and unexpectedly broad query results. This monitor action addresses a different case (unity catalog row filter), where a shared table currently returns all regions to every reader. Unity Catalog row filters can restrict row visibility through governed policies. In the stated problem, the individual's credentials create unnecessary privilege and lifecycle risk; this choice would not establish the required service principal access decision.
C:Monitor view definition changes and access to raw source. This monitor action addresses a different case (sensitive data suppression), where a broad SELECT still includes unstructured notes. Column suppression and data minimization prevent unnecessary sensitive-field release. In the stated problem, the individual's credentials create unnecessary privilege and lifecycle risk; this choice would not establish the required service principal access decision.
Question 661
A deployment owner at Horizon Travel needs to implement a remedy: supplier CSV feeds contain commas inside quoted customer names. The present failure is: An ingestion reader splits quoted names into extra columns. Which change should be applied?
- Configure CSV quote/escape and malformed-record capture
- Configure the JSON reader and evolution options supported by the ingestion mechanism
- Configure cloudFiles ingestion into a Delta bronze streaming table
- Configure the supported XML reader with the correct rowTag and schema
Explanation
Correct Answer
A:Configure CSV quote/escape and malformed-record capture. This directly changes the feature or logic behind an ingestion reader splits quoted names into extra columns. CSV quoting and escape configurations must match the source serialization. It is the concrete configuration for the stated incident.
Incorrect Answers
B:Configure the JSON reader and evolution options supported by the ingestion mechanism. This implement action addresses a different case (nested json ingest), where the initial reader assumes one unchanging flat schema and loses new fields. Explicit schema management and rescued data handling protect evolving semi-structured ingestion. In the stated problem, an ingestion reader splits quoted names into extra columns; this choice would not establish the required malformed csv quotes decision.
C:Configure cloudFiles ingestion into a Delta bronze streaming table. This implement action addresses a different case (auto loader file discovery), where listing the full historical directory every run exceeds the ingestion SLA. Auto Loader is designed for incremental new-file discovery with persisted state. In the stated problem, an ingestion reader splits quoted names into extra columns; this choice would not establish the required malformed csv quotes decision.
D:Configure the supported XML reader with the correct rowTag and schema. This implement action addresses a different case (xml repeating elements), where an XML ingest incorrectly flattens multiple line items into one text field. XML ingestion must specify the repeating row element and preserve nested structure. In the stated problem, an ingestion reader splits quoted names into extra columns; this choice would not establish the required malformed csv quotes decision.
Question 662
At Summit Consumer Goods, a nightly compliance job is owned by an individual with broad administrator permissions. Architecture constraint: The individual's credentials create unnecessary privilege and lifecycle risk. Which design best addresses this requirement?
- Plan secret rotation with a new value and safe rollback
- Assign a dedicated production service principal with only required grants
- Assign distinct deployment operators and data stewards with scoped privileges
- Use group-based least-privilege workspace object permissions
Explanation
Correct Answer
B:Assign a dedicated production service principal with only required grants. This design addresses the individual's credentials create unnecessary privilege and lifecycle risk. Production automation should use least-privilege service principals for stable identity. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Plan secret rotation with a new value and safe rollback. This design action addresses a different case (credential rotation procedure), where the hard-coded service password has not rotated for a year. Secret management and controlled rotation reduce long-lived credential exposure. In the stated problem, the individual's credentials create unnecessary privilege and lifecycle risk; this choice would not establish the required service principal access decision.
C:Assign distinct deployment operators and data stewards with scoped privileges. This design action addresses a different case (separation of duties), where all developers currently own production catalogs and can grant access. Separate deploy and governance roles reduce accidental privilege escalation. In the stated problem, the individual's credentials create unnecessary privilege and lifecycle risk; this choice would not establish the required service principal access decision.
D:Use group-based least-privilege workspace object permissions. This design action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, the individual's credentials create unnecessary privilege and lifecycle risk; this choice would not establish the required service principal access decision.
Question 663
Quality assurance at Pinecrest Holdings checks a corrected workload where a foreachBatch sink writes aggregates to an external database. The original fault was: A retry of a completed batch produces duplicate sink rows. Which verification is most decisive?
- Confirm target exchange or join behavior changed and query results remain equivalent
- Confirm each key has one selected row matching the maximum sequence
- Replay a batch ID and confirm target rows are not duplicated
- Replay update and delete events and confirm one accurate current row per key
Explanation
Correct Answer
C:Replay a batch ID and confirm target rows are not duplicated. This tests the expected result against a retry of a completed batch produces duplicate sink rows, not simply whether a job or object exists. foreachBatch can execute a batch again and requires idempotent external writes for exactly-once effects.
Incorrect Answers
A:Confirm target exchange or join behavior changed and query results remain equivalent. This validate action addresses a different case (spark explain plans), where developers tune executor size without inspecting actual exchanges. The physical execution plan exposes exchanges, scan filters and join strategy. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.
B:Confirm each key has one selected row matching the maximum sequence. This validate action addresses a different case (window latest record), where the pipeline picks an arbitrary record instead of the newest sequence. ROW_NUMBER ordered by a deterministic sequence within each key selects the desired latest row. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.
D:Replay update and delete events and confirm one accurate current row per key. This validate action addresses a different case (auto cdc keys), where applying events as append-only rows leaves outdated active accounts. AUTO CDC or applicable APPLY CHANGES needs stable keys and sequence ordering for stateful upserts. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.
Question 664
A data engineer at Northfield Analytics must troubleshoot this case: a schema receives new governed tables every week. Evidence: Analysts must repeatedly request identical SELECT grants for each table. What should be inspected first to confirm the failure mechanism?
- Inspect catalog and schema managed-location settings
- Inspect current owner and authorized ownership transfer process
- Inspect current tags and security classification coverage
- Inspect the schema permission model and current group grants
Explanation
Correct Answer
D:Inspect the schema permission model and current group grants. This checks the specific evidence behind analysts must repeatedly request identical SELECT grants for each table. Unity Catalog schema-level privileges can apply to existing and future child objects. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect catalog and schema managed-location settings. This diagnose action addresses a different case (managed storage ownership), where the data platform cannot apply consistent lifecycle governance. Managed locations centralize storage ownership for Unity Catalog managed assets. In the stated problem, analysts must repeatedly request identical SELECT grants for each table; this choice would not establish the required schema-level grant inheritance decision.
B:Inspect current owner and authorized ownership transfer process. This diagnose action addresses a different case (ownership reassignment), where new data stewards cannot maintain its grants. Unity Catalog ownership should be assigned to a maintained principal. In the stated problem, analysts must repeatedly request identical SELECT grants for each table; this choice would not establish the required schema-level grant inheritance decision.
C:Inspect current tags and security classification coverage. This diagnose action addresses a different case (governed tags), where naming conventions alone are inconsistent across teams. Governed tags can classify assets for discovery and policy workflows. In the stated problem, analysts must repeatedly request identical SELECT grants for each table; this choice would not establish the required schema-level grant inheritance decision.
Question 665
A data engineer at Bayview Insurance must troubleshoot this case: a production workflow runs as the notebook author's personal identity. Evidence: An employee departure causes permissions failures during scheduled runs. What should be inspected first to confirm the failure mechanism?
- Inspect job run-as user and dependent Unity Catalog grants
- Inspect input rate, processed rate and microbatch duration during peaks
- Inspect dependency graph and upstream task statuses
- Inspect Python UDF execution time and batch serialization in Spark UI
Explanation
Correct Answer
A:Inspect job run-as user and dependent Unity Catalog grants. This checks the specific evidence behind an employee departure causes permissions failures during scheduled runs. A stable least-privilege service principal can provide durable job identity. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
B:Inspect input rate, processed rate and microbatch duration during peaks. This diagnose action addresses a different case (streaming trigger tuning), where microbatch processing time exceeds the configured trigger interval. Shortening a trigger cannot make a saturated single streaming query process overlapping batches faster. In the stated problem, an employee departure causes permissions failures during scheduled runs; this choice would not establish the required job run-as identity decision.
C:Inspect dependency graph and upstream task statuses. This diagnose action addresses a different case (job task dag), where dependent tasks sometimes run before the bronze load finishes. Lakeflow Job task dependencies form a directed acyclic execution graph. In the stated problem, an employee departure causes permissions failures during scheduled runs; this choice would not establish the required job run-as identity decision.
D:Inspect Python UDF execution time and batch serialization in Spark UI. This diagnose action addresses a different case (vectorized udf), where row-by-row Python invocation dominates stage runtime. Pandas UDFs process batches through Arrow rather than per-row Python calls. In the stated problem, an employee departure causes permissions failures during scheduled runs; this choice would not establish the required job run-as identity decision.
Question 666
At Cobalt Biotech, a diagnostic notebook prints a database password after secret lookup. Observed issue: Databricks notebook redaction is incorrectly treated as a comprehensive security control. Before applying a fix, which investigation is most directly relevant?
- Inspect identifier entropy and whether the transform leaks raw values
- Inspect log statements and secret retrieval paths without revealing values
- Inspect secret age, job dependency and allowed rotation window
- Inspect notebook and folder ACLs for broad edit permissions
Explanation
Correct Answer
B:Inspect log statements and secret retrieval paths without revealing values. This checks the specific evidence behind databricks notebook redaction is incorrectly treated as a comprehensive security control. Secrets should never be deliberately printed or put in logs, regardless of redaction behavior. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect identifier entropy and whether the transform leaks raw values. This diagnose action addresses a different case (pseudonymized join keys), where a naive hash without controlled key management risks predictable identifiers. Keyed pseudonymization can preserve joins while reducing identifier exposure. In the stated problem, databricks notebook redaction is incorrectly treated as a comprehensive security control; this choice would not establish the required secret retrieval logs decision.
C:Inspect secret age, job dependency and allowed rotation window. This diagnose action addresses a different case (credential rotation procedure), where the hard-coded service password has not rotated for a year. Secret management and controlled rotation reduce long-lived credential exposure. In the stated problem, databricks notebook redaction is incorrectly treated as a comprehensive security control; this choice would not establish the required secret retrieval logs decision.
D:Inspect notebook and folder ACLs for broad edit permissions. This diagnose action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, databricks notebook redaction is incorrectly treated as a comprehensive security control; this choice would not establish the required secret retrieval logs decision.
Question 667
At Westhaven Services, a CDC source occasionally emits two records for one key at the same sequence number. Architecture constraint: The processing order for tied source events is nondeterministic. Which design best addresses this requirement?
- Define a deterministic window partitioned by key and ordered by event sequence
- Scale the bottleneck or reduce work before changing trigger cadence
- Define a composite or otherwise unique ordering key for competing CDC events
- Define validated region and catalog parameters for each invocation
Explanation
Correct Answer
C:Define a composite or otherwise unique ordering key for competing CDC events. This design addresses the processing order for tied source events is nondeterministic. A deterministic sequencing contract needs a unique ordered event position. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Define a deterministic window partitioned by key and ordered by event sequence. This design action addresses a different case (window latest record), where the pipeline picks an arbitrary record instead of the newest sequence. ROW_NUMBER ordered by a deterministic sequence within each key selects the desired latest row. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
B:Scale the bottleneck or reduce work before changing trigger cadence. This design action addresses a different case (streaming trigger tuning), where microbatch processing time exceeds the configured trigger interval. Shortening a trigger cannot make a saturated single streaming query process overlapping batches faster. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
D:Define validated region and catalog parameters for each invocation. This design action addresses a different case (job parameters), where code contains hard-coded region IDs and catalog names. Task parameters allow runtime-specific values without changing notebook source. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
Question 668
A data engineer at Meadowland Software must troubleshoot this case: a critical table remains owned by a departed contractor. Evidence: New data stewards cannot maintain its grants. What should be inspected first to confirm the failure mechanism?
- Inspect effective parent usage and table SELECT grants
- Inspect current tags and security classification coverage
- Inspect asset comments and ownership for critical reporting tables
- Inspect current owner and authorized ownership transfer process
Explanation
Correct Answer
D:Inspect current owner and authorized ownership transfer process. This checks the specific evidence behind new data stewards cannot maintain its grants. Unity Catalog ownership should be assigned to a maintained principal. The investigation distinguishes the mechanism from an unrelated operational symptom.
Incorrect Answers
A:Inspect effective parent usage and table SELECT grants. This diagnose action addresses a different case (catalog privilege inheritance), where the team lacks USE CATALOG and USE SCHEMA on the parent containers. Unity Catalog object access generally requires relevant parent usage privileges. In the stated problem, new data stewards cannot maintain its grants; this choice would not establish the required ownership reassignment decision.
B:Inspect current tags and security classification coverage. This diagnose action addresses a different case (governed tags), where naming conventions alone are inconsistent across teams. Governed tags can classify assets for discovery and policy workflows. In the stated problem, new data stewards cannot maintain its grants; this choice would not establish the required ownership reassignment decision.
C:Inspect asset comments and ownership for critical reporting tables. This diagnose action addresses a different case (catalog descriptions), where table and column metadata lack meaningful descriptions. Unity Catalog comments improve discoverability of governed data assets. In the stated problem, new data stewards cannot maintain its grants; this choice would not establish the required ownership reassignment decision.
Question 669
At Westhaven Services, a CDC source occasionally emits two records for one key at the same sequence number. A change was deployed to address: The processing order for tied source events is nondeterministic. Which test result provides the strongest evidence that the requirement is now met?
- Reverse input arrival order and verify identical final target state
- Replay representative late events and verify expected acceptance or exclusion
- Load-test peak input and verify processed rate stays above incoming rate
- Verify production tasks receive the production catalog and input paths
Explanation
Correct Answer
A:Reverse input arrival order and verify identical final target state. This tests the expected result against the processing order for tied source events is nondeterministic, not simply whether a job or object exists. A deterministic sequencing contract needs a unique ordered event position.
Incorrect Answers
B:Replay representative late events and verify expected acceptance or exclusion. This validate action addresses a different case (late event semantics), where reports silently discard legitimate late events under an aggressive watermark. A watermark trades supported lateness for state size and timeliness. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
C:Load-test peak input and verify processed rate stays above incoming rate. This validate action addresses a different case (streaming trigger tuning), where microbatch processing time exceeds the configured trigger interval. Shortening a trigger cannot make a saturated single streaming query process overlapping batches faster. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
D:Verify production tasks receive the production catalog and input paths. This validate action addresses a different case (bundle target overrides), where development job arguments were deployed into the production target. Bundle targets can override environment-specific variables and resource settings. In the stated problem, the processing order for tied source events is nondeterministic; this choice would not establish the required cdc duplicate sequence decision.
Question 670
At Orion Capital, a group creates many tables in random external paths. Architecture constraint: The data platform cannot apply consistent lifecycle governance. Which design best addresses this requirement?
- Use a durable steward group as owner rather than a personal principal
- Choose approved managed storage locations for governed tables
- Adopt centrally governed classification tags for sensitive assets
- Use a narrowly scoped external location with a controlled storage credential
Explanation
Correct Answer
B:Choose approved managed storage locations for governed tables. This design addresses the data platform cannot apply consistent lifecycle governance. Managed locations centralize storage ownership for Unity Catalog managed assets. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Use a durable steward group as owner rather than a personal principal. This design action addresses a different case (ownership reassignment), where new data stewards cannot maintain its grants. Unity Catalog ownership should be assigned to a maintained principal. In the stated problem, the data platform cannot apply consistent lifecycle governance; this choice would not establish the required managed storage ownership decision.
C:Adopt centrally governed classification tags for sensitive assets. This design action addresses a different case (governed tags), where naming conventions alone are inconsistent across teams. Governed tags can classify assets for discovery and policy workflows. In the stated problem, the data platform cannot apply consistent lifecycle governance; this choice would not establish the required managed storage ownership decision.
D:Use a narrowly scoped external location with a controlled storage credential. This design action addresses a different case (external location governance), where raw cloud permissions bypass central catalog governance. External locations map cloud paths to governed Unity Catalog storage credentials. In the stated problem, the data platform cannot apply consistent lifecycle governance; this choice would not establish the required managed storage ownership decision.