Databricks Certified Data Engineer Professional Practice Test 25
Question 240
An architecture review at Summit Consumer Goods covers a workload where an orchestrator retries a failed object-storage batch. The unresolved constraint is: A partially processed input file is loaded twice into Delta. Which design is most appropriate?
- Use Auto Loader with managed file discovery and checkpoint/schema locations
- Use a stable checkpoint with a compatible idempotent Delta sink
- Document delimiters, quote rules and corrupt-record handling for the feed
- Use one durable checkpoint and clear idempotency keys for replay
Explanation
Correct Answer
D:Use one durable checkpoint and clear idempotency keys for replay. This design addresses a partially processed input file is loaded twice into Delta. Checkpointed incremental ingestion or stable event-level deduplication prevents repeated effects. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Use Auto Loader with managed file discovery and checkpoint/schema locations. This design action addresses a different case (auto loader file discovery), where listing the full historical directory every run exceeds the ingestion SLA. Auto Loader is designed for incremental new-file discovery with persisted state. In the stated problem, a partially processed input file is loaded twice into Delta; this choice would not establish the required retry-safe file loads decision.
B:Use a stable checkpoint with a compatible idempotent Delta sink. This design action addresses a different case (kafka offset durability), where restarting with a fresh checkpoint replays previously ingested messages. A stable streaming checkpoint maintains Kafka offsets and sink progress across restarts. In the stated problem, a partially processed input file is loaded twice into Delta; this choice would not establish the required retry-safe file loads decision.
C:Document delimiters, quote rules and corrupt-record handling for the feed. This design action addresses a different case (malformed csv quotes), where an ingestion reader splits quoted names into extra columns. CSV quoting and escape configurations must match the source serialization. In the stated problem, a partially processed input file is loaded twice into Delta; this choice would not establish the required retry-safe file loads decision.
Question 241
At Clearstone Media, data consumers cannot discover existing data products without requesting table SELECT. A change was deployed to address: Teams grant SELECT merely to expose object listings. Which test result provides the strongest evidence that the requirement is now met?
- Confirm users can discover metadata but cannot read restricted table rows
- Establish a test table and confirm approved analysts can query it
- Confirm managed table storage resolves to the approved location
- Confirm stewards can manage the table while readers retain intended access
Explanation
Correct Answer
A:Confirm users can discover metadata but cannot read restricted table rows. This tests the expected result against teams grant SELECT merely to expose object listings, not simply whether a job or object exists. BROWSE enables controlled metadata discovery without granting data read access.
Incorrect Answers
B:Create a test table and confirm approved analysts can query it. This validate action addresses a different case (schema-level grant inheritance), where analysts must repeatedly request identical SELECT grants for each table. Unity Catalog schema-level privileges can apply to existing and future child objects. In the stated problem, teams grant SELECT merely to expose object listings; this choice would not establish the required browse metadata privilege decision.
C:Confirm managed table storage resolves to the approved location. This validate action addresses a different case (managed storage ownership), where the data platform cannot apply consistent lifecycle governance. Managed locations centralize storage ownership for Unity Catalog managed assets. In the stated problem, teams grant SELECT merely to expose object listings; this choice would not establish the required browse metadata privilege decision.
D:Confirm stewards can manage the table while readers retain intended access. This validate action addresses a different case (ownership reassignment), where new data stewards cannot maintain its grants. Unity Catalog ownership should be assigned to a maintained principal. In the stated problem, teams grant SELECT merely to expose object listings; this choice would not establish the required browse metadata privilege decision.
Question 242
Operations at Summit Consumer Goods owns a workload where a large Delta table receives frequent narrow DELETE operations. The previously identified issue was: Each small delete rewrites large data files and increases write cost. Which metric or audit check best detects drift?
- Monitor cache hit rate and eviction pressure
- Monitor deletion-vector accumulation and subsequent compaction
- Monitor small-file accumulation and optimization jobs
- Monitor optimization operations and cost savings
Explanation
Correct Answer
B:Monitor deletion-vector accumulation and subsequent compaction. This watches the failure mode described by each small delete rewrites large data files and increases write cost. Deletion vectors can mark deleted rows without immediate whole-file rewrite when supported. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor cache hit rate and eviction pressure. This monitor action addresses a different case (cache reuse), where repeated recomputation dominates each branch's execution. Persistence can avoid repeated work if reused data fits the memory budget. In the stated problem, each small delete rewrites large data files and increases write cost; this choice would not establish the required deletion vectors decision.
C:Monitor small-file accumulation and optimization jobs. This monitor action addresses a different case (small file compaction), where read overhead grows because query plans open excessive small files. OPTIMIZE can compact compatible Delta files without changing logical table contents. In the stated problem, each small delete rewrites large data files and increases write cost; this choice would not establish the required deletion vectors decision.
D:Monitor optimization operations and cost savings. This monitor action addresses a different case (predictive optimization), where operators run blanket manual OPTIMIZE jobs regardless of table use. Predictive optimization can automate supported managed-table maintenance based on workload signals. In the stated problem, each small delete rewrites large data files and increases write cost; this choice would not establish the required deletion vectors decision.
Question 243
At Clearstone Media, a message bus sends Avro-encoded order events with added optional fields. A change is required because: An older consumer cannot deserialize records from a newer producer schema. Which implementation action most directly corrects the configuration?
- Configure the JSON reader and evolution options supported by the ingestion mechanism
- Cast or quarantine incompatible records before writing the destination
- Update the reader schema handling and test a versioned schema rollout
- Configure the supported XML reader with the correct rowTag and schema
Explanation
Correct Answer
C:Update the reader schema handling and test a versioned schema rollout. This directly changes the feature or logic behind an older consumer cannot deserialize records from a newer producer schema. Avro writer-reader schema compatibility must be maintained for evolution. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Configure the JSON reader and evolution options supported by the ingestion mechanism. This implement action addresses a different case (nested json ingest), where the initial reader assumes one unchanging flat schema and loses new fields. Explicit schema management and rescued data handling protect evolving semi-structured ingestion. In the stated problem, an older consumer cannot deserialize records from a newer producer schema; this choice would not establish the required avro schema evolution decision.
B:Cast or quarantine incompatible records before writing the destination. This implement action addresses a different case (parquet schema mismatch), where a merge step fails after a producer changes an integer field to string. Parquet schema compatibility matters even when column names are unchanged. In the stated problem, an older consumer cannot deserialize records from a newer producer schema; this choice would not establish the required avro schema evolution decision.
D:Configure the supported XML reader with the correct rowTag and schema. This implement action addresses a different case (xml repeating elements), where an XML ingest incorrectly flattens multiple line items into one text field. XML ingestion must specify the repeating row element and preserve nested structure. In the stated problem, an older consumer cannot deserialize records from a newer producer schema; this choice would not establish the required avro schema evolution decision.
Question 244
Operations at Harborline Utilities owns a workload where a daily pipeline must load bronze before parallel silver transformations. The previously identified issue was: Dependent tasks sometimes run before the bronze load finishes. Which metric or audit check best detects drift?
- Monitor task retries and repeated sink key counts
- Track dependency upgrades and compatibility test failures
- Monitor entrypoint failures after runtime or package upgrades
- Monitor repeated dependency failures and overall critical-path duration
Explanation
Correct Answer
D:Monitor repeated dependency failures and overall critical-path duration. This watches the failure mode described by dependent tasks sometimes run before the bronze load finishes. Lakeflow Job task dependencies form a directed acyclic execution graph. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor task retries and repeated sink key counts. This monitor action addresses a different case (job retry semantics), where the downstream write repeats non-idempotent side effects on retry. Retries re-execute task logic; application code must protect repeated effects. In the stated problem, dependent tasks sometimes run before the bronze load finishes; this choice would not establish the required job task dag decision.
B:Track dependency upgrades and compatibility test failures. This monitor action addresses a different case (library version pinning), where a transitive dependency updated to an incompatible version. Explicit compatible dependency versions improve reproducibility across new compute environments. In the stated problem, dependent tasks sometimes run before the bronze load finishes; this choice would not establish the required job task dag decision.
C:Monitor entrypoint failures after runtime or package upgrades. This monitor action addresses a different case (python task entrypoints), where the task exits before setting up its argument parser. A standard Python entrypoint and explicit arguments avoid notebook runtime assumptions. In the stated problem, dependent tasks sometimes run before the bronze load finishes; this choice would not establish the required job task dag decision.
Question 245
At Horizon Travel, an analyst may see only their assigned regional customers. The known risk is: A shared table currently returns all regions to every reader. After remediation, which ongoing signal best detects recurrence?
- Audit row filter changes and unexpectedly broad query results
- Monitor view definition changes and access to raw source
- Audit unauthorized changes to workspace objects
- Audit privilege escalation attempts and run-as changes
Explanation
Correct Answer
A:Audit row filter changes and unexpectedly broad query results. This watches the failure mode described by a shared table currently returns all regions to every reader. Unity Catalog row filters can restrict row visibility through governed policies. It supplies a regression signal after remediation.
Incorrect Answers
B:Monitor view definition changes and access to raw source. This monitor action addresses a different case (sensitive data suppression), where a broad SELECT still includes unstructured notes. Column suppression and data minimization prevent unnecessary sensitive-field release. In the stated problem, a shared table currently returns all regions to every reader; this choice would not establish the required unity catalog row filter decision.
C:Audit unauthorized changes to workspace objects. This monitor action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, a shared table currently returns all regions to every reader; this choice would not establish the required unity catalog row filter decision.
D:Audit privilege escalation attempts and run-as changes. This monitor action addresses a different case (service principal access), where the individual's credentials create unnecessary privilege and lifecycle risk. Production automation should use least-privilege service principals for stable identity. In the stated problem, a shared table currently returns all regions to every reader; this choice would not establish the required unity catalog row filter decision.
Question 246
Operations at Riverbend Energy owns a workload where a logistics partner sends XML with repeated line-item elements. The previously identified issue was: An XML ingest incorrectly flattens multiple line items into one text field. Which metric or audit check best detects drift?
- Monitor schema changes and rescued-field counts
- Monitor missing child-element and parser error counts
- Monitor unexpected file sizes and unsupported media types
- Monitor duplicated ingest event IDs and retry pressure
Explanation
Correct Answer
B:Monitor missing child-element and parser error counts. This watches the failure mode described by an XML ingest incorrectly flattens multiple line items into one text field. XML ingestion must specify the repeating row element and preserve nested structure. It supplies a regression signal after remediation.
Incorrect Answers
A:Monitor schema changes and rescued-field counts. This monitor action addresses a different case (nested json ingest), where the initial reader assumes one unchanging flat schema and loses new fields. Explicit schema management and rescued data handling protect evolving semi-structured ingestion. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.
C:Monitor unexpected file sizes and unsupported media types. This monitor action addresses a different case (binary payload metadata), where a line-oriented text reader corrupts the raw binary payload. binaryFile input preserves raw file bytes and path metadata rather than decoding lines. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.
D:Monitor duplicated ingest event IDs and retry pressure. This monitor action addresses a different case (retry-safe file loads), where a partially processed input file is loaded twice into Delta. Checkpointed incremental ingestion or stable event-level deduplication prevents repeated effects. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.
Question 247
At Meadowland Software, an ETL team writes a large analytic table with high-cardinality strings. A change was deployed to address: Row-oriented text storage increases scans and transfer volumes. Which test result provides the strongest evidence that the requirement is now met?
- Run small and large test inputs and compare execution strategies
- Delete a few rows and confirm snapshot consistency without unnecessary rewrites
- Reconcile rows and compare scan cost against text storage
- Reconcile changed rows and compare scanned file counts
Explanation
Correct Answer
C:Reconcile rows and compare scan cost against text storage. This tests the expected result against row-oriented text storage increases scans and transfer volumes, not simply whether a job or object exists. Columnar formats and appropriate encoding improve analytical read efficiency.
Incorrect Answers
A:Run small and large test inputs and compare execution strategies. This validate action addresses a different case (adaptive execution), where static join strategy performs poorly at varying input volumes. Adaptive Query Execution can adjust supported shuffle partitions and join strategies at runtime. In the stated problem, row-oriented text storage increases scans and transfer volumes; this choice would not establish the required columnar compression decision.
B:Delete a few rows and confirm snapshot consistency without unnecessary rewrites. This validate action addresses a different case (deletion vectors), where each small delete rewrites large data files and increases write cost. Deletion vectors can mark deleted rows without immediate whole-file rewrite when supported. In the stated problem, row-oriented text storage increases scans and transfer volumes; this choice would not establish the required columnar compression decision.
D:Reconcile changed rows and compare scanned file counts. This validate action addresses a different case (predicate-limited merge), where the merge scans unrelated historical partitions. Constraining merge matches to relevant partitions can reduce unnecessary target scanning. In the stated problem, row-oriented text storage increases scans and transfer volumes; this choice would not establish the required columnar compression decision.
Question 248
At Lakewood Distribution, a new workspace team needs read-only access to a vendor landing bucket. A change is required because: Engineers propose a credential allowing access to all company buckets. Which implementation action most directly corrects the configuration?
- Remove overly broad edit access and grant view permission to readers
- Revoke unnecessary production ownership and grant task-specific roles
- Create the aggregation view and grant SELECT only on the view
- Grant only required read privileges on the external location
Explanation
Correct Answer
D:Grant only required read privileges on the external location. This directly changes the feature or logic behind engineers propose a credential allowing access to all company buckets. Unity Catalog storage credentials and external locations can constrain storage access. It is the concrete configuration for the stated incident.
Incorrect Answers
A:Remove overly broad edit access and grant view permission to readers. This implement action addresses a different case (workspace object acl), where the team needs read-only access for most users. Workspace object permissions can restrict who can edit or manage shared code. In the stated problem, engineers propose a credential allowing access to all company buckets; this choice would not establish the required storage credential scope decision.
B:Revoke unnecessary production ownership and grant task-specific roles. This implement action addresses a different case (separation of duties), where all developers currently own production catalogs and can grant access. Separate deploy and governance roles reduce accidental privilege escalation. In the stated problem, engineers propose a credential allowing access to all company buckets; this choice would not establish the required storage credential scope decision.
C:Create the aggregation view and grant SELECT only on the view. This implement action addresses a different case (secure view sharing), where granting SELECT on the raw fact table violates minimization. A governed aggregation view can expose only approved derived information. In the stated problem, engineers propose a credential allowing access to all company buckets; this choice would not establish the required storage credential scope decision.
Question 249
At Granite Edge Labs, a managed Delta table serves workloads whose filter columns change each quarter. Architecture constraint: Manual clustering keys quickly become stale. Which design best addresses this requirement?
- Use CLUSTER BY AUTO for supported Unity Catalog managed Delta tables
- Use supported AQE with appropriate shuffle and join settings
- Choose supported liquid clustering on commonly filtered device fields
- Choose a supported columnar Delta or Parquet layout for analytics
Explanation
Correct Answer
A:Use CLUSTER BY AUTO for supported Unity Catalog managed Delta tables. This design addresses manual clustering keys quickly become stale. Automatic liquid clustering can select keys using query workload when platform prerequisites are met. It establishes the right contract or architecture before execution changes.
Incorrect Answers
B:Use supported AQE with appropriate shuffle and join settings. This design action addresses a different case (adaptive execution), where static join strategy performs poorly at varying input volumes. Adaptive Query Execution can adjust supported shuffle partitions and join strategies at runtime. In the stated problem, manual clustering keys quickly become stale; this choice would not establish the required automatic clustering decision.
C:Choose supported liquid clustering on commonly filtered device fields. This design action addresses a different case (liquid clustering cardinality), where partitioning by device would produce many tiny partitions. Liquid clustering organizes data by useful filter keys without high-cardinality partition directories. In the stated problem, manual clustering keys quickly become stale; this choice would not establish the required automatic clustering decision.
D:Choose a supported columnar Delta or Parquet layout for analytics. This design action addresses a different case (columnar compression), where row-oriented text storage increases scans and transfer volumes. Columnar formats and appropriate encoding improve analytical read efficiency. In the stated problem, manual clustering keys quickly become stale; this choice would not establish the required automatic clustering decision.
Question 250
An architecture review at Bayview Insurance covers a workload where a team has SELECT on a table but receives a permission-denied error. The unresolved constraint is: The team lacks USE CATALOG and USE SCHEMA on the parent containers. Which design is most appropriate?
- Permit metadata discovery while requiring separate approvals for SELECT
- Grant parent usage plus specific SELECT rather than broad ownership
- Use an approved schema-level SELECT grant where broad table access is intended
- Use a durable steward group as owner rather than a personal principal
Explanation
Correct Answer
B:Grant parent usage plus specific SELECT rather than broad ownership. This design addresses the team lacks USE CATALOG and USE SCHEMA on the parent containers. Unity Catalog object access generally requires relevant parent usage privileges. It establishes the right contract or architecture before execution changes.
Incorrect Answers
A:Permit metadata discovery while requiring separate approvals for SELECT. This design action addresses a different case (browse metadata privilege), where teams grant SELECT merely to expose object listings. BROWSE enables controlled metadata discovery without granting data read access. In the stated problem, the team lacks USE CATALOG and USE SCHEMA on the parent containers; this choice would not establish the required catalog privilege inheritance decision.
C:Use an approved schema-level SELECT grant where broad table access is intended. This design action addresses a different case (schema-level grant inheritance), where analysts must repeatedly request identical SELECT grants for each table. Unity Catalog schema-level privileges can apply to existing and future child objects. In the stated problem, the team lacks USE CATALOG and USE SCHEMA on the parent containers; this choice would not establish the required catalog privilege inheritance decision.
D:Use a durable steward group as owner rather than a personal principal. This design action addresses a different case (ownership reassignment), where new data stewards cannot maintain its grants. Unity Catalog ownership should be assigned to a maintained principal. In the stated problem, the team lacks USE CATALOG and USE SCHEMA on the parent containers; this choice would not establish the required catalog privilege inheritance decision.