Pass Databricks Certified Data Engineer Professional Exam in First Attempt Guaranteed!

Use supported AQE with appropriate shuffle and join settings. This design action addresses a different case (adaptive execution), where static join strategy performs poorly at varying input volumes. Adaptive Query Execution can adjust supported shuffle partitions and join strategies at runtime. In the stated problem, executor memory pressure grows even though input rate is stable; this choice would not establish the required streaming state growth decision.

Schedule appropriate compaction or predictive optimization for the table. This design action addresses a different case (small file compaction), where read overhead grows because query plans open excessive small files. OPTIMIZE can compact compatible Delta files without changing logical table contents. In the stated problem, executor memory pressure grows even though input rate is stable; this choice would not establish the required streaming state growth decision.

Choose a partition count based on volume and executor capacity. This design action addresses a different case (shuffle partition count), where the current partition count is too low for data volume. Shuffle partition sizing affects memory pressure and task parallelism. In the stated problem, executor memory pressure grows even though input rate is stable; this choice would not establish the required streaming state growth decision.

Inspect row-level expectation metrics alongside pipeline event types. This diagnose action addresses a different case (quality metric interpretation), where analysts misread a quality count as the number of pipeline task failures. Expectation metrics describe data rows and must be interpreted separately from orchestration status. In the stated problem, an order-level aggregation mistakenly treats the array as a single item; this choice would not establish the required array explosion decision.

Inspect the streaming table definition and expectation violation counts. This diagnose action addresses a different case (expectation drop action), where invalid rows currently reach downstream aggregates. A Lakeflow expectation configured to drop invalid rows filters violations from target output. In the stated problem, an order-level aggregation mistakenly treats the array as a single item; this choice would not establish the required array explosion decision.

Inspect per-key frequencies and Spark task duration distribution. This diagnose action addresses a different case (skewed group aggregation), where a single campaign dominates rows and one reduce task runs for minutes. A hot aggregation key causes skew despite ample parallelism. In the stated problem, an order-level aggregation mistakenly treats the array as a single item; this choice would not establish the required array explosion decision.

Monitor job failure rate and latency over time. This monitor action addresses a different case (job operational history), where operators only see the most recent run in the Jobs UI. Jobs system records support historical analysis of runs and tasks. In the stated problem, most provisioned resources sit idle between job runs; this choice would not establish the required compute utilization decision.

Monitor API failures, lag and pagination completeness. This monitor action addresses a different case (jobs rest monitoring), where a manual UI report omits failed tasks and paginated runs. The Jobs API exposes run records that a program can paginate and aggregate. In the stated problem, most provisioned resources sit idle between job runs; this choice would not establish the required compute utilization decision.

Alert on cost spikes and untagged spend. This monitor action addresses a different case (billing usage monitoring), where cost reports omit some job and workspace labels. Databricks system billing tables can expose usage records for cost attribution. In the stated problem, most provisioned resources sit idle between job runs; this choice would not establish the required compute utilization decision.

Inspect Python UDF execution time and batch serialization in Spark UI. This diagnose action addresses a different case (vectorized udf), where row-by-row Python invocation dominates stage runtime. Pandas UDFs process batches through Arrow rather than per-row Python calls. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.

Inspect retry attempt logs and write transaction identifiers. This diagnose action addresses a different case (job retry semantics), where the downstream write repeats non-idempotent side effects on retry. Retries re-execute task logic; application code must protect repeated effects. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.

Check ordering and duplicate keys in the source window specification. This diagnose action addresses a different case (window latest record), where the pipeline picks an arbitrary record instead of the newest sequence. ROW_NUMBER ordered by a deterministic sequence within each key selects the desired latest row. In the stated problem, a retry of a completed batch produces duplicate sink rows; this choice would not establish the required streaming idempotent sink decision.

Apply row_number partitioned by key with explicit descending sequence. This implement action addresses a different case (latest-per-key with ties), where two events have the same event time and yield different surviving rows. A deterministic tie-breaker is required in a per-key window ranking. In the stated problem, events from different regions are ordered incorrectly in a UTC dashboard; this choice would not establish the required timestamp timezone normalization decision.

Apply a timestamp validation rule and route invalid source events to quarantine. This implement action addresses a different case (quarantine bad records), where dropping invalid rows silently hides upstream quality failures. Quarantine preserves invalid events separately while allowing valid records through. In the stated problem, events from different regions are ordered incorrectly in a UTC dashboard; this choice would not establish the required timestamp timezone normalization decision.

Change to a left join and explicitly handle missing dimension fields. This implement action addresses a different case (join with nullable keys), where rows with null external IDs disappear from an inner join. SQL null equality and join type determine whether unmatched rows survive. In the stated problem, events from different regions are ordered incorrectly in a UTC dashboard; this choice would not establish the required timestamp timezone normalization decision.

Audit share membership changes against approvals. This monitor action addresses a different case (share catalog scope), where the recipient can discover unintended datasets. A Delta Sharing share should contain only explicitly approved assets. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.

Audit federation-specific grants and unexpected access. This monitor action addresses a different case (foreign catalog permission), where catalog discovery was mistaken for full table access. Unity Catalog usage and object privileges are still required for foreign data objects. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.

Monitor provider refresh and recipient visibility delays. This monitor action addresses a different case (shared table freshness), where consumers assume the share is a nightly static export. Delta Sharing exposes current authorized data as the provider updates the source. In the stated problem, the partner cannot use an internal Databricks-to-Databricks credential flow; this choice would not establish the required open protocol recipient decision.

Monitor automatic key decisions and maintenance operations. This monitor action addresses a different case (automatic clustering), where manual clustering keys quickly become stale. Automatic liquid clustering can select keys using query workload when platform prerequisites are met. In the stated problem, the proposed maintenance script also runs ZORDER on that clustered table; this choice would not establish the required clustering versus zorder decision.

Monitor deletion-vector accumulation and subsequent compaction. This monitor action addresses a different case (deletion vectors), where each small delete rewrites large data files and increases write cost. Deletion vectors can mark deleted rows without immediate whole-file rewrite when supported. In the stated problem, the proposed maintenance script also runs ZORDER on that clustered table; this choice would not establish the required clustering versus zorder decision.

Monitor plans that lose partition pruning. This monitor action addresses a different case (partition pruning), where a query wraps business_date in a transformation and reads many partitions. Direct predicates on partition keys can enable partition pruning. In the stated problem, the proposed maintenance script also runs ZORDER on that clustered table; this choice would not establish the required clustering versus zorder decision.

Create a supported metrics query for failed expectation records. This implement action addresses a different case (expectation monitoring), where analysts confuse successful pipeline status with zero rejected records. Quality metric events reveal input and dropped records separately from update status. In the stated problem, operators manually check the run page every morning; this choice would not establish the required job email notification decision.

Implement Jobs API run enumeration and status extraction. This implement action addresses a different case (jobs rest monitoring), where a manual UI report omits failed tasks and paginated runs. The Jobs API exposes run records that a program can paginate and aggregate. In the stated problem, operators manually check the run page every morning; this choice would not establish the required job email notification decision.

Configure SQL alert evaluation and approved notification destination. This implement action addresses a different case (sql alert threshold), where the current dashboard has no automated alert for missing units. SQL alerts can evaluate a query result against a configured threshold. In the stated problem, operators manually check the run page every morning; this choice would not establish the required job email notification decision.

Mock external reads in unit tests and run real dependency checks only in integration stages. This implement action addresses a different case (unit versus integration tests), where small logic regressions are discovered late and costly to debug. Unit tests isolate transformations while integration tests validate external contracts. In the stated problem, tests only assert that the DataFrame is nonempty; this choice would not establish the required dataframe testing decision.

Compose transformations with DataFrame.transform and explicit configuration. This implement action addresses a different case (transform composability), where business logic depends on notebook state and widgets. DataFrame.transform supports composable DataFrame-to-DataFrame functions. In the stated problem, tests only assert that the DataFrame is nonempty; this choice would not establish the required dataframe testing decision.

Use row_number over the window and retain only rank one. This implement action addresses a different case (window latest record), where the pipeline picks an arbitrary record instead of the newest sequence. ROW_NUMBER ordered by a deterministic sequence within each key selects the desired latest row. In the stated problem, tests only assert that the DataFrame is nonempty; this choice would not establish the required dataframe testing decision.

Inspect CLI target resolution and workspace host for the command. This diagnose action addresses a different case (target-safe deployment), where the wrong workspace is selected because target is implicit. Explicit bundle target selection prevents accidental environment deployment. In the stated problem, re-running a huge source in development is slow and risky; this choice would not establish the required pipeline debug sampling decision.

Inspect effective grants on source, target and parent containers for the run-as principal. This diagnose action addresses a different case (job run permission debug), where the run-as principal lacks MODIFY on the target table. Unity Catalog authorization applies to the job execution identity. In the stated problem, re-running a huge source in development is slow and risky; this choice would not establish the required pipeline debug sampling decision.

Inspect the bundle manifest schema and validation result. This diagnose action addresses a different case (bundle validation), where an invalid resource key would be rejected only after production change. Bundle validation detects many configuration problems before deployment. In the stated problem, re-running a huge source in development is slow and risky; this choice would not establish the required pipeline debug sampling decision.

Use a binaryFile ingest path for nontext payloads. This design action addresses a different case (binary payload metadata), where a line-oriented text reader corrupts the raw binary payload. binaryFile input preserves raw file bytes and path metadata rather than decoding lines. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.

Use one durable checkpoint and clear idempotency keys for replay. This design action addresses a different case (retry-safe file loads), where a partially processed input file is loaded twice into Delta. Checkpointed incremental ingestion or stable event-level deduplication prevents repeated effects. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.

Choose compatible Avro schema evolution and default-value conventions. This design action addresses a different case (avro schema evolution), where an older consumer cannot deserialize records from a newer producer schema. Avro writer-reader schema compatibility must be maintained for evolution. In the stated problem, an XML ingest incorrectly flattens multiple line items into one text field; this choice would not establish the required xml repeating elements decision.