Databricks Certified Machine Learning Professional Practice Test 6
Question 50
In an enterprise ML workflow, the reported issue is: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. Before modifying the pipeline, which evidence should be collected first?
- Review rolling-window statistics alongside a fixed approved baseline for the same cohort.
- Compare feature drift trends with label-matured error rates and calibration by model version.
- Compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
- Compare inference IDs and prediction times against actual confirmed label availability.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Review rolling-window statistics alongside a fixed approved baseline for the same cohort. The underlying technical issue is that comparing consecutive windows and comparing to a stable reference answer different drift questions.
Incorrect Answers
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A risk service measures feature drift but neglects observed-label performance and continues serving an accurate-looking yet ineffective model. For the stated incident, the decisive observation is review rolling-window statistics alongside a fixed approved baseline for the same cohort.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. For the stated incident, the decisive observation is review rolling-window statistics alongside a fixed approved baseline for the same cohort.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A fraud model predicts instantly but labels arrive 21 days later; an inference dashboard treats unlabelled transactions as negative examples. For the stated incident, the decisive observation is review rolling-window statistics alongside a fixed approved baseline for the same cohort.
Question 51
The engineering team encounters the following issue: A payment classifier's on-demand feature division returns infinity for zero denominators and fails serialization in the serving endpoint. Which technical principle most directly explains the root of the issue?
- Point-in-time training joins must select features available at or before each label observation, not the latest record irrespective of date.
- On-demand feature functions must explicitly handle invalid input and output type constraints.
- Online feature access should use least-privilege identity and scoped permissions rather than universal shared credentials.
- Streaming derived features should deduplicate at a stable business-event key before non-idempotent aggregation.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. On-demand feature functions must explicitly handle invalid input and output type constraints. To test it, test zero, null, negative and normal synthetic input combinations against the declared feature schema.
Incorrect Answers
A:That underlying principle is directed at a different advanced feature store concepts situation: A fraud training row is labeled on March 10 but an ordinary entity join attaches an account balance updated on March 12. The current incident instead turns on on-demand feature functions must explicitly handle invalid input and output type constraints.
C:That underlying principle is directed at a different advanced feature store concepts situation: A model serving service uses a broadly privileged shared token to read every online feature table when it needs only one approved feature set. The current incident instead turns on on-demand feature functions must explicitly handle invalid input and output type constraints.
D:That underlying principle is directed at a different advanced feature store concepts situation: A real-time feature job updates user purchase totals without checking whether upstream transaction events are delivered twice. The current incident instead turns on on-demand feature functions must explicitly handle invalid input and output type constraints.
Question 52
Review this production model incident: A model is promoted to production with its code unchanged but a different unpinned training library version is installed, altering prediction preprocessing. What end-to-end verification is most persuasive here?
- A clean production-like environment loads the promoted artifact and matches approved prediction fixtures.
- A replay from retained lineage recreates an example prediction within documented tolerance.
- Negative permission tests deny unapproved endpoint and registry changes while allowing permitted experiment logging.
- A deployment record traces production predictions to the approved model version and generating run.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. A clean production-like environment loads the promoted artifact and matches approved prediction fixtures. It confirms whether the targeted action succeeded: pin and validate supported dependency versions as part of the promoted model/code release.
Incorrect Answers
B:That acceptance test is directed at a different model lifecycle management situation: An organization has reliable model serving but never archives feature definitions or experiment parameters and cannot reproduce predictions after drift. Here the actual success criterion is a clean production-like environment loads the promoted artifact and matches approved prediction fixtures.
C:That acceptance test is directed at a different model lifecycle management situation: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. Here the actual success criterion is a clean production-like environment loads the promoted artifact and matches approved prediction fixtures.
D:That acceptance test is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. Here the actual success criterion is a clean production-like environment loads the promoted artifact and matches approved prediction fixtures.
Question 53
Consider the following fictional Databricks production case: A team hides all preprocessing logic inside one long notebook, making it hard to test independently without starting a Spark cluster. Which observation most directly tests the suspected failure boundary?
- Compare class-index mappings and positive/negative fixture outputs from training to serving.
- Run a tiny constant-column fixture and inspect finite-value validation after transformation.
- Identify deterministic data transformations entangled with session setup and filesystem side effects.
- Inspect test-environment evidence and the presence of function-level synthetic fixtures.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Identify deterministic data transformations entangled with session setup and filesystem side effects. The underlying technical issue is that separating pure functions from orchestration can make unit tests faster while retaining notebook-driven workflows.
Incorrect Answers
A:That diagnostic evidence is directed at a different validation testing situation: An end-to-end validation uses only a synthetic positive label and fails to detect a swapped positive/negative class mapping in inference. For the stated incident, the decisive observation is identify deterministic data transformations entangled with session setup and filesystem side effects.
B:That diagnostic evidence is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. For the stated incident, the decisive observation is identify deterministic data transformations entangled with session setup and filesystem side effects.
D:That diagnostic evidence is directed at a different validation testing situation: A developer changes a feature transform in a notebook and directly uses production data as the only test case. For the stated incident, the decisive observation is identify deterministic data transformations entangled with session setup and filesystem side effects.
Question 54
Review this production model incident: An operations model uses a SparkML estimator that itself distributes data, yet the team launches dozens of concurrent full-cluster fits on the same fixed small cluster. What end-to-end verification is most persuasive here?
- A measured concurrency setting improves total completed trials per hour without memory or scheduling failures.
- A single training step completes within per-device memory limits under the chosen model placement.
- Tracing shows the intended supported distributed training path with correct outputs and resource use.
- A scheduled trial set uses distinct GPU reservations and runs without memory contention.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. A measured concurrency setting improves total completed trials per hour without memory or scheduling failures. It confirms whether the targeted action succeeded: tune outer concurrency based on actual cluster capacity and each trial's resource demand.
Incorrect Answers
B:That acceptance test is directed at a different scaling and tuning situation: An image model needs 90 GB accelerator memory for one training replica while each available GPU has 24 GB and engineers propose only adding more identical nodes. Here the actual success criterion is a measured concurrency setting improves total completed trials per hour without memory or scheduling failures.
C:That acceptance test is directed at a different scaling and tuning situation: A team switches a SparkML training stage to Ray without verifying that its chosen estimator supports Ray's distributed fit contracts. Here the actual success criterion is a measured concurrency setting improves total completed trials per hour without memory or scheduling failures.
D:That acceptance test is directed at a different scaling and tuning situation: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. Here the actual success criterion is a measured concurrency setting improves total completed trials per hour without memory or scheduling failures.
Question 55
In an enterprise ML workflow, the reported issue is: A test suite mocks every external interface so aggressively that even an incompatible feature table schema appears valid. Which change addresses the real cause without undermining other requirements?
- Guard zero-variance inputs and assert expected fallback or explicit failure semantics in unit tests.
- Add a release integration assertion for the expected immutable model version at the endpoint.
- Retain fast mocked unit tests and add realistic integration contract tests for critical boundaries.
- Use approved synthetic or appropriately deidentified test data with controlled production smoke verification.
Explanation
Correct Answer
C:This corrects the relevant failure boundary instead of merely hiding the symptom. Retain fast mocked unit tests and add realistic integration contract tests for critical boundaries. The issue matters because the system passes isolated tests and fails immediately after deployment.
Incorrect Answers
A:That corrective action is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. The relevant corrective action here is retain fast mocked unit tests and add realistic integration contract tests for critical boundaries.
B:That corrective action is directed at a different validation testing situation: A model version is registered but the deployment job points to an old alias after promotion and continues serving last week's model. The relevant corrective action here is retain fast mocked unit tests and add realistic integration contract tests for critical boundaries.
D:That corrective action is directed at a different validation testing situation: An engineer uses only production replay data to validate a code change and exposes customer events in an unrestricted test workspace. The relevant corrective action here is retain fast mocked unit tests and add realistic integration contract tests for critical boundaries.
Question 56
A production ML review contains this situation: A training notebook reports F1 for several models but only logs the final model's parameters in MLflow, leaving metrics and hyperparameters disconnected. Which statement best explains the failure mode?
- A custom MLflow PyFunc's input contract and online feature lookups must be designed for request-time availability and governance.
- MLflow run-level logging should attach parameters and evaluated metrics to the corresponding candidate run.
- The model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
- A resumed search needs stable experiment and parent-child provenance across partial reruns to support comparison.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. MLflow run-level logging should attach parameters and evaluated metrics to the corresponding candidate run. To test it, compare run parameter maps against logged evaluation metrics and the local experiment output.
Incorrect Answers
A:That underlying principle is directed at a different advanced mlflow usage situation: A real-time churn endpoint accepts customer IDs and must fetch current allowed account features, but a custom model class instead expects the caller to supply the entire internal training row. The current incident instead turns on mLflow run-level logging should attach parameters and evaluated metrics to the corresponding candidate run.
C:That underlying principle is directed at a different advanced mlflow usage situation: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. The current incident instead turns on mLflow run-level logging should attach parameters and evaluated metrics to the corresponding candidate run.
D:That underlying principle is directed at a different advanced mlflow usage situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. The current incident instead turns on mLflow run-level logging should attach parameters and evaluated metrics to the corresponding candidate run.
Question 57
The engineering team encounters the following issue: One enterprise account holds 80% of historical events, and grouped pandas training for that account exceeds executor memory while other accounts finish. What direct consequence should the owner prioritize if no fix is made?
- The rewritten job runs only on one worker or fails during model serialization.
- Unnecessary model partitioning adds network latency with little memory benefit.
- Available accelerators sit idle while the tuning window expires.
- A single skewed group repeatedly kills workers despite healthy median group runtime.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. A single skewed group repeatedly kills workers despite healthy median group runtime. The causal constraint is: The largest per-group pandas object, not average group size, controls peak memory for grouped model fits.
Incorrect Answers
A:That direct consequence is directed at a different scaling and tuning situation: A team switches a SparkML training stage to Ray without verifying that its chosen estimator supports Ray's distributed fit contracts. The more immediate risk here is a single skewed group repeatedly kills workers despite healthy median group runtime.
B:That direct consequence is directed at a different scaling and tuning situation: A tabular classifier fits comfortably on one GPU and 100 million independent examples are available, but a team proposes splitting the small model's layers across hosts. The more immediate risk here is a single skewed group repeatedly kills workers despite healthy median group runtime.
C:That direct consequence is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. The more immediate risk here is a single skewed group repeatedly kills workers despite healthy median group runtime.
Question 58
Consider the following fictional Databricks production case: A platform engineer creates a custom serving endpoint manually and assumes the exact settings can be recovered after an accidental deletion. What consequence most specifically follows from leaving the issue unchanged?
- The client amplifies throttling and creates a self-sustaining inference outage.
- Stale candidate predictions persist after traffic has returned to the champion.
- Reviewers cannot audit or grant precise access to a reproducible registered version.
- Recovery requires guessing traffic splits, permissions and served model versions during an outage.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. Recovery requires guessing traffic splits, permissions and served model versions during an outage. The causal constraint is: Repeatable endpoint deployment should preserve a controlled configuration definition and registered model identity.
Incorrect Answers
A:That direct consequence is directed at a different custom model serving situation: A real-time client retries every 429 rate-limit error immediately in an unbounded loop and multiplies load on the serving endpoint. The more immediate risk here is recovery requires guessing traffic splits, permissions and served model versions during an outage.
B:That direct consequence is directed at a different custom model serving situation: An integration caches predictions without recording the model version and returns yesterday's values after a controlled rollback. The more immediate risk here is recovery requires guessing traffic splits, permissions and served model versions during an outage.
C:That direct consequence is directed at a different custom model serving situation: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. The more immediate risk here is recovery requires guessing traffic splits, permissions and served model versions during an outage.
Question 59
The engineering team encounters the following issue: A hyperparameter search operates on large Spark-managed feature tables but each candidate model is small and independent, and workers repeatedly convert the entire table for each Ray task. What investigation would most effectively isolate this behavior?
- Inspect concurrent log timestamps and artifact overwrite patterns by MLflow run ID.
- Compare driver utilization, active executor CPU, task backlog and partition parallelism.
- Measure repeated dataset exports and per-trial deserialization relative to model-fitting time.
- Compare per-replica parameter memory with device capacity and supported sharding topology.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Measure repeated dataset exports and per-trial deserialization relative to model-fitting time. The underlying technical issue is that combining Spark feature preparation with Ray trial execution requires explicit materialization and data locality planning.
Incorrect Answers
A:That diagnostic evidence is directed at a different scaling and tuning situation: An Optuna optimization reuses the same MLflow run ID for concurrent trials, causing overlapping parameter values and artifacts. For the stated incident, the decisive observation is measure repeated dataset exports and per-trial deserialization relative to model-fitting time.
B:That diagnostic evidence is directed at a different scaling and tuning situation: A feature-engineering job is CPU-bound on hundreds of independent daily partitions and a team upgrades only the driver to a much larger machine. For the stated incident, the decisive observation is measure repeated dataset exports and per-trial deserialization relative to model-fitting time.
D:That diagnostic evidence is directed at a different scaling and tuning situation: A language model's weights cannot fit on one accelerator yet a team launches pure data-parallel replicas, each loading the entire model. For the stated incident, the decisive observation is measure repeated dataset exports and per-trial deserialization relative to model-fitting time.
Question 60
The engineering team encounters the following issue: An online behavioral feature counter consumes a streaming source but does not retain checkpoint state after worker restarts. Which technical principle most directly explains the root of the issue?
- Idempotent feature pipeline writes need stable entity/time keys and safe retry semantics.
- Online feature lookup keys must be stable and uniquely identify the intended entity.
- Fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency.
- A time-series feature table needs a documented timestamp key and an as-of lookup to avoid multiplying labels or attaching future values.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency. To test it, inspect checkpoint paths and duplicate feature writes after a simulated restart.
Incorrect Answers
A:That underlying principle is directed at a different advanced feature store concepts situation: A feature refresh writes duplicate records for each entity and logical timestamp after retries, breaking unique lookups. The current incident instead turns on fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency.
B:That underlying principle is directed at a different advanced feature store concepts situation: A low-latency feature table uses external customer names as lookup keys; names change and are not unique. The current incident instead turns on fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency.
D:That underlying principle is directed at a different advanced feature store concepts situation: A training dataset combines a feature table with multiple timestamped rows per device using an equality join on device ID only. The current incident instead turns on fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency.