Databricks Certified Machine Learning Professional Practice Test 3
Question 20
A production ML review contains this situation: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. What is the most material immediate risk in this situation?
- Online performance degrades because the model receives different kinds of feature values than it learned.
- Duplicate or omitted trials distort comparison when two partial histories are treated as independent full searches.
- Important diagnostic evidence is absent from model review despite a reported scalar score.
- Tracking artifacts create unauthorized exposure paths for customers' personal data.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. Online performance degrades because the model receives different kinds of feature values than it learned. The causal constraint is: The model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
Incorrect Answers
B:That direct consequence is directed at a different advanced mlflow usage situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. The more immediate risk here is online performance degrades because the model receives different kinds of feature values than it learned.
C:That direct consequence is directed at a different advanced mlflow usage situation: An ML engineer logs a calibration plot as a string metric, then downstream reviewers cannot inspect the visualization or its input version. The more immediate risk here is online performance degrades because the model receives different kinds of feature values than it learned.
D:That direct consequence is directed at a different advanced mlflow usage situation: A regulated analytics team uploads full customer identifiers inside MLflow model artifacts during troubleshooting. The more immediate risk here is online performance degrades because the model receives different kinds of feature values than it learned.
Question 21
Consider the following fictional Databricks production case: A fairness reviewer needs the difference in false-negative rates between two approved cohorts, but the monitor computes an aggregate without cohort labels. Which observation most directly tests the suspected failure boundary?
- Inspect baseline table versions and the order of writes relative to metric computations.
- Compare inference IDs and prediction times against actual confirmed label availability.
- Inspect subgroup membership, prediction/label joins and false-negative denominator in the source table.
- Compute recall for new and established accounts with matched mature labels and stable cohort definitions.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect subgroup membership, prediction/label joins and false-negative denominator in the source table. The underlying technical issue is that a custom fairness comparison needs matched labels, model predictions and governed subgroup definitions.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A baseline table is overwritten every night with the current day's data before a drift calculation, making the comparison always look unchanged. For the stated incident, the decisive observation is inspect subgroup membership, prediction/label joins and false-negative denominator in the source table.
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A fraud model predicts instantly but labels arrive 21 days later; an inference dashboard treats unlabelled transactions as negative examples. For the stated incident, the decisive observation is inspect subgroup membership, prediction/label joins and false-negative denominator in the source table.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An account-risk model's fraud recall is stable overall but unexpectedly falls for new customers in their first 30 days. For the stated incident, the decisive observation is inspect subgroup membership, prediction/label joins and false-negative denominator in the source table.
Question 22
The engineering team encounters the following issue: A feature-engineering job is CPU-bound on hundreds of independent daily partitions and a team upgrades only the driver to a much larger machine. Which technical principle most directly explains the root of the issue?
- Independent per-group models are suitable for distributing grouped pandas-function work, subject to available compute and partition size.
- Parallelizable fraction and upstream service limits bound the benefits of adding model-training workers.
- Data-parallel workloads may benefit from more executor cores and appropriately partitioned tasks rather than vertical driver scaling.
- Tuning runs must use equivalent evaluation data and metric definitions before ranking configurations.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Data-parallel workloads may benefit from more executor cores and appropriately partitioned tasks rather than vertical driver scaling. To test it, compare driver utilization, active executor CPU, task backlog and partition parallelism.
Incorrect Answers
A:That underlying principle is directed at a different scaling and tuning situation: An ecommerce analyst fits one independent scikit-learn model per 3,000 product categories, but the driver sequentially loops through all category subsets. The current incident instead turns on data-parallel workloads may benefit from more executor cores and appropriately partitioned tasks rather than vertical driver scaling.
B:That underlying principle is directed at a different scaling and tuning situation: An ML project has a single sequential external feature-service dependency and expects data-parallel workers to increase throughput 20-fold. The current incident instead turns on data-parallel workloads may benefit from more executor cores and appropriately partitioned tasks rather than vertical driver scaling.
D:That underlying principle is directed at a different scaling and tuning situation: A hyperparameter team compares two Optuna studies with different train/validation splits and chooses the lower loss as though both metrics were directly comparable. The current incident instead turns on data-parallel workloads may benefit from more executor cores and appropriately partitioned tasks rather than vertical driver scaling.
Question 23
In an enterprise ML workflow, the reported issue is: A feature stream loses half its source events and a retraining policy treats the resulting input shift as proof that the learned concept changed. How should the engineer validate the correction?
- A source-outage fixture triggers pipeline remediation rather than automatic model replacement.
- Repeated evaluation or confidence testing supports the chosen promotion decision at the required confidence.
- A candidate promotion gate rejects the high-value-account regression in controlled testing.
- The chosen model passes both untouched predictive evaluation and endpoint load tests.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. A source-outage fixture triggers pipeline remediation rather than automatic model replacement. It confirms whether the targeted action succeeded: restore missing data and validate collection health before deciding whether a model retrain is warranted.
Incorrect Answers
B:That acceptance test is directed at a different automated retraining situation: A new model beats the champion by 0.1 percentage points on a tiny evaluation set, and the team automatically promotes it despite large uncertainty. Here the actual success criterion is a source-outage fixture triggers pipeline remediation rather than automatic model replacement.
C:That acceptance test is directed at a different automated retraining situation: A new fraud classifier improves overall F1 but doubles false negatives for high-value payment accounts, and an automated policy chooses it on F1 alone. Here the actual success criterion is a source-outage fixture triggers pipeline remediation rather than automatic model replacement.
D:That acceptance test is directed at a different automated retraining situation: An AutoML training job always chooses the model with the lowest training loss, even when the deployed service has a strict memory and P95 latency budget. Here the actual success criterion is a source-outage fixture triggers pipeline remediation rather than automatic model replacement.
Question 24
Consider the following fictional Databricks production case: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. Which implementation is most appropriate for the stated incident?
- Tune outer concurrency based on actual cluster capacity and each trial's resource demand.
- Connect supported Optuna study execution to MLflow per-trial tracking and preserve the selected objective value.
- Select a suitable larger device or validated model-parallel/sharded execution design before scaling replicas.
- Declare correct per-trial GPU reservations and reduce concurrent trials until memory fits.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Declare correct per-trial GPU reservations and reduce concurrent trials until memory fits. The issue matters because multiple trials contend for the same accelerator and fail with memory exhaustion.
Incorrect Answers
A:That corrective action is directed at a different scaling and tuning situation: An operations model uses a SparkML estimator that itself distributes data, yet the team launches dozens of concurrent full-cluster fits on the same fixed small cluster. The relevant corrective action here is declare correct per-trial GPU reservations and reduce concurrent trials until memory fits.
B:That corrective action is directed at a different scaling and tuning situation: An Optuna search runs 80 hyperparameter trials but records only the last winning score in MLflow, making comparisons impossible. The relevant corrective action here is declare correct per-trial GPU reservations and reduce concurrent trials until memory fits.
C:That corrective action is directed at a different scaling and tuning situation: An image model needs 90 GB accelerator memory for one training replica while each available GPU has 24 GB and engineers propose only adding more identical nodes. The relevant corrective action here is declare correct per-trial GPU reservations and reduce concurrent trials until memory fits.
Question 25
In an enterprise ML workflow, the reported issue is: A customer segmentation project promotes a scoring job but omits a database migration required by its new feature schema. Before modifying the pipeline, which evidence should be collected first?
- Inventory model run IDs, registered versions, job configuration and deployment references.
- Compare required feature schema in the release with deployed table and endpoint contracts.
- Trace a sample production prediction backward through model, feature snapshot and logged run metadata.
- Compare experiment write rights with registered model and serving endpoint permissions for test identities.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Compare required feature schema in the release with deployed table and endpoint contracts. The underlying technical issue is that model lifecycle deployments require compatibility checks among code, schema, feature tables and serving targets.
Incorrect Answers
A:That diagnostic evidence is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. For the stated incident, the decisive observation is compare required feature schema in the release with deployed table and endpoint contracts.
C:That diagnostic evidence is directed at a different model lifecycle management situation: An organization has reliable model serving but never archives feature definitions or experiment parameters and cannot reproduce predictions after drift. For the stated incident, the decisive observation is compare required feature schema in the release with deployed table and endpoint contracts.
D:That diagnostic evidence is directed at a different model lifecycle management situation: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. For the stated incident, the decisive observation is compare required feature schema in the release with deployed table and endpoint contracts.
Question 26
A production ML review contains this situation: A batch scoring pipeline trains one per-region model and then joins model predictions back using region alone, duplicating rows because each region contains multiple versioned models. Which diagnostic check provides the strongest direct evidence?
- Inspect the Optuna trial table and compare its IDs and metrics against MLflow runs.
- Inspect model registry versions, join keys and output cardinality against source record IDs.
- Measure repeated dataset exports and per-trial deserialization relative to model-fitting time.
- Inspect runtime GPU memory per worker alongside Ray trial resource requests.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Inspect model registry versions, join keys and output cardinality against source record IDs. The underlying technical issue is that per-group model predictions require unambiguous region-and-model-version associations before row-level output joins.
Incorrect Answers
A:That diagnostic evidence is directed at a different scaling and tuning situation: An Optuna search runs 80 hyperparameter trials but records only the last winning score in MLflow, making comparisons impossible. For the stated incident, the decisive observation is inspect model registry versions, join keys and output cardinality against source record IDs.
C:That diagnostic evidence is directed at a different scaling and tuning situation: A hyperparameter search operates on large Spark-managed feature tables but each candidate model is small and independent, and workers repeatedly convert the entire table for each Ray task. For the stated incident, the decisive observation is inspect model registry versions, join keys and output cardinality against source record IDs.
D:That diagnostic evidence is directed at a different scaling and tuning situation: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. For the stated incident, the decisive observation is inspect model registry versions, join keys and output cardinality against source record IDs.
Question 27
A production ML review contains this situation: A batch scoring service loads a classifier version from last month while using today's new feature indexer, silently changing categorical positions. What production-safe change best matches the evidence?
- Use Tokenizer.transform in the correct pipeline position while reserving fit for actual Estimators.
- Deploy the fitted preprocessing and estimator together as one versioned pipeline artifact.
- Evaluate the model with class-sensitive metrics and choose thresholds aligned with false-positive and missed-fraud tradeoffs.
- Build an explicit parameter grid with a compatible CrossValidator or TrainValidationSplit and preserve a final untouched test set.
Explanation
Correct Answer
B:This corrects the relevant failure boundary instead of merely hiding the symptom. Deploy the fitted preprocessing and estimator together as one versioned pipeline artifact. The issue matters because the same raw category maps to a different vector position and produces misleading predictions.
Incorrect Answers
A:That corrective action is directed at a different using spark ml situation: A text pipeline repeatedly fits a Tokenizer stage even though tokenization simply converts strings into token arrays without learning a model. The relevant corrective action here is deploy the fitted preprocessing and estimator together as one versioned pipeline artifact.
C:That corrective action is directed at a different using spark ml situation: A fraud detector predicts the majority no-fraud class for 99.4% of cases and advertises 99.4% accuracy while missing most positives. The relevant corrective action here is deploy the fitted preprocessing and estimator together as one versioned pipeline artifact.
D:That corrective action is directed at a different using spark ml situation: An ML engineer manually compares two logistic-regression regularization values on a single holdout but has no reusable SparkML parameter-search plan. The relevant corrective action here is deploy the fitted preprocessing and estimator together as one versioned pipeline artifact.
Question 28
In an enterprise ML workflow, the reported issue is: An operations model uses a SparkML estimator that itself distributes data, yet the team launches dozens of concurrent full-cluster fits on the same fixed small cluster. What likely operational or model-quality harm does this issue create?
- Repeated cross-engine transfers dominate tuning time and may exhaust network capacity.
- Available accelerators sit idle while the tuning window expires.
- Extra orchestration lowers efficiency and makes reproducibility more complex.
- Oversubscribed trials spend time waiting and can destabilize shared model training.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. Oversubscribed trials spend time waiting and can destabilize shared model training. The causal constraint is: Parallel outer trials of already distributed estimators can contend for the same workers and increase overall elapsed time.
Incorrect Answers
A:That direct consequence is directed at a different scaling and tuning situation: A hyperparameter search operates on large Spark-managed feature tables but each candidate model is small and independent, and workers repeatedly convert the entire table for each Ray task. The more immediate risk here is oversubscribed trials spend time waiting and can destabilize shared model training.
B:That direct consequence is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. The more immediate risk here is oversubscribed trials spend time waiting and can destabilize shared model training.
C:That direct consequence is directed at a different scaling and tuning situation: A moderately sized dataset fits on one worker but a team provisions a huge distributed cluster and observes overhead higher than model fit. The more immediate risk here is oversubscribed trials spend time waiting and can destabilize shared model training.
Question 29
The engineering team encounters the following issue: A model serving endpoint is created manually in the UI and every staging deployment differs subtly from production because no deployment definition is saved. Which technical principle most directly explains the root of the issue?
- Environment architecture should isolate access, data destinations and ownership for different stages.
- Production ML jobs need owned service identities, durable environments and resource configuration independent of individuals.
- Declarative automation or asset-bundle definitions can version environment-specific ML assets and reproduce deployments.
- Scalable environment design must account for data locality, residency and endpoint boundaries.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Declarative automation or asset-bundle definitions can version environment-specific ML assets and reproduce deployments. To test it, compare staging and production endpoint, experiment and model registry settings against source-controlled configuration.
Incorrect Answers
A:That underlying principle is directed at a different environment architectures situation: A company operates development and production models in one workspace under the same broad service account and frequently overwrites production tables during experiments. The current incident instead turns on declarative automation or asset-bundle definitions can version environment-specific ML assets and reproduce deployments.
B:That underlying principle is directed at a different environment architectures situation: A distributed retraining workflow uses ephemeral personal compute and fails when the original engineer's account is disabled. The current incident instead turns on declarative automation or asset-bundle definitions can version environment-specific ML assets and reproduce deployments.
D:That underlying principle is directed at a different environment architectures situation: A multilingual prediction service must handle regional data residency, but the team copies a shared feature cache across regions without governance. The current incident instead turns on declarative automation or asset-bundle definitions can version environment-specific ML assets and reproduce deployments.
Question 30
The engineering team encounters the following issue: An alert from a feature monitor automatically replaces the live model with a newly trained artifact without testing or approval. Which experiment most directly demonstrates a successful fix?
- A candidate promotion gate rejects the high-value-account regression in controlled testing.
- Repeated evaluation or confidence testing supports the chosen promotion decision at the required confidence.
- An intentionally bad candidate is rejected while an approved candidate proceeds through controlled release.
- A repeated fixed-dataset evaluation yields reproducible candidate ordering.
Explanation
Correct Answer
C:This verification measures the required result rather than just process liveness. An intentionally bad candidate is rejected while an approved candidate proceeds through controlled release. It confirms whether the targeted action succeeded: run candidate training after the trigger but require explicit quality/security gates before any live promotion.
Incorrect Answers
A:That acceptance test is directed at a different automated retraining situation: A new fraud classifier improves overall F1 but doubles false negatives for high-value payment accounts, and an automated policy chooses it on F1 alone. Here the actual success criterion is an intentionally bad candidate is rejected while an approved candidate proceeds through controlled release.
B:That acceptance test is directed at a different automated retraining situation: A new model beats the champion by 0.1 percentage points on a tiny evaluation set, and the team automatically promotes it despite large uncertainty. Here the actual success criterion is an intentionally bad candidate is rejected while an approved candidate proceeds through controlled release.
D:That acceptance test is directed at a different automated retraining situation: A retraining pipeline ranks candidate models using metrics calculated on different dataset versions, then selects the numerically highest result. Here the actual success criterion is an intentionally bad candidate is rejected while an approved candidate proceeds through controlled release.