Databricks Certified Machine Learning Professional Practice Test 12
Question 110
The engineering team encounters the following issue: A monitor is pointed at a Unity Catalog table but the automation identity lacks the privileges needed to generate profile outputs. Which technical principle most directly explains the root of the issue?
- Monitor creation and ongoing metric writes depend on governed resource access and configured output locations.
- Monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence.
- Reference populations should match relevant seasonality and business context when interpreting drift.
- Observability starts with traceable, appropriately governed prediction logging before statistical analysis.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Monitor creation and ongoing metric writes depend on governed resource access and configured output locations. To test it, inspect effective owner permissions, catalog/schema access and creation error logs for the target table.
Incorrect Answers
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A research team captures only daily batch inference totals with no row-level timestamps but insists on per-minute live inference drift analysis. The current incident instead turns on monitor creation and ongoing metric writes depend on governed resource access and configured output locations.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. The current incident instead turns on monitor creation and ongoing metric writes depend on governed resource access and configured output locations.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A serving endpoint appears healthy but nobody records model version, request identifier or prediction value, so a drift incident cannot be reconstructed. The current incident instead turns on monitor creation and ongoing metric writes depend on governed resource access and configured output locations.
Question 111
Review this production model incident: A release pipeline runs development unit tests against a different dependency image from the one deployed to production. What is the central engineering constraint behind this behavior?
- Integration fixtures must check semantic prediction values and class labels, not only response existence.
- Release-stage testing must validate the actual artifact and dependency environment being promoted.
- Deterministic function tests should include degenerate data distributions that violate assumed mathematical denominators.
- Separating pure functions from orchestration can make unit tests faster while retaining notebook-driven workflows.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Release-stage testing must validate the actual artifact and dependency environment being promoted. To test it, compare image digests and dependency manifests across unit, integration and deployment stages.
Incorrect Answers
A:That underlying principle is directed at a different validation testing situation: An end-to-end validation uses only a synthetic positive label and fails to detect a swapped positive/negative class mapping in inference. The current incident instead turns on release-stage testing must validate the actual artifact and dependency environment being promoted.
C:That underlying principle is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. The current incident instead turns on release-stage testing must validate the actual artifact and dependency environment being promoted.
D:That underlying principle is directed at a different validation testing situation: A team hides all preprocessing logic inside one long notebook, making it hard to test independently without starting a Spark cluster. The current incident instead turns on release-stage testing must validate the actual artifact and dependency environment being promoted.
Question 112
A production ML review contains this situation: An account-risk model's fraud recall is stable overall but unexpectedly falls for new customers in their first 30 days. What is the most material immediate risk in this situation?
- Selection bias looks like model input deterioration even though each subgroup is stable.
- Early-account fraud receives inadequate detection despite acceptable combined recall.
- Traffic growth is mistaken for increasing risk because the raw count rises mechanically.
- The monitoring pipeline either floods the SOC or ignores new material shifts.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. Early-account fraud receives inadequate detection despite acceptable combined recall. The causal constraint is: Life-cycle cohort slices can expose onboarding weaknesses that broad averages obscure.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A customer-risk monitoring job compares an unfiltered current high-risk segment to an all-customer historical baseline and interprets the difference as temporal drift. The more immediate risk here is early-account fraud receives inadequate detection despite acceptable combined recall.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A monitoring team wants a business metric for the share of transactions exceeding a risk threshold but records only the raw count of high-risk transactions. The more immediate risk here is early-account fraud receives inadequate detection despite acceptable combined recall.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. The more immediate risk here is early-account fraud receives inadequate detection despite acceptable combined recall.
Question 113
A production ML review contains this situation: A loan-risk feature usually has median debt-to-income 0.34, but a new cohort has median 0.61 while the average stays almost unchanged due to outliers. What production-safe change best matches the evidence?
- Add delayed-ground-truth performance tracking alongside existing drift metrics and health signals.
- Track distribution-sensitive numeric drift with a justified threshold and investigate the affected population.
- Configure a supported time-series profile with the appropriate timestamp and daily granularity.
- Retain a stable reference comparison in addition to short-term change detection.
Explanation
Correct Answer
B:This corrects the relevant failure boundary instead of merely hiding the symptom. Track distribution-sensitive numeric drift with a justified threshold and investigate the affected population. The issue matters because the model receives materially shifted numeric inputs while a mean-only dashboard falsely appears stable.
Incorrect Answers
A:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A risk service measures feature drift but neglects observed-label performance and continues serving an accurate-looking yet ineffective model. The relevant corrective action here is track distribution-sensitive numeric drift with a justified threshold and investigate the affected population.
C:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A fleet sensor table records measurement timestamps and analysts need daily drift trends; a snapshot-only profile erases the time dimension. The relevant corrective action here is track distribution-sensitive numeric drift with a justified threshold and investigate the affected population.
D:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. The relevant corrective action here is track distribution-sensitive numeric drift with a justified threshold and investigate the affected population.
Question 114
A production ML review contains this situation: A monitor alerts whenever a ratio exceeds 0.3 for a single minute, although normal traffic produces brief spikes and operations require ten-minute persistence. What is the most material immediate risk in this situation?
- Gradual cumulative drift becomes invisible despite material divergence from the trained population.
- On-call teams are paged repeatedly for harmless short-lived noise.
- Concept drift or changed label relationships degrade decisions without an input-drift alarm.
- The monitor fabricates apparent minute-level precision from daily totals and misleads responders.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. On-call teams are paged repeatedly for harmless short-lived noise. The causal constraint is: Alert windows and persistence conditions must align with the service objective to avoid transient false positives.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. The more immediate risk here is on-call teams are paged repeatedly for harmless short-lived noise.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A risk service measures feature drift but neglects observed-label performance and continues serving an accurate-looking yet ineffective model. The more immediate risk here is on-call teams are paged repeatedly for harmless short-lived noise.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A research team captures only daily batch inference totals with no row-level timestamps but insists on per-minute live inference drift analysis. The more immediate risk here is on-call teams are paged repeatedly for harmless short-lived noise.
Question 115
The engineering team encounters the following issue: A loan-risk feature usually has median debt-to-income 0.34, but a new cohort has median 0.61 while the average stays almost unchanged due to outliers. What direct consequence should the owner prioritize if no fix is made?
- Performance comparisons by deployed model version cannot be calculated from the generic snapshot configuration.
- On-call teams are paged repeatedly for harmless short-lived noise.
- The model receives materially shifted numeric inputs while a mean-only dashboard falsely appears stable.
- Routine weekend traffic launches unnecessary retraining and distracts engineers.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. The model receives materially shifted numeric inputs while a mean-only dashboard falsely appears stable. The causal constraint is: Distributional drift cannot be ruled out by one stable mean; numeric quantiles and supported statistical comparisons can expose shape changes.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A churn model logs requests, predictions, model versions and later ground-truth labels, but monitoring is configured as a generic feature snapshot. The more immediate risk here is the model receives materially shifted numeric inputs while a mean-only dashboard falsely appears stable.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A monitor alerts whenever a ratio exceeds 0.3 for a single minute, although normal traffic produces brief spikes and operations require ten-minute persistence. The more immediate risk here is the model receives materially shifted numeric inputs while a mean-only dashboard falsely appears stable.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. The more immediate risk here is the model receives materially shifted numeric inputs while a mean-only dashboard falsely appears stable.
Question 116
The engineering team encounters the following issue: An ML pipeline computes training features in one notebook and duplicates slightly different SQL in the online pipeline for the same entities. What direct consequence should the owner prioritize if no fix is made?
- Credential compromise exposes unrelated customers' private feature data.
- One purchase increases the spending feature twice, affecting downstream risk scores.
- Training-serving skew creates inconsistent values for identically named features.
- Invalid numeric features produce serving errors or hidden downstream score distortions.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. Training-serving skew creates inconsistent values for identically named features. The causal constraint is: Feature reuse requires a shared governed definition rather than divergent training and serving transformations.
Incorrect Answers
A:That direct consequence is directed at a different advanced feature store concepts situation: A model serving service uses a broadly privileged shared token to read every online feature table when it needs only one approved feature set. The more immediate risk here is training-serving skew creates inconsistent values for identically named features.
B:That direct consequence is directed at a different advanced feature store concepts situation: A real-time feature job updates user purchase totals without checking whether upstream transaction events are delivered twice. The more immediate risk here is training-serving skew creates inconsistent values for identically named features.
D:That direct consequence is directed at a different advanced feature store concepts situation: A payment classifier's on-demand feature division returns infinity for zero denominators and fails serialization in the serving endpoint. The more immediate risk here is training-serving skew creates inconsistent values for identically named features.
Question 117
The engineering team encounters the following issue: A model is promoted to production with its code unchanged but a different unpinned training library version is installed, altering prediction preprocessing. What corrective action should the engineer prefer?
- Bind training outputs to registered immutable versions and promote them through governed jobs or serving endpoints.
- Record end-to-end lineage and immutable versions at each necessary lifecycle boundary.
- Pin and validate supported dependency versions as part of the promoted model/code release.
- Use a supported governed model registry and controlled permissions for production model versions.
Explanation
Correct Answer
C:This corrects the relevant failure boundary instead of merely hiding the symptom. Pin and validate supported dependency versions as part of the promoted model/code release. The issue matters because apparently identical code yields different results or fails to load under the new runtime.
Incorrect Answers
A:That corrective action is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. The relevant corrective action here is pin and validate supported dependency versions as part of the promoted model/code release.
B:That corrective action is directed at a different model lifecycle management situation: An organization has reliable model serving but never archives feature definitions or experiment parameters and cannot reproduce predictions after drift. The relevant corrective action here is pin and validate supported dependency versions as part of the promoted model/code release.
D:That corrective action is directed at a different model lifecycle management situation: An engineer stores model weights in a personal workspace folder and treats that path as a production artifact registry with audit history. The relevant corrective action here is pin and validate supported dependency versions as part of the promoted model/code release.
Question 118
A production ML review contains this situation: A streaming anomaly team uses a static read of a message directory once, believing the trained SparkML model will continuously produce scores for future arriving events. Which check shows that this particular problem has been resolved?
- The same versioned fitted model produces reproducible predictions for the saved scoring fixture.
- A realistic concurrent request test meets the stated end-to-end latency and correctness target.
- The selected candidate has the minimum validated RMSE among comparable parameter configurations.
- A newly emitted synthetic event is scored and appears downstream without restarting the job.
Explanation
Correct Answer
D:This verification measures the required result rather than just process liveness. A newly emitted synthetic event is scored and appears downstream without restarting the job. It confirms whether the targeted action succeeded: construct a supported streaming ingestion and scoring query with checkpointing and compatible output semantics.
Incorrect Answers
A:That acceptance test is directed at a different using spark ml situation: An analyst loads a fitted LogisticRegressionModel but calls fit on it for every scored batch rather than transform. Here the actual success criterion is a newly emitted synthetic event is scored and appears downstream without restarting the job.
B:That acceptance test is directed at a different using spark ml situation: An online pricing service requires a personalized response within 70 milliseconds per request but its team schedules a full overnight batch job as the only inference route. Here the actual success criterion is a newly emitted synthetic event is scored and appears downstream without restarting the job.
C:That acceptance test is directed at a different using spark ml situation: A SparkML grid-search experiment selects the model with the largest error metric even though the evaluator reports RMSE to minimize. Here the actual success criterion is a newly emitted synthetic event is scored and appears downstream without restarting the job.
Question 119
Consider the following fictional Databricks production case: A Databricks Model Serving endpoint hosts champion and candidate versions, but a release config accidentally routes 100% of traffic to the candidate instead of 10%. What consequence most specifically follows from leaving the issue unchanged?
- Rebuilding the old model under outage pressure extends an avoidable recovery window.
- All requests are exposed to the candidate during what operators believed was a small canary.
- A premature full rollout promotes a model before enough rare fraud outcomes can be observed.
- Regulated traffic reaches an unapproved model version contrary to residency controls.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. All requests are exposed to the candidate during what operators believed was a small canary. The causal constraint is: Traffic-routing configuration is an explicit version allocation and must match the authorized canary percentage.
Incorrect Answers
A:That direct consequence is directed at a different deployment strategies situation: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. The more immediate risk here is all requests are exposed to the candidate during what operators believed was a small canary.
C:That direct consequence is directed at a different deployment strategies situation: A candidate fraud model has near-identical average error to the champion, but an organization wants to compare rare fraudulent transactions during a brief canary. The more immediate risk here is all requests are exposed to the candidate during what operators believed was a small canary.
D:That direct consequence is directed at a different deployment strategies situation: A model release requires all clients from a restricted region to keep the old version, but a simple percentage split sends some of those clients to the candidate. The more immediate risk here is all requests are exposed to the candidate during what operators believed was a small canary.
Question 120
In an enterprise ML workflow, the reported issue is: A regulated analytics team uploads full customer identifiers inside MLflow model artifacts during troubleshooting. What technical distinction is essential to this investigation?
- Experiment tracking must minimize sensitive data and enforce artifact permissions rather than treating all model logs as harmless.
- Custom MLflow model packaging must include declared compatible dependencies for the independent serving environment.
- A resumed search needs stable experiment and parent-child provenance across partial reruns to support comparison.
- Request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Experiment tracking must minimize sensitive data and enforce artifact permissions rather than treating all model logs as harmless. To test it, review artifact payload fields, access controls and permitted data classifications before reuse.
Incorrect Answers
B:That underlying principle is directed at a different advanced mlflow usage situation: A team logs a custom Python model that imports a package installed only in the author's notebook, and the serving endpoint fails when loading the artifact. The current incident instead turns on experiment tracking must minimize sensitive data and enforce artifact permissions rather than treating all model logs as harmless.
C:That underlying principle is directed at a different advanced mlflow usage situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. The current incident instead turns on experiment tracking must minimize sensitive data and enforce artifact permissions rather than treating all model logs as harmless.
D:That underlying principle is directed at a different advanced mlflow usage situation: A fraud PyFunc uses a global mutable feature cache initialized by the first request and mixes data between subsequent users. The current incident instead turns on experiment tracking must minimize sensitive data and enforce artifact permissions rather than treating all model logs as harmless.