Databricks Certified Machine Learning Professional Practice Test 87
Question 860
Consider the following fictional Databricks production case: A custom drift metric is expressed as today-minus-yesterday percentage, but yesterday's eligible count can be zero. What underlying principle determines the most appropriate interpretation?
- An alert is operationally useful only with explicit routing, threshold, urgency and escalation ownership.
- Performance trends require model-version provenance to attribute changes to the correct release.
- Custom derived metrics need deterministic handling for undefined denominators and small samples.
- Life-cycle cohort slices can expose onboarding weaknesses that broad averages obscure.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Custom derived metrics need deterministic handling for undefined denominators and small samples. To test it, test the metric with zero, small and normal baseline denominators and inspect SQL null behavior.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. The current incident instead turns on custom derived metrics need deterministic handling for undefined denominators and small samples.
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. The current incident instead turns on custom derived metrics need deterministic handling for undefined denominators and small samples.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: An account-risk model's fraud recall is stable overall but unexpectedly falls for new customers in their first 30 days. The current incident instead turns on custom derived metrics need deterministic handling for undefined denominators and small samples.
Question 861
Review this production model incident: A custom model endpoint meets mean response time of 75 milliseconds but P99 latency reaches 1.8 seconds during peak load. Which mitigation resolves the defect rather than concealing it?
- Monitor percentile latency and address verified bottlenecks under realistic peak traffic.
- Add documented first-time-user slices with adequate sample gates to inference performance monitoring.
- Create a dedicated approved critical-region alert with suitable minimum volume and failure-rate conditions.
- Normalize timestamp semantics before applying approved local-hour performance slices.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Monitor percentile latency and address verified bottlenecks under realistic peak traffic. The issue matters because a small but critical share of customers experiences repeated slow predictions.
Incorrect Answers
B:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. The relevant corrective action here is monitor percentile latency and address verified bottlenecks under realistic peak traffic.
C:That corrective action is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. The relevant corrective action here is monitor percentile latency and address verified bottlenecks under realistic peak traffic.
D:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A model's prediction error appears to spike at 2 a.m. local time but the monitoring table groups timestamps in UTC for a globally distributed region. The relevant corrective action here is monitor percentile latency and address verified bottlenecks under realistic peak traffic.
Question 862
The engineering team encounters the following issue: A new model version uses twice the memory per replica, but capacity monitoring tracks only CPU and crash counts. What corrective action should the engineer prefer?
- Calculate slice-specific error rates and a documented derived disparity metric with privacy safeguards.
- Add memory saturation alarms and right-size replicas or model footprint with a staged rollout.
- Create a dedicated approved critical-region alert with suitable minimum volume and failure-rate conditions.
- Maintain a model-version-linked training baseline alongside rolling comparisons.
Explanation
Correct Answer
B:This corrects the relevant failure boundary instead of merely hiding the symptom. Add memory saturation alarms and right-size replicas or model footprint with a staged rollout. The issue matters because instances restart unpredictably from memory pressure and disrupt scoring.
Incorrect Answers
A:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A fairness reviewer needs the difference in false-negative rates between two approved cohorts, but the monitor computes an aggregate without cohort labels. The relevant corrective action here is add memory saturation alarms and right-size replicas or model footprint with a staged rollout.
C:That corrective action is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. The relevant corrective action here is add memory saturation alarms and right-size replicas or model footprint with a staged rollout.
D:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A model trained on last year's customer mix is monitored only against yesterday's already shifted cohort, so its original training-population mismatch is hidden. The relevant corrective action here is add memory saturation alarms and right-size replicas or model footprint with a staged rollout.
Question 863
Consider the following fictional Databricks production case: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. Which observation most directly tests the suspected failure boundary?
- Review rolling-window statistics alongside a fixed approved baseline for the same cohort.
- Compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
- Inspect the stored snapshot rows and timestamps to determine whether historical hourly observations are retained.
- Bucket predictions by confidence and compare matured empirical positive rates for each bucket.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Compare historical baselines, current seasonal patterns and change approvals for the existing threshold. The underlying technical issue is that monitoring alert limits need controlled baseline revisions and documented operational validity.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. For the stated incident, the decisive observation is compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A retail profile is created on a static materialized daily inventory table; a team expects its initial snapshot to show each prior hour as a separate trend. For the stated incident, the decisive observation is compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A binary model's average predicted probability remains 0.4, but outcomes show that 0.8 predictions are now correct only half the time. For the stated incident, the decisive observation is compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
Question 864
The engineering team encounters the following issue: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. What direct consequence should the owner prioritize if no fix is made?
- The metric fails or silently excludes malformed observations, distorting the reported trend.
- Investigators cannot tell whether a particular model version produced the anomalous scores.
- The monitoring pipeline either floods the SOC or ignores new material shifts.
- Missing historical windows are treated as evidence of zero incidents or zero data change.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. The monitoring pipeline either floods the SOC or ignores new material shifts. The causal constraint is: Monitoring alert limits need controlled baseline revisions and documented operational validity.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An ML owner adds a custom metric computed from the average of a string-typed currency column without validating conversion. The more immediate risk here is the monitoring pipeline either floods the SOC or ignores new material shifts.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A serving endpoint appears healthy but nobody records model version, request identifier or prediction value, so a drift incident cannot be reconstructed. The more immediate risk here is the monitoring pipeline either floods the SOC or ignores new material shifts.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A retail profile is created on a static materialized daily inventory table; a team expects its initial snapshot to show each prior hour as a separate trend. The more immediate risk here is the monitoring pipeline either floods the SOC or ignores new material shifts.
Question 865
Consider the following fictional Databricks production case: During a controlled release, the candidate model's P95 latency triples and engineers want to change the registered model alias without adjusting endpoint routing. Which observation most directly tests the suspected failure boundary?
- Check independent health and version identity for both active and standby deployment configurations before cutover.
- Compare effective serving traffic split, model identities and alias resolution in the live endpoint.
- Test candidate-routing behavior for protected region identifiers across many simulated requests.
- Compare replica memory, P95 latency and model quality for both versions under realistic canary load.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Compare effective serving traffic split, model identities and alias resolution in the live endpoint. The underlying technical issue is that rollout recovery must address the actual endpoint traffic configuration, not assume an alias automatically changes every active route.
Incorrect Answers
A:That diagnostic evidence is directed at a different deployment strategies situation: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. For the stated incident, the decisive observation is compare effective serving traffic split, model identities and alias resolution in the live endpoint.
C:That diagnostic evidence is directed at a different deployment strategies situation: A model release requires all clients from a restricted region to keep the old version, but a simple percentage split sends some of those clients to the candidate. For the stated incident, the decisive observation is compare effective serving traffic split, model identities and alias resolution in the live endpoint.
D:That diagnostic evidence is directed at a different deployment strategies situation: A new recommendation model consumes three times as much memory and the team plans a canary based solely on prediction accuracy, ignoring serving capacity. For the stated incident, the decisive observation is compare effective serving traffic split, model identities and alias resolution in the live endpoint.
Question 866
In an enterprise ML workflow, the reported issue is: A continuous-delivery job deploys a notebook change through a bundle but skips validation of referenced experiment paths and job permissions. How should the engineer validate the correction?
- Load tests show production SLAs hold while research workloads respect assigned capacity limits.
- Region-specific test requests reach only authorized feature storage and compliant endpoints.
- The release pipeline validates all references and executes an authorized staging test job.
- A development identity fails negative write tests against production assets while normal dev jobs succeed.
Explanation
Correct Answer
C:This verification measures the required result rather than just process liveness. The release pipeline validates all references and executes an authorized staging test job. It confirms whether the targeted action succeeded: add bundle validation, permission checks and staging smoke tests before production promotion.
Incorrect Answers
A:That acceptance test is directed at a different environment architectures situation: A production deployment scales to 30 teams but all jobs share one compute pool with no capacity limits or cost attribution. Here the actual success criterion is the release pipeline validates all references and executes an authorized staging test job.
B:That acceptance test is directed at a different environment architectures situation: A multilingual prediction service must handle regional data residency, but the team copies a shared feature cache across regions without governance. Here the actual success criterion is the release pipeline validates all references and executes an authorized staging test job.
D:That acceptance test is directed at a different environment architectures situation: A company operates development and production models in one workspace under the same broad service account and frequently overwrites production tables during experiments. Here the actual success criterion is the release pipeline validates all references and executes an authorized staging test job.
Question 867
The engineering team encounters the following issue: A team logs a score labeled accuracy with values between 0 and 100 in some runs and between 0 and 1 in others, then ranks runs automatically. What direct consequence should the owner prioritize if no fix is made?
- Trial histories become detached from the actual optimization process and candidate provenance is lost.
- Online performance degrades because the model receives different kinds of feature values than it learned.
- A model can win solely because its reported metric uses a different scale.
- The deployed artifact cannot be tied to the training configuration that earned the winning metric.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. A model can win solely because its reported metric uses a different scale. The causal constraint is: Logged metric names require consistent definitions and units before their numerical values can be compared.
Incorrect Answers
A:That direct consequence is directed at a different advanced mlflow usage situation: An experiment tunes 40 candidates but logs each trial as an unrelated root MLflow run, so reviewers cannot reconstruct which search produced a winner. The more immediate risk here is a model can win solely because its reported metric uses a different scale.
B:That direct consequence is directed at a different advanced mlflow usage situation: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. The more immediate risk here is a model can win solely because its reported metric uses a different scale.
D:That direct consequence is directed at a different advanced mlflow usage situation: A data scientist selects the lowest validation loss from nested runs but reports only the parent run ID as the model-producing run. The more immediate risk here is a model can win solely because its reported metric uses a different scale.
Question 868
Consider the following fictional Databricks production case: An ML pipeline computes training features in one notebook and duplicates slightly different SQL in the online pipeline for the same entities. Which observation most directly tests the suspected failure boundary?
- Compare offline and serving results for synthetic zero-income, missing and normal customer fixtures.
- Compare last available reading age with each prediction cutoff and its permitted freshness window.
- Inspect online table key cardinality and mismatches before and after a synthetic rename.
- Diff the offline and live feature transformation definitions for the same synthetic entity inputs.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Diff the offline and live feature transformation definitions for the same synthetic entity inputs. The underlying technical issue is that feature reuse requires a shared governed definition rather than divergent training and serving transformations.
Incorrect Answers
A:That diagnostic evidence is directed at a different advanced feature store concepts situation: A customer-risk feature needs a ratio of current debt to income, but online serving recomputes it differently from the training pipeline. For the stated incident, the decisive observation is diff the offline and live feature transformation definitions for the same synthetic entity inputs.
B:That diagnostic evidence is directed at a different advanced feature store concepts situation: A sensor classifier silently reuses features from a year-old device after months without readings, despite a maximum allowed lookback of seven days. For the stated incident, the decisive observation is diff the offline and live feature transformation definitions for the same synthetic entity inputs.
C:That diagnostic evidence is directed at a different advanced feature store concepts situation: A low-latency feature table uses external customer names as lookup keys; names change and are not unique. For the stated incident, the decisive observation is diff the offline and live feature transformation definitions for the same synthetic entity inputs.
Question 869
The engineering team encounters the following issue: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. Which experiment most directly demonstrates a successful fix?
- A controlled example maps known training categories to stable numeric indices using the fitted model.
- A newly emitted synthetic event is scored and appears downstream without restarting the job.
- Holdout-only categories no longer influence the training-fitted indexer and metrics are recomputed on untouched holdout records.
- The accepted model meets both aggregate error and tail-error thresholds on untouched periods.
Explanation
Correct Answer
D:This verification measures the required result rather than just process liveness. The accepted model meets both aggregate error and tail-error thresholds on untouched periods. It confirms whether the targeted action succeeded: add a justified tail-sensitive error measure such as RMSE and inspect failure slices in model selection.
Incorrect Answers
A:That acceptance test is directed at a different using spark ml situation: A new Spark developer calls transform on an unfitted StringIndexer instance and expects learned category labels to appear automatically. Here the actual success criterion is the accepted model meets both aggregate error and tail-error thresholds on untouched periods.
B:That acceptance test is directed at a different using spark ml situation: A streaming anomaly team uses a static read of a message directory once, believing the trained SparkML model will continuously produce scores for future arriving events. Here the actual success criterion is the accepted model meets both aggregate error and tail-error thresholds on untouched periods.
C:That acceptance test is directed at a different using spark ml situation: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. Here the actual success criterion is the accepted model meets both aggregate error and tail-error thresholds on untouched periods.
Question 870
The engineering team encounters the following issue: An engineer stores model weights in a personal workspace folder and treats that path as a production artifact registry with audit history. What direct consequence should the owner prioritize if no fix is made?
- Production inference can access unrelated protected data or train on unintended tables.
- The artifact can disappear with an account change or be overwritten without reliable provenance.
- A healthy deployed job crashes on missing fields or silently reads wrong defaults.
- The live model logic diverges from tested code and cannot be reproduced during an incident.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. The artifact can disappear with an account change or be overwritten without reliable provenance. The causal constraint is: Governed model registration and access controls provide traceability beyond an ad hoc personal file path.
Incorrect Answers
A:That direct consequence is directed at a different model lifecycle management situation: A team trains on curated development features but deploys the same job with credentials that accidentally read unapproved production source tables. The more immediate risk here is the artifact can disappear with an account change or be overwritten without reliable provenance.
C:That direct consequence is directed at a different model lifecycle management situation: A customer segmentation project promotes a scoring job but omits a database migration required by its new feature schema. The more immediate risk here is the artifact can disappear with an account change or be overwritten without reliable provenance.
D:That direct consequence is directed at a different model lifecycle management situation: A regulated company changes deployed Python notebook code directly in production to correct a model transform while its test environment runs a different untracked revision. The more immediate risk here is the artifact can disappear with an account change or be overwritten without reliable provenance.