Databricks Certified Machine Learning Professional Practice Test 96
Question 950
Consider the following fictional Databricks production case: A retailer's features arrive two hours late, and offline training uses event time alone even though the online system could not have seen those late arrivals at prediction time. What underlying principle determines the most appropriate interpretation?
- Point-in-time correctness may require both event time and actual feature availability when ingestion latency matters.
- Low-latency serving requires online feature access with a documented freshness and lookup contract.
- On-demand feature serving must account for dependency latency and failure budgets within the endpoint SLA.
- Fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Point-in-time correctness may require both event time and actual feature availability when ingestion latency matters. To test it, compare feature event time, publish time and prediction decision time for a delayed fixture.
Incorrect Answers
B:That underlying principle is directed at a different advanced feature store concepts situation: A pricing endpoint queries a large offline Delta feature table for every user request, and P95 response time reaches several seconds. The current incident instead turns on point-in-time correctness may require both event time and actual feature availability when ingestion latency matters.
C:That underlying principle is directed at a different advanced feature store concepts situation: A streaming recommender's request-time feature function calls a slow third-party API synchronously on every prediction. The current incident instead turns on point-in-time correctness may require both event time and actual feature availability when ingestion latency matters.
D:That underlying principle is directed at a different advanced feature store concepts situation: An online behavioral feature counter consumes a streaming source but does not retain checkpoint state after worker restarts. The current incident instead turns on point-in-time correctness may require both event time and actual feature availability when ingestion latency matters.
Question 951
Consider the following fictional Databricks production case: A team logs a score labeled accuracy with values between 0 and 100 in some runs and between 0 and 1 in others, then ranks runs automatically. Which observation most directly tests the suspected failure boundary?
- Replay interleaved synthetic customer requests and inspect whether cache keys and outputs remain isolated.
- Inspect metric computation code and raw values across runs for incompatible scale conventions.
- Correlate worker logs and metric histories by run ID to expose simultaneous writes to the same run.
- Compare actual serving request schema, feature lookup capabilities and the model's predict signature.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Inspect metric computation code and raw values across runs for incompatible scale conventions. The underlying technical issue is that logged metric names require consistent definitions and units before their numerical values can be compared.
Incorrect Answers
A:That diagnostic evidence is directed at a different advanced mlflow usage situation: A fraud PyFunc uses a global mutable feature cache initialized by the first request and mixes data between subsequent users. For the stated incident, the decisive observation is inspect metric computation code and raw values across runs for incompatible scale conventions.
C:That diagnostic evidence is directed at a different advanced mlflow usage situation: A training orchestrator starts parallel child fits but all workers accidentally log into one active MLflow run and overwrite each other's metrics. For the stated incident, the decisive observation is inspect metric computation code and raw values across runs for incompatible scale conventions.
D:That diagnostic evidence is directed at a different advanced mlflow usage situation: A real-time churn endpoint accepts customer IDs and must fetch current allowed account features, but a custom model class instead expects the caller to supply the entire internal training row. For the stated incident, the decisive observation is inspect metric computation code and raw values across runs for incompatible scale conventions.
Question 952
A production ML review contains this situation: An ML operator promotes a new model version but its automated rollback points to a model that was deleted from the endpoint configuration. Which statement best explains the failure mode?
- Rare-outcome evaluation requires sufficient exposure, observation time and mature labels before concluding parity.
- Random traffic splitting alone does not guarantee policy-based tenant or regional routing.
- An actionable rollback plan needs a retained deployable champion artifact and a tested restoration procedure.
- Rollout safety needs operational latency and memory gates alongside predictive quality metrics.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. An actionable rollback plan needs a retained deployable champion artifact and a tested restoration procedure. To test it, inspect previous model artifact availability, endpoint configuration snapshots and rollback permissions.
Incorrect Answers
A:That underlying principle is directed at a different deployment strategies situation: A candidate fraud model has near-identical average error to the champion, but an organization wants to compare rare fraudulent transactions during a brief canary. The current incident instead turns on an actionable rollback plan needs a retained deployable champion artifact and a tested restoration procedure.
B:That underlying principle is directed at a different deployment strategies situation: A model release requires all clients from a restricted region to keep the old version, but a simple percentage split sends some of those clients to the candidate. The current incident instead turns on an actionable rollback plan needs a retained deployable champion artifact and a tested restoration procedure.
D:That underlying principle is directed at a different deployment strategies situation: A new recommendation model consumes three times as much memory and the team plans a canary based solely on prediction accuracy, ignoring serving capacity. The current incident instead turns on an actionable rollback plan needs a retained deployable champion artifact and a tested restoration procedure.
Question 953
Consider the following fictional Databricks production case: An ML engineer logs a calibration plot as a string metric, then downstream reviewers cannot inspect the visualization or its input version. What consequence most specifically follows from leaving the issue unchanged?
- Metrics and model artifacts are incorrectly attributed, making the champion selection unauditable.
- Clients cannot satisfy the model input requirements, so live predictions fail or leak internal fields.
- A trained model works locally but cannot initialize outside the developer's session.
- Important diagnostic evidence is absent from model review despite a reported scalar score.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. Important diagnostic evidence is absent from model review despite a reported scalar score. The causal constraint is: Complex objects such as plots and reports belong in versioned MLflow artifacts rather than scalar metrics.
Incorrect Answers
A:That direct consequence is directed at a different advanced mlflow usage situation: A training orchestrator starts parallel child fits but all workers accidentally log into one active MLflow run and overwrite each other's metrics. The more immediate risk here is important diagnostic evidence is absent from model review despite a reported scalar score.
B:That direct consequence is directed at a different advanced mlflow usage situation: A real-time churn endpoint accepts customer IDs and must fetch current allowed account features, but a custom model class instead expects the caller to supply the entire internal training row. The more immediate risk here is important diagnostic evidence is absent from model review despite a reported scalar score.
C:That direct consequence is directed at a different advanced mlflow usage situation: A team logs a custom Python model that imports a package installed only in the author's notebook, and the serving endpoint fails when loading the artifact. The more immediate risk here is important diagnostic evidence is absent from model review despite a reported scalar score.
Question 954
A production ML review contains this situation: An operations model uses a SparkML estimator that itself distributes data, yet the team launches dozens of concurrent full-cluster fits on the same fixed small cluster. Which diagnostic check provides the strongest direct evidence?
- Compare executor saturation, shuffle backlog and trial overlap for individual versus concurrent fits.
- Measure trial startup, scheduling, serialization and actual objective runtime separately.
- Compare driver utilization, active executor CPU, task backlog and partition parallelism.
- Inspect worker weight hashes, gradient aggregation checkpoints and update barriers.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Compare executor saturation, shuffle backlog and trial overlap for individual versus concurrent fits. The underlying technical issue is that parallel outer trials of already distributed estimators can contend for the same workers and increase overall elapsed time.
Incorrect Answers
B:That diagnostic evidence is directed at a different scaling and tuning situation: A researcher distributes 500 tiny one-second trials across many Ray workers and sees coordination cost exceed compute time. For the stated incident, the decisive observation is compare executor saturation, shuffle backlog and trial overlap for individual versus concurrent fits.
C:That diagnostic evidence is directed at a different scaling and tuning situation: A feature-engineering job is CPU-bound on hundreds of independent daily partitions and a team upgrades only the driver to a much larger machine. For the stated incident, the decisive observation is compare executor saturation, shuffle backlog and trial overlap for individual versus concurrent fits.
D:That diagnostic evidence is directed at a different scaling and tuning situation: A training team averages gradients from multiple workers but lets each worker use a different untracked version of the model. For the stated incident, the decisive observation is compare executor saturation, shuffle backlog and trial overlap for individual versus concurrent fits.
Question 955
Consider the following fictional Databricks production case: A serving endpoint appears healthy but nobody records model version, request identifier or prediction value, so a drift incident cannot be reconstructed. Which implementation is most appropriate for the stated incident?
- Join labels to their originating predictions using stable IDs and a defined maturity window.
- Use an appropriate time-series event table/profile if hourly history is truly required, keeping snapshot monitoring for current state.
- Align baseline and current cohort definitions before computing temporal drift.
- Log permitted request/prediction metadata to a governed inference table with retention and redaction controls.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Log permitted request/prediction metadata to a governed inference table with retention and redaction controls. The issue matters because investigators cannot tell whether a particular model version produced the anomalous scores.
Incorrect Answers
A:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. The relevant corrective action here is log permitted request/prediction metadata to a governed inference table with retention and redaction controls.
B:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A retail profile is created on a static materialized daily inventory table; a team expects its initial snapshot to show each prior hour as a separate trend. The relevant corrective action here is log permitted request/prediction metadata to a governed inference table with retention and redaction controls.
C:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A customer-risk monitoring job compares an unfiltered current high-risk segment to an all-customer historical baseline and interprets the difference as temporal drift. The relevant corrective action here is log permitted request/prediction metadata to a governed inference table with retention and redaction controls.
Question 956
A production ML review contains this situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. Which diagnostic check provides the strongest direct evidence?
- Compare original search ID, resumed run tags and candidate-set coverage after restoration.
- Inspect the run's metric keys versus artifact store and recover the expected plot file.
- Compare training feature timestamp definitions with live feature retrieval windows on a matched historical request.
- Compare actual serving request schema, feature lookup capabilities and the model's predict signature.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Compare original search ID, resumed run tags and candidate-set coverage after restoration. The underlying technical issue is that a resumed search needs stable experiment and parent-child provenance across partial reruns to support comparison.
Incorrect Answers
B:That diagnostic evidence is directed at a different advanced mlflow usage situation: An ML engineer logs a calibration plot as a string metric, then downstream reviewers cannot inspect the visualization or its input version. For the stated incident, the decisive observation is compare original search ID, resumed run tags and candidate-set coverage after restoration.
C:That diagnostic evidence is directed at a different advanced mlflow usage situation: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. For the stated incident, the decisive observation is compare original search ID, resumed run tags and candidate-set coverage after restoration.
D:That diagnostic evidence is directed at a different advanced mlflow usage situation: A real-time churn endpoint accepts customer IDs and must fetch current allowed account features, but a custom model class instead expects the caller to supply the entire internal training row. For the stated incident, the decisive observation is compare original search ID, resumed run tags and candidate-set coverage after restoration.
Question 957
Consider the following fictional Databricks production case: A model's prediction error appears to spike at 2 a.m. local time but the monitoring table groups timestamps in UTC for a globally distributed region. What underlying principle determines the most appropriate interpretation?
- Monitoring alert limits need controlled baseline revisions and documented operational validity.
- Time-of-day slices require explicit timezone interpretation and region-specific boundaries.
- Aggregate performance can conceal failure in important subpopulations and lifecycle stages.
- Comparing consecutive windows and comparing to a stable reference answer different drift questions.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Time-of-day slices require explicit timezone interpretation and region-specific boundaries. To test it, inspect source timestamps and selected monitoring timezone for affected geographic cohorts.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. The current incident instead turns on time-of-day slices require explicit timezone interpretation and region-specific boundaries.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. The current incident instead turns on time-of-day slices require explicit timezone interpretation and region-specific boundaries.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. The current incident instead turns on time-of-day slices require explicit timezone interpretation and region-specific boundaries.
Question 958
A production ML review contains this situation: A custom model endpoint meets mean response time of 75 milliseconds but P99 latency reaches 1.8 seconds during peak load. Which check shows that this particular problem has been resolved?
- An injected shift remains visible against the preserved baseline after repeated monitor runs.
- A sustained load test meets the approved P95/P99 objectives rather than just average response time.
- All denominator boundary fixtures produce defined permitted outputs and appropriate alert behavior.
- A synthetic region-only failure triggers the scoped alert without misclassifying healthy regions.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. A sustained load test meets the approved P95/P99 objectives rather than just average response time. It confirms whether the targeted action succeeded: monitor percentile latency and address verified bottlenecks under realistic peak traffic.
Incorrect Answers
A:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A baseline table is overwritten every night with the current day's data before a drift calculation, making the comparison always look unchanged. Here the actual success criterion is a sustained load test meets the approved P95/P99 objectives rather than just average response time.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A custom drift metric is expressed as today-minus-yesterday percentage, but yesterday's eligible count can be zero. Here the actual success criterion is a sustained load test meets the approved P95/P99 objectives rather than just average response time.
D:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. Here the actual success criterion is a sustained load test meets the approved P95/P99 objectives rather than just average response time.
Question 959
A production ML review contains this situation: A model is promoted to production with its code unchanged but a different unpinned training library version is installed, altering prediction preprocessing. Which diagnostic check provides the strongest direct evidence?
- Inventory model run IDs, registered versions, job configuration and deployment references.
- Trace a sample production prediction backward through model, feature snapshot and logged run metadata.
- Compare the logged training environment, lockfile and production model-load dependency versions.
- Inspect artifact provenance, ownership, version history and permitted access to the proposed model storage.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Compare the logged training environment, lockfile and production model-load dependency versions. The underlying technical issue is that environment transitions must preserve or explicitly test library and runtime dependencies as well as source code.
Incorrect Answers
A:That diagnostic evidence is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. For the stated incident, the decisive observation is compare the logged training environment, lockfile and production model-load dependency versions.
B:That diagnostic evidence is directed at a different model lifecycle management situation: An organization has reliable model serving but never archives feature definitions or experiment parameters and cannot reproduce predictions after drift. For the stated incident, the decisive observation is compare the logged training environment, lockfile and production model-load dependency versions.
D:That diagnostic evidence is directed at a different model lifecycle management situation: An engineer stores model weights in a personal workspace folder and treats that path as a production artifact registry with audit history. For the stated incident, the decisive observation is compare the logged training environment, lockfile and production model-load dependency versions.
Question 960
A production ML review contains this situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. What is the most material immediate risk in this situation?
- The monitoring pipeline either floods the SOC or ignores new material shifts.
- Routine weekend traffic launches unnecessary retraining and distracts engineers.
- The metric fails or silently excludes malformed observations, distorting the reported trend.
- Errors are assigned to the wrong predictions and reported performance becomes meaningless.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. Routine weekend traffic launches unnecessary retraining and distracts engineers. The causal constraint is: Reference populations should match relevant seasonality and business context when interpreting drift.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. The more immediate risk here is routine weekend traffic launches unnecessary retraining and distracts engineers.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An ML owner adds a custom metric computed from the average of a string-typed currency column without validating conversion. The more immediate risk here is routine weekend traffic launches unnecessary retraining and distracts engineers.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. The more immediate risk here is routine weekend traffic launches unnecessary retraining and distracts engineers.