Databricks Certified Machine Learning Professional Practice Test 73
Question 720
A production ML review contains this situation: A baseline table is overwritten every night with the current day's data before a drift calculation, making the comparison always look unchanged. Which diagnostic check provides the strongest direct evidence?
- Inspect inference table version IDs and label-matched errors before and after traffic shift.
- Inspect baseline table versions and the order of writes relative to metric computations.
- Compare high-risk event counts against all eligible scored records for several differing traffic volumes.
- Inspect received alert fields and the corresponding monitor query, model tag and comparison window.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Inspect baseline table versions and the order of writes relative to metric computations. The underlying technical issue is that a fixed baseline must be versioned and immutable for the intended comparison period.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. For the stated incident, the decisive observation is inspect baseline table versions and the order of writes relative to metric computations.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A monitoring team wants a business metric for the share of transactions exceeding a risk threshold but records only the raw count of high-risk transactions. For the stated incident, the decisive observation is inspect baseline table versions and the order of writes relative to metric computations.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A drift alert fires but its message contains only the word anomaly without dataset, model version or baseline window. For the stated incident, the decisive observation is inspect baseline table versions and the order of writes relative to metric computations.
Question 721
In an enterprise ML workflow, the reported issue is: A feature-refresh task writes rows with event time in local timezone while the training join assumes UTC; unit tests of each component pass. Which change addresses the real cause without undermining other requirements?
- Write explicit notebook unit tests for missingness and correct zero handling, then fix the transformation if needed.
- Keep reusable logic and tests in separate versioned units with explicit CI test stages.
- Add a full-path integration test with explicit time-zone and cutoff assertions.
- Add an environment-level integration test for table, model and serving input/output contracts.
Explanation
Correct Answer
C:This corrects the relevant failure boundary instead of merely hiding the symptom. Add a full-path integration test with explicit time-zone and cutoff assertions. The issue matters because a hidden offset causes training-serving skew and potential future-data leakage.
Incorrect Answers
A:That corrective action is directed at a different validation testing situation: A feature function maps a null purchase amount to zero without distinguishing missing observations, and a notebook unit test covers only positive amounts. The relevant corrective action here is add a full-path integration test with explicit time-zone and cutoff assertions.
B:That corrective action is directed at a different validation testing situation: An ML project stores test files directly in each production notebook and execution of a deployed notebook inadvertently reruns expensive tests. The relevant corrective action here is add a full-path integration test with explicit time-zone and cutoff assertions.
D:That corrective action is directed at a different validation testing situation: A team unit-tests each model component independently but its feature table and serving endpoint disagree on column names. The relevant corrective action here is add a full-path integration test with explicit time-zone and cutoff assertions.
Question 722
Review this production model incident: A retraining pipeline ranks candidate models using metrics calculated on different dataset versions, then selects the numerically highest result. Which failure risk follows most closely from the unresolved behavior?
- Normal weekly seasonality launches wasteful retraining runs and noisy alerts.
- A poorer or unsafe model reaches customers just because monitoring fired.
- A model wins because it was tested on easier data rather than because it performs better.
- Training on incomplete observations worsens the model instead of correcting the ingestion outage.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. A model wins because it was tested on easier data rather than because it performs better. The causal constraint is: Model candidates need evaluation on the same versioned holdout and metric definition for fair comparison.
Incorrect Answers
A:That direct consequence is directed at a different automated retraining situation: An automated retraining job triggers whenever a numerical feature mean shifts by 0.1, regardless of normal seasonal changes or sample size. The more immediate risk here is a model wins because it was tested on easier data rather than because it performs better.
B:That direct consequence is directed at a different automated retraining situation: An alert from a feature monitor automatically replaces the live model with a newly trained artifact without testing or approval. The more immediate risk here is a model wins because it was tested on easier data rather than because it performs better.
D:That direct consequence is directed at a different automated retraining situation: A feature stream loses half its source events and a retraining policy treats the resulting input shift as proof that the learned concept changed. The more immediate risk here is a model wins because it was tested on easier data rather than because it performs better.
Question 723
Review this production model incident: A risk service measures feature drift but neglects observed-label performance and continues serving an accurate-looking yet ineffective model. What end-to-end verification is most persuasive here?
- New row-level timestamped records support the intended monitoring windows without interpolation guesses.
- A labeled regression fixture raises the performance signal even when input features remain stable.
- A synthetic increase in missing values raises the intended data-quality signal at unchanged non-null values.
- A safe synthetic threshold crossing reaches the correct team with the monitored metric and window.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. A labeled regression fixture raises the performance signal even when input features remain stable. It confirms whether the targeted action succeeded: add delayed-ground-truth performance tracking alongside existing drift metrics and health signals.
Incorrect Answers
A:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A research team captures only daily batch inference totals with no row-level timestamps but insists on per-minute live inference drift analysis. Here the actual success criterion is a labeled regression fixture raises the performance signal even when input features remain stable.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A nightly scoring feed drops all null-feature rows before calculating drift, making missingness jump invisible. Here the actual success criterion is a labeled regression fixture raises the performance signal even when input features remain stable.
D:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. Here the actual success criterion is a labeled regression fixture raises the performance signal even when input features remain stable.
Question 724
Review this production model incident: An ML pipeline computes training features in one notebook and duplicates slightly different SQL in the online pipeline for the same entities. What is the central engineering constraint behind this behavior?
- Feature reuse requires a shared governed definition rather than divergent training and serving transformations.
- Fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency.
- On-demand feature serving must account for dependency latency and failure budgets within the endpoint SLA.
- A time-series feature table needs a documented timestamp key and an as-of lookup to avoid multiplying labels or attaching future values.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Feature reuse requires a shared governed definition rather than divergent training and serving transformations. To test it, diff the offline and live feature transformation definitions for the same synthetic entity inputs.
Incorrect Answers
B:That underlying principle is directed at a different advanced feature store concepts situation: An online behavioral feature counter consumes a streaming source but does not retain checkpoint state after worker restarts. The current incident instead turns on feature reuse requires a shared governed definition rather than divergent training and serving transformations.
C:That underlying principle is directed at a different advanced feature store concepts situation: A streaming recommender's request-time feature function calls a slow third-party API synchronously on every prediction. The current incident instead turns on feature reuse requires a shared governed definition rather than divergent training and serving transformations.
D:That underlying principle is directed at a different advanced feature store concepts situation: A training dataset combines a feature table with multiple timestamped rows per device using an equality join on device ID only. The current incident instead turns on feature reuse requires a shared governed definition rather than divergent training and serving transformations.
Question 725
The engineering team encounters the following issue: An endpoint receives a large request burst every morning and its monitoring alert reacts only to 5xx errors after autoscaling cannot keep up. What investigation would most effectively isolate this behavior?
- Inspect received alert fields and the corresponding monitor query, model tag and comparison window.
- Review rolling-window statistics alongside a fixed approved baseline for the same cohort.
- Compute recall for new and established accounts with matched mature labels and stable cohort definitions.
- Compare incoming request rate, provisioned concurrency and scaling lag before observed errors.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Compare incoming request rate, provisioned concurrency and scaling lag before observed errors. The underlying technical issue is that request-rate and capacity forecasts can provide earlier warning than error counts alone.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A drift alert fires but its message contains only the word anomaly without dataset, model version or baseline window. For the stated incident, the decisive observation is compare incoming request rate, provisioned concurrency and scaling lag before observed errors.
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. For the stated incident, the decisive observation is compare incoming request rate, provisioned concurrency and scaling lag before observed errors.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An account-risk model's fraud recall is stable overall but unexpectedly falls for new customers in their first 30 days. For the stated incident, the decisive observation is compare incoming request rate, provisioned concurrency and scaling lag before observed errors.
Question 726
The engineering team encounters the following issue: A fraud model predicts instantly but labels arrive 21 days later; an inference dashboard treats unlabelled transactions as negative examples. Which technical principle most directly explains the root of the issue?
- Time-of-day slices require explicit timezone interpretation and region-specific boundaries.
- An alert is operationally useful only with explicit routing, threshold, urgency and escalation ownership.
- Custom derived metrics need deterministic handling for undefined denominators and small samples.
- True recall and precision require mature observed labels, not fabricated negatives for pending outcomes.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. True recall and precision require mature observed labels, not fabricated negatives for pending outcomes. To test it, compare inference IDs and prediction times against actual confirmed label availability.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A model's prediction error appears to spike at 2 a.m. local time but the monitoring table groups timestamps in UTC for a globally distributed region. The current incident instead turns on true recall and precision require mature observed labels, not fabricated negatives for pending outcomes.
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. The current incident instead turns on true recall and precision require mature observed labels, not fabricated negatives for pending outcomes.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A custom drift metric is expressed as today-minus-yesterday percentage, but yesterday's eligible count can be zero. The current incident instead turns on true recall and precision require mature observed labels, not fabricated negatives for pending outcomes.
Question 727
In an enterprise ML workflow, the reported issue is: A team logs custom MLflow model weights but forgets to include required Python package versions in the model environment. What technical distinction is essential to this investigation?
- Serving inference payloads must conform to the endpoint and model's accepted request/response schema.
- Endpoint deployment requires controlled ownership and conflict-safe update/versioning practices.
- MLflow custom-model portability depends on the declared runtime dependencies as well as artifact bytes.
- Client integrations should parse the documented model output contract instead of assuming a scalar response.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. MLflow custom-model portability depends on the declared runtime dependencies as well as artifact bytes. To test it, compare training import versions and logged model environment metadata with serving import failures.
Incorrect Answers
A:That underlying principle is directed at a different custom model serving situation: A notebook uses the MLflow Deployments SDK to send a column-major payload to a custom model endpoint expecting a different documented dataframe schema. The current incident instead turns on mLflow custom-model portability depends on the declared runtime dependencies as well as artifact bytes.
B:That underlying principle is directed at a different custom model serving situation: A team edits a model endpoint through the UI while its CI pipeline concurrently applies an older configuration and reverts the intended traffic split. The current incident instead turns on mLflow custom-model portability depends on the declared runtime dependencies as well as artifact bytes.
D:That underlying principle is directed at a different custom model serving situation: A serving client calls the REST endpoint expecting one number but the custom PyFunc returns a structured list of prediction and confidence fields. The current incident instead turns on mLflow custom-model portability depends on the declared runtime dependencies as well as artifact bytes.
Question 728
A production ML review contains this situation: An ML pipeline computes training features in one notebook and duplicates slightly different SQL in the online pipeline for the same entities. What production-safe change best matches the evidence?
- Centralize the approved on-demand ratio transformation and its edge-case semantics.
- Centralize eligible feature definitions and reuse a controlled feature computation workflow where supported.
- Use stable checkpoints and an idempotent feature update design for replayed source events.
- Configure an appropriate supported online table and governed feature lookup for the endpoint.
Explanation
Correct Answer
B:This corrects the relevant failure boundary instead of merely hiding the symptom. Centralize eligible feature definitions and reuse a controlled feature computation workflow where supported. The issue matters because training-serving skew creates inconsistent values for identically named features.
Incorrect Answers
A:That corrective action is directed at a different advanced feature store concepts situation: A customer-risk feature needs a ratio of current debt to income, but online serving recomputes it differently from the training pipeline. The relevant corrective action here is centralize eligible feature definitions and reuse a controlled feature computation workflow where supported.
C:That corrective action is directed at a different advanced feature store concepts situation: An online behavioral feature counter consumes a streaming source but does not retain checkpoint state after worker restarts. The relevant corrective action here is centralize eligible feature definitions and reuse a controlled feature computation workflow where supported.
D:That corrective action is directed at a different advanced feature store concepts situation: A pricing endpoint queries a large offline Delta feature table for every user request, and P95 response time reaches several seconds. The relevant corrective action here is centralize eligible feature definitions and reuse a controlled feature computation workflow where supported.
Question 729
In an enterprise ML workflow, the reported issue is: A real-time churn endpoint accepts customer IDs and must fetch current allowed account features, but a custom model class instead expects the caller to supply the entire internal training row. What likely operational or model-quality harm does this issue create?
- Trial histories become detached from the actual optimization process and candidate provenance is lost.
- Clients cannot satisfy the model input requirements, so live predictions fail or leak internal fields.
- A model can win solely because its reported metric uses a different scale.
- Metrics and model artifacts are incorrectly attributed, making the champion selection unauditable.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. Clients cannot satisfy the model input requirements, so live predictions fail or leak internal fields. The causal constraint is: A custom MLflow PyFunc's input contract and online feature lookups must be designed for request-time availability and governance.
Incorrect Answers
A:That direct consequence is directed at a different advanced mlflow usage situation: An experiment tunes 40 candidates but logs each trial as an unrelated root MLflow run, so reviewers cannot reconstruct which search produced a winner. The more immediate risk here is clients cannot satisfy the model input requirements, so live predictions fail or leak internal fields.
C:That direct consequence is directed at a different advanced mlflow usage situation: A team logs a score labeled accuracy with values between 0 and 100 in some runs and between 0 and 1 in others, then ranks runs automatically. The more immediate risk here is clients cannot satisfy the model input requirements, so live predictions fail or leak internal fields.
D:That direct consequence is directed at a different advanced mlflow usage situation: A training orchestrator starts parallel child fits but all workers accidentally log into one active MLflow run and overwrite each other's metrics. The more immediate risk here is clients cannot satisfy the model input requirements, so live predictions fail or leak internal fields.
Question 730
A production ML review contains this situation: A feature refresh writes duplicate records for each entity and logical timestamp after retries, breaking unique lookups. Which statement best explains the failure mode?
- A time-series feature table needs a documented timestamp key and an as-of lookup to avoid multiplying labels or attaching future values.
- Idempotent feature pipeline writes need stable entity/time keys and safe retry semantics.
- On-demand feature definitions should apply consistent logic and missing/zero-denominator handling across contexts.
- Low-latency serving requires online feature access with a documented freshness and lookup contract.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Idempotent feature pipeline writes need stable entity/time keys and safe retry semantics. To test it, check entity/timestamp duplicate counts around job retries and writer transaction history.
Incorrect Answers
A:That underlying principle is directed at a different advanced feature store concepts situation: A training dataset combines a feature table with multiple timestamped rows per device using an equality join on device ID only. The current incident instead turns on idempotent feature pipeline writes need stable entity/time keys and safe retry semantics.
C:That underlying principle is directed at a different advanced feature store concepts situation: A customer-risk feature needs a ratio of current debt to income, but online serving recomputes it differently from the training pipeline. The current incident instead turns on idempotent feature pipeline writes need stable entity/time keys and safe retry semantics.
D:That underlying principle is directed at a different advanced feature store concepts situation: A pricing endpoint queries a large offline Delta feature table for every user request, and P95 response time reaches several seconds. The current incident instead turns on idempotent feature pipeline writes need stable entity/time keys and safe retry semantics.