Databricks Certified Machine Learning Professional Practice Test 5
Question 40
A production ML review contains this situation: A monitor is pointed at a Unity Catalog table but the automation identity lacks the privileges needed to generate profile outputs. Which check shows that this particular problem has been resolved?
- A scheduled profile run writes metrics under the approved identity without unauthorized table access.
- Weekend fixtures match appropriate seasonality while atypical weekend changes still alert.
- A negligible large-sample fixture is suppressed while a meaningful effect-size shift still escalates.
- The profile generates expected current-table completeness and distribution statistics without inventing prediction metadata.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. A scheduled profile run writes metrics under the approved identity without unauthorized table access. It confirms whether the targeted action succeeded: grant only the documented required monitoring permissions and configure approved metric destinations.
Incorrect Answers
B:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. Here the actual success criterion is a scheduled profile run writes metrics under the approved identity without unauthorized table access.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A monitoring dashboard reports a statistically significant tiny difference after one million observations, and the team immediately treats it as operationally material drift. Here the actual success criterion is a scheduled profile run writes metrics under the approved identity without unauthorized table access.
D:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: An analyst wants to inspect today’s static customer table for missing values and profile changes but chooses an inference monitor requiring model IDs and predictions. Here the actual success criterion is a scheduled profile run writes metrics under the approved identity without unauthorized table access.
Question 41
A production ML review contains this situation: An endpoint update request succeeds but the newly configured model version is still loading; monitoring marks the release complete immediately. What is the most material immediate risk in this situation?
- A premature full rollout promotes a model before enough rare fraud outcomes can be observed.
- Rebuilding the old model under outage pressure extends an avoidable recovery window.
- A full immediate cutover could expose every booking to an untested model defect.
- Production traffic reaches a model not yet loaded or compatible with runtime dependencies.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. Production traffic reaches a model not yet loaded or compatible with runtime dependencies. The causal constraint is: Model Serving configuration acceptance is not equivalent to endpoint/model readiness for user traffic.
Incorrect Answers
A:That direct consequence is directed at a different deployment strategies situation: A candidate fraud model has near-identical average error to the champion, but an organization wants to compare rare fraudulent transactions during a brief canary. The more immediate risk here is production traffic reaches a model not yet loaded or compatible with runtime dependencies.
B:That direct consequence is directed at a different deployment strategies situation: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. The more immediate risk here is production traffic reaches a model not yet loaded or compatible with runtime dependencies.
C:That direct consequence is directed at a different deployment strategies situation: A travel site serves 30,000 bookings per minute and wants to evaluate a new fraud model while limiting customer impact during its first hour. The more immediate risk here is production traffic reaches a model not yet loaded or compatible with runtime dependencies.
Question 42
The engineering team encounters the following issue: A monitor owner writes a custom metric that logs unaggregated account-level health data in a broadly shared results table. What investigation would most effectively isolate this behavior?
- Compare outcome error and cost weighted exposure in the high-value shipment cohort.
- Review the metric query's output grain, fields and result-table access against data classification.
- Review rolling-window statistics alongside a fixed approved baseline for the same cohort.
- Inspect inference table version IDs and label-matched errors before and after traffic shift.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Review the metric query's output grain, fields and result-table access against data classification. The underlying technical issue is that custom monitoring outputs should preserve necessary aggregate signals without exposing unnecessary entity-level sensitive data.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An ML model performs well on routine orders but has a severe error spike on high-value shipments that comprise just 2% of volume. For the stated incident, the decisive observation is review the metric query's output grain, fields and result-table access against data classification.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. For the stated incident, the decisive observation is review the metric query's output grain, fields and result-table access against data classification.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. For the stated incident, the decisive observation is review the metric query's output grain, fields and result-table access against data classification.
Question 43
The engineering team encounters the following issue: An integration caches predictions without recording the model version and returns yesterday's values after a controlled rollback. What corrective action should the engineer prefer?
- Key or invalidate caches based on the approved model-version contract and required freshness.
- Poll or otherwise verify documented readiness, then perform a permitted smoke prediction before marking the release ready.
- Implement explicit schema validation and approved default/missing-value treatment inside the custom model.
- Remove embedded secrets and configure an approved scoped secret retrieval mechanism for the serving process.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Key or invalidate caches based on the approved model-version contract and required freshness. The issue matters because stale candidate predictions persist after traffic has returned to the champion.
Incorrect Answers
B:That corrective action is directed at a different custom model serving situation: A Databricks SDK deployment script sends a model-version update but does not wait for the endpoint to reach a usable ready state before announcing success. The relevant corrective action here is key or invalidate caches based on the approved model-version contract and required freshness.
C:That corrective action is directed at a different custom model serving situation: A custom PyFunc's predict method assumes every request contains a numeric feature that is optional under the documented inference schema. The relevant corrective action here is key or invalidate caches based on the approved model-version contract and required freshness.
D:That corrective action is directed at a different custom model serving situation: An engineer logs a customer-specific API token inside the pickled PythonModel object before registering it to Unity Catalog. The relevant corrective action here is key or invalidate caches based on the approved model-version contract and required freshness.
Question 44
A production ML review contains this situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. Which check shows that this particular problem has been resolved?
- A fixture with two model versions produces identifiable versioned prediction metrics when labels are available.
- A safe synthetic threshold crossing reaches the correct team with the monitored metric and window.
- A cohort with large cumulative distance from its training baseline raises a distinct signal.
- Precision and recall populate only for complete labeled cohorts, while unlabeled drift remains separately reported.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. A safe synthetic threshold crossing reaches the correct team with the monitored metric and window. It confirms whether the targeted action succeeded: configure a documented threshold with the responsible on-call team and a tested delivery route.
Incorrect Answers
A:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A churn model logs requests, predictions, model versions and later ground-truth labels, but monitoring is configured as a generic feature snapshot. Here the actual success criterion is a safe synthetic threshold crossing reaches the correct team with the monitored metric and window.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A model trained on last year's customer mix is monitored only against yesterday's already shifted cohort, so its original training-population mismatch is hidden. Here the actual success criterion is a safe synthetic threshold crossing reaches the correct team with the monitored metric and window.
D:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A model's inference table records output probabilities but omits ground-truth labels, yet reviewers expect immediate precision and recall metrics. Here the actual success criterion is a safe synthetic threshold crossing reaches the correct team with the monitored metric and window.
Question 45
In an enterprise ML workflow, the reported issue is: A retailer trains a demand classifier on 180 million Delta rows already partitioned across a Spark cluster, but an analyst attempts to convert the entire training set to a local pandas object. Which change addresses the real cause without undermining other requirements?
- Hold out a final test set before tuning and score the selected pipeline exactly once after selection.
- Apply VectorAssembler to produce the features vector before fitting the chosen learning algorithm.
- Deploy the fitted preprocessing and estimator together as one versioned pipeline artifact.
- Keep the eligible features as distributed Spark DataFrames and fit a suitable SparkML pipeline on the cluster.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Keep the eligible features as distributed Spark DataFrames and fit a suitable SparkML pipeline on the cluster. The issue matters because the single driver can exhaust memory before training begins despite ample executor capacity.
Incorrect Answers
A:That corrective action is directed at a different using spark ml situation: After choosing the best parameter map, a data scientist reports the same cross-validation score as an independent final test result without any untouched data. The relevant corrective action here is keep the eligible features as distributed Spark DataFrames and fit a suitable SparkML pipeline on the cluster.
B:That corrective action is directed at a different using spark ml situation: A team wants to convert several numeric inputs into one features vector but treats VectorAssembler as a clustering Estimator that should output predictions. The relevant corrective action here is keep the eligible features as distributed Spark DataFrames and fit a suitable SparkML pipeline on the cluster.
C:That corrective action is directed at a different using spark ml situation: A batch scoring service loads a classifier version from last month while using today's new feature indexer, silently changing categorical positions. The relevant corrective action here is keep the eligible features as distributed Spark DataFrames and fit a suitable SparkML pipeline on the cluster.
Question 46
In an enterprise ML workflow, the reported issue is: An ML model performs well on routine orders but has a severe error spike on high-value shipments that comprise just 2% of volume. How should the engineer validate the correction?
- A privacy scan confirms only approved aggregate values appear in the shared metric output.
- Injected 503 responses trigger the service-error alarm and disappear after controlled remediation.
- Known labeled fixtures join one-to-one with their source predictions and yield the expected error rate.
- A high-value-only misprediction fixture increases the appropriate slice metric and escalation priority.
Explanation
Correct Answer
D:This verification measures the required result rather than just process liveness. A high-value-only misprediction fixture increases the appropriate slice metric and escalation priority. It confirms whether the targeted action succeeded: monitor an authorized high-impact transaction slice with enough observations and tailored severity.
Incorrect Answers
A:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A monitor owner writes a custom metric that logs unaggregated account-level health data in a broadly shared results table. Here the actual success criterion is a high-value-only misprediction fixture increases the appropriate slice metric and escalation priority.
B:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A prediction endpoint returns HTTP 503 for intermittent bursts while the model's offline accuracy remains unchanged. Here the actual success criterion is a high-value-only misprediction fixture increases the appropriate slice metric and escalation priority.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. Here the actual success criterion is a high-value-only misprediction fixture increases the appropriate slice metric and escalation priority.
Question 47
A production ML review contains this situation: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. What is the most material immediate risk in this situation?
- A costly driver upgrade leaves worker-side backlog and deadline misses unchanged.
- A trial's metrics and model artifacts overwrite evidence from another configuration.
- Multiple trials contend for the same accelerator and fail with memory exhaustion.
- Unnecessary model partitioning adds network latency with little memory benefit.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. Multiple trials contend for the same accelerator and fail with memory exhaustion. The causal constraint is: Ray resource declarations must match actual device use to avoid GPU oversubscription.
Incorrect Answers
A:That direct consequence is directed at a different scaling and tuning situation: A feature-engineering job is CPU-bound on hundreds of independent daily partitions and a team upgrades only the driver to a much larger machine. The more immediate risk here is multiple trials contend for the same accelerator and fail with memory exhaustion.
B:That direct consequence is directed at a different scaling and tuning situation: An Optuna optimization reuses the same MLflow run ID for concurrent trials, causing overlapping parameter values and artifacts. The more immediate risk here is multiple trials contend for the same accelerator and fail with memory exhaustion.
D:That direct consequence is directed at a different scaling and tuning situation: A tabular classifier fits comfortably on one GPU and 100 million independent examples are available, but a team proposes splitting the small model's layers across hosts. The more immediate risk here is multiple trials contend for the same accelerator and fail with memory exhaustion.
Question 48
The engineering team encounters the following issue: A team trains on curated development features but deploys the same job with credentials that accidentally read unapproved production source tables. What investigation would most effectively isolate this behavior?
- Trace a sample production prediction backward through model, feature snapshot and logged run metadata.
- Compare experiment write rights with registered model and serving endpoint permissions for test identities.
- Inspect environment-specific data-source references and effective service-identity permissions.
- Inventory model run IDs, registered versions, job configuration and deployment references.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect environment-specific data-source references and effective service-identity permissions. The underlying technical issue is that release environment promotion must separate code portability from data-governance and identity bindings.
Incorrect Answers
A:That diagnostic evidence is directed at a different model lifecycle management situation: An organization has reliable model serving but never archives feature definitions or experiment parameters and cannot reproduce predictions after drift. For the stated incident, the decisive observation is inspect environment-specific data-source references and effective service-identity permissions.
B:That diagnostic evidence is directed at a different model lifecycle management situation: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. For the stated incident, the decisive observation is inspect environment-specific data-source references and effective service-identity permissions.
D:That diagnostic evidence is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. For the stated incident, the decisive observation is inspect environment-specific data-source references and effective service-identity permissions.
Question 49
A production ML review contains this situation: An inference pipeline sends new categorical labels unseen during training to StringIndexerModel with error handling configured to reject unknown categories. What is the most material immediate risk in this situation?
- The supposedly unseen holdout can affect model inputs and inflate reported generalization.
- A rare new category can fail an entire scoring batch rather than produce a controlled result.
- A per-row driver bottleneck creates lag and breaks recovery guarantees.
- The selected hyperparameter reflects an unrepeatable or misreported single holdout trial.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. A rare new category can fail an entire scoring batch rather than produce a controlled result. The causal constraint is: Training-fitted preprocessors may reject unseen labels; inference handling must be planned and tested explicitly.
Incorrect Answers
A:That direct consequence is directed at a different using spark ml situation: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. The more immediate risk here is a rare new category can fail an entire scoring batch rather than produce a controlled result.
C:That direct consequence is directed at a different using spark ml situation: An energy plant needs one model prediction for every new sensor micro-batch but proposes a synchronous REST call per row from a driver collect loop. The more immediate risk here is a rare new category can fail an entire scoring batch rather than produce a controlled result.
D:That direct consequence is directed at a different using spark ml situation: An ML engineer manually compares two logistic-regression regularization values on a single holdout but has no reusable SparkML parameter-search plan. The more immediate risk here is a rare new category can fail an entire scoring batch rather than produce a controlled result.
Question 50
In an enterprise ML workflow, the reported issue is: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. Before modifying the pipeline, which evidence should be collected first?
- Review rolling-window statistics alongside a fixed approved baseline for the same cohort.
- Compare feature drift trends with label-matured error rates and calibration by model version.
- Compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
- Compare inference IDs and prediction times against actual confirmed label availability.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Review rolling-window statistics alongside a fixed approved baseline for the same cohort. The underlying technical issue is that comparing consecutive windows and comparing to a stable reference answer different drift questions.
Incorrect Answers
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A risk service measures feature drift but neglects observed-label performance and continues serving an accurate-looking yet ineffective model. For the stated incident, the decisive observation is review rolling-window statistics alongside a fixed approved baseline for the same cohort.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. For the stated incident, the decisive observation is review rolling-window statistics alongside a fixed approved baseline for the same cohort.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A fraud model predicts instantly but labels arrive 21 days later; an inference dashboard treats unlabelled transactions as negative examples. For the stated incident, the decisive observation is review rolling-window statistics alongside a fixed approved baseline for the same cohort.