Databricks Certified Machine Learning Professional Practice Test 47
Question 460
Review this production model incident: After choosing the best parameter map, a data scientist reports the same cross-validation score as an independent final test result without any untouched data. Which failure risk follows most closely from the unresolved behavior?
- Reported performance is optimistically biased because the evaluation influenced configuration choice.
- The supposedly unseen holdout can affect model inputs and inflate reported generalization.
- No training-derived category mapping exists for the requested transformation.
- A per-row driver bottleneck creates lag and breaks recovery guarantees.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. Reported performance is optimistically biased because the evaluation influenced configuration choice. The causal constraint is: Hyperparameter selection uses validation evidence; the final unbiased estimate requires an independent untouched test split.
Incorrect Answers
B:That direct consequence is directed at a different using spark ml situation: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. The more immediate risk here is reported performance is optimistically biased because the evaluation influenced configuration choice.
C:That direct consequence is directed at a different using spark ml situation: A new Spark developer calls transform on an unfitted StringIndexer instance and expects learned category labels to appear automatically. The more immediate risk here is reported performance is optimistically biased because the evaluation influenced configuration choice.
D:That direct consequence is directed at a different using spark ml situation: An energy plant needs one model prediction for every new sensor micro-batch but proposes a synchronous REST call per row from a driver collect loop. The more immediate risk here is reported performance is optimistically biased because the evaluation influenced configuration choice.
Question 461
A production ML review contains this situation: A payment classifier's on-demand feature division returns infinity for zero denominators and fails serialization in the serving endpoint. Which check shows that this particular problem has been resolved?
- A failover replay produces the same feature totals as an uninterrupted control run.
- Renaming a display label does not alter the entity's online lookup result.
- A past labeled event can still retrieve the correct historical feature state.
- Boundary-value fixtures yield schema-valid outputs with expected missing/invalid handling.
Explanation
Correct Answer
D:This verification measures the required result rather than just process liveness. Boundary-value fixtures yield schema-valid outputs with expected missing/invalid handling. It confirms whether the targeted action succeeded: validate inputs and implement deterministic finite-value or missing-value behavior approved for the model.
Incorrect Answers
A:That acceptance test is directed at a different advanced feature store concepts situation: An online behavioral feature counter consumes a streaming source but does not retain checkpoint state after worker restarts. Here the actual success criterion is boundary-value fixtures yield schema-valid outputs with expected missing/invalid handling.
B:That acceptance test is directed at a different advanced feature store concepts situation: A low-latency feature table uses external customer names as lookup keys; names change and are not unique. Here the actual success criterion is boundary-value fixtures yield schema-valid outputs with expected missing/invalid handling.
C:That acceptance test is directed at a different advanced feature store concepts situation: A model's feature generation step overwrites all feature-table history daily even though time-aware training requires earlier snapshots. Here the actual success criterion is boundary-value fixtures yield schema-valid outputs with expected missing/invalid handling.
Question 462
In an enterprise ML workflow, the reported issue is: A release pipeline runs development unit tests against a different dependency image from the one deployed to production. How should the engineer validate the correction?
- Multiple clean sessions produce the same approved encoded indices for every fixture category.
- The production image digest matches the tested release and reproduces expected fixture outputs.
- A serving query and endpoint configuration both identify the authorized promoted model version.
- Normal production execution does not run tests, while CI executes the full isolated suite.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. The production image digest matches the tested release and reproduces expected fixture outputs. It confirms whether the targeted action succeeded: run integration and acceptance tests on the same version-pinned build that will be promoted.
Incorrect Answers
A:That acceptance test is directed at a different validation testing situation: A notebook's label encoder produces a different category order depending on an unordered Python set, but tests assert only output length. Here the actual success criterion is the production image digest matches the tested release and reproduces expected fixture outputs.
C:That acceptance test is directed at a different validation testing situation: A model version is registered but the deployment job points to an old alias after promotion and continues serving last week's model. Here the actual success criterion is the production image digest matches the tested release and reproduces expected fixture outputs.
D:That acceptance test is directed at a different validation testing situation: An ML project stores test files directly in each production notebook and execution of a deployed notebook inadvertently reruns expensive tests. Here the actual success criterion is the production image digest matches the tested release and reproduces expected fixture outputs.
Question 463
Consider the following fictional Databricks production case: A fraud PyFunc uses a global mutable feature cache initialized by the first request and mixes data between subsequent users. What consequence most specifically follows from leaving the issue unchanged?
- One customer's features can contaminate another customer's live score.
- Tracking artifacts create unauthorized exposure paths for customers' personal data.
- The apparent best F1 cannot be reproduced because its hyperparameters were never captured.
- The deployed artifact cannot be tied to the training configuration that earned the winning metric.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. One customer's features can contaminate another customer's live score. The causal constraint is: Request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
Incorrect Answers
B:That direct consequence is directed at a different advanced mlflow usage situation: A regulated analytics team uploads full customer identifiers inside MLflow model artifacts during troubleshooting. The more immediate risk here is one customer's features can contaminate another customer's live score.
C:That direct consequence is directed at a different advanced mlflow usage situation: A training notebook reports F1 for several models but only logs the final model's parameters in MLflow, leaving metrics and hyperparameters disconnected. The more immediate risk here is one customer's features can contaminate another customer's live score.
D:That direct consequence is directed at a different advanced mlflow usage situation: A data scientist selects the lowest validation loss from nested runs but reports only the parent run ID as the model-producing run. The more immediate risk here is one customer's features can contaminate another customer's live score.
Question 464
A production ML review contains this situation: An ML owner adds a custom metric computed from the average of a string-typed currency column without validating conversion. What is the most material immediate risk in this situation?
- Real drift is erased because the comparator silently changes with the observed sample.
- A regression in v8 is masked by higher-volume predictions from v7.
- The deployment wastes compute while inference delays and timeouts continue increasing.
- The metric fails or silently excludes malformed observations, distorting the reported trend.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. The metric fails or silently excludes malformed observations, distorting the reported trend. The causal constraint is: Custom monitor metrics need defined input types and missing/invalid-value semantics.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A baseline table is overwritten every night with the current day's data before a drift calculation, making the comparison always look unchanged. The more immediate risk here is the metric fails or silently excludes malformed observations, distorting the reported trend.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. The more immediate risk here is the metric fails or silently excludes malformed observations, distorting the reported trend.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A production endpoint scales CPU instances up when traffic rises, but request queues and P95 response time keep growing despite low CPU usage. The more immediate risk here is the metric fails or silently excludes malformed observations, distorting the reported trend.
Question 465
Review this production model incident: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. What is the central engineering constraint behind this behavior?
- Monitoring alert limits need controlled baseline revisions and documented operational validity.
- A custom fairness comparison needs matched labels, model predictions and governed subgroup definitions.
- Production input should sometimes be compared to a retained training reference, not only recent windows.
- Aggregate performance can conceal failure in important subpopulations and lifecycle stages.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Monitoring alert limits need controlled baseline revisions and documented operational validity. To test it, compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
Incorrect Answers
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A fairness reviewer needs the difference in false-negative rates between two approved cohorts, but the monitor computes an aggregate without cohort labels. The current incident instead turns on monitoring alert limits need controlled baseline revisions and documented operational validity.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A model trained on last year's customer mix is monitored only against yesterday's already shifted cohort, so its original training-population mismatch is hidden. The current incident instead turns on monitoring alert limits need controlled baseline revisions and documented operational validity.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. The current incident instead turns on monitoring alert limits need controlled baseline revisions and documented operational validity.
Question 466
In an enterprise ML workflow, the reported issue is: An inference pipeline sends new categorical labels unseen during training to StringIndexerModel with error handling configured to reject unknown categories. What technical distinction is essential to this investigation?
- Per-request low-latency inference calls for an appropriately sized online serving path, not batch-only scheduling.
- Pipeline stages execute in declared sequence and each estimator expects columns already produced upstream.
- VectorAssembler is a Transformer that creates a vector column, not a trainable predictor.
- Training-fitted preprocessors may reject unseen labels; inference handling must be planned and tested explicitly.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. Training-fitted preprocessors may reject unseen labels; inference handling must be planned and tested explicitly. To test it, inspect fitted category mappings, handleInvalid behavior and a sample newly introduced category.
Incorrect Answers
A:That underlying principle is directed at a different using spark ml situation: An online pricing service requires a personalized response within 70 milliseconds per request but its team schedules a full overnight batch job as the only inference route. The current incident instead turns on training-fitted preprocessors may reject unseen labels; inference handling must be planned and tested explicitly.
B:That underlying principle is directed at a different using spark ml situation: A pipeline engineer fits a vectorizer after an estimator in the stage list and then tries to use the estimator's prediction as the vectorizer input. The current incident instead turns on training-fitted preprocessors may reject unseen labels; inference handling must be planned and tested explicitly.
C:That underlying principle is directed at a different using spark ml situation: A team wants to convert several numeric inputs into one features vector but treats VectorAssembler as a clustering Estimator that should output predictions. The current incident instead turns on training-fitted preprocessors may reject unseen labels; inference handling must be planned and tested explicitly.
Question 467
A production ML review contains this situation: A feature stream loses half its source events and a retraining policy treats the resulting input shift as proof that the learned concept changed. What production-safe change best matches the evidence?
- Restore missing data and validate collection health before deciding whether a model retrain is warranted.
- Rank only eligible candidates that pass accuracy and operational SLA gates before promotion.
- Reject candidates that violate required safety segment thresholds even when aggregate F1 increases.
- Evaluate all eligible candidates against one controlled frozen benchmark and rank only comparable results.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Restore missing data and validate collection health before deciding whether a model retrain is warranted. The issue matters because training on incomplete observations worsens the model instead of correcting the ingestion outage.
Incorrect Answers
B:That corrective action is directed at a different automated retraining situation: An AutoML training job always chooses the model with the lowest training loss, even when the deployed service has a strict memory and P95 latency budget. The relevant corrective action here is restore missing data and validate collection health before deciding whether a model retrain is warranted.
C:That corrective action is directed at a different automated retraining situation: A new fraud classifier improves overall F1 but doubles false negatives for high-value payment accounts, and an automated policy chooses it on F1 alone. The relevant corrective action here is restore missing data and validate collection health before deciding whether a model retrain is warranted.
D:That corrective action is directed at a different automated retraining situation: A retraining pipeline ranks candidate models using metrics calculated on different dataset versions, then selects the numerically highest result. The relevant corrective action here is restore missing data and validate collection health before deciding whether a model retrain is warranted.
Question 468
In an enterprise ML workflow, the reported issue is: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. What likely operational or model-quality harm does this issue create?
- False optimism or pessimism distorts the model's apparent accuracy and promotion decisions.
- A small but important region remains unavailable while overall service metrics seem healthy.
- The metric fails or silently excludes malformed observations, distorting the reported trend.
- Routine weekend traffic launches unnecessary retraining and distracts engineers.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. A small but important region remains unavailable while overall service metrics seem healthy. The causal constraint is: Global thresholds can hide local critical slices; alerting should reflect operational segmentation where justified.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A fraud model predicts instantly but labels arrive 21 days later; an inference dashboard treats unlabelled transactions as negative examples. The more immediate risk here is a small but important region remains unavailable while overall service metrics seem healthy.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An ML owner adds a custom metric computed from the average of a string-typed currency column without validating conversion. The more immediate risk here is a small but important region remains unavailable while overall service metrics seem healthy.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. The more immediate risk here is a small but important region remains unavailable while overall service metrics seem healthy.
Question 469
Consider the following fictional Databricks production case: A research team captures only daily batch inference totals with no row-level timestamps but insists on per-minute live inference drift analysis. What consequence most specifically follows from leaving the issue unchanged?
- Missing historical windows are treated as evidence of zero incidents or zero data change.
- The monitor fabricates apparent minute-level precision from daily totals and misleads responders.
- Routine weekend traffic launches unnecessary retraining and distracts engineers.
- The dashboard reports yesterday's metrics as current and misses a real shift.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. The monitor fabricates apparent minute-level precision from daily totals and misleads responders. The causal constraint is: Monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A retail profile is created on a static materialized daily inventory table; a team expects its initial snapshot to show each prior hour as a separate trend. The more immediate risk here is the monitor fabricates apparent minute-level precision from daily totals and misleads responders.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. The more immediate risk here is the monitor fabricates apparent minute-level precision from daily totals and misleads responders.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A team has accurate inference logs but its monitoring job never refreshes computed drift metrics after a table schema change. The more immediate risk here is the monitor fabricates apparent minute-level precision from daily totals and misleads responders.
Question 470
A production ML review contains this situation: A multilingual prediction service must handle regional data residency, but the team copies a shared feature cache across regions without governance. What production-safe change best matches the evidence?
- Add bundle validation, permission checks and staging smoke tests before production promotion.
- Deploy region-appropriate governed feature and serving assets with controlled cross-region access.
- Define the supported model-serving and MLflow resources declaratively with validated environment targets.
- Migrate or retain the configuration only after verifying current supported fields and running an explicit bundle validation.
Explanation
Correct Answer
B:This corrects the relevant failure boundary instead of merely hiding the symptom. Deploy region-appropriate governed feature and serving assets with controlled cross-region access. The issue matters because sensitive records cross an unapproved residency boundary through convenience caching.
Incorrect Answers
A:That corrective action is directed at a different environment architectures situation: A continuous-delivery job deploys a notebook change through a bundle but skips validation of referenced experiment paths and job permissions. The relevant corrective action here is deploy region-appropriate governed feature and serving assets with controlled cross-region access.
C:That corrective action is directed at a different environment architectures situation: A model serving endpoint is created manually in the UI and every staging deployment differs subtly from production because no deployment definition is saved. The relevant corrective action here is deploy region-appropriate governed feature and serving assets with controlled cross-region access.
D:That corrective action is directed at a different environment architectures situation: An ML platform moves its configuration from older Databricks Asset Bundles terminology to newer Declarative Automation Bundles references and assumes every resource schema stayed identical. The relevant corrective action here is deploy region-appropriate governed feature and serving assets with controlled cross-region access.