Databricks Certified Machine Learning Professional Practice Test 26
Question 250
Review this production model incident: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Compare training feature timestamp definitions with live feature retrieval windows on a matched historical request.
- Correlate worker logs and metric histories by run ID to expose simultaneous writes to the same run.
- Inspect the run's metric keys versus artifact store and recover the expected plot file.
- Inspect parent_run_id tags, candidate run IDs and the experiment's tracked search configuration.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Compare training feature timestamp definitions with live feature retrieval windows on a matched historical request. The underlying technical issue is that the model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
Incorrect Answers
B:That diagnostic evidence is directed at a different advanced mlflow usage situation: A training orchestrator starts parallel child fits but all workers accidentally log into one active MLflow run and overwrite each other's metrics. For the stated incident, the decisive observation is compare training feature timestamp definitions with live feature retrieval windows on a matched historical request.
C:That diagnostic evidence is directed at a different advanced mlflow usage situation: An ML engineer logs a calibration plot as a string metric, then downstream reviewers cannot inspect the visualization or its input version. For the stated incident, the decisive observation is compare training feature timestamp definitions with live feature retrieval windows on a matched historical request.
D:That diagnostic evidence is directed at a different advanced mlflow usage situation: An experiment tunes 40 candidates but logs each trial as an unrelated root MLflow run, so reviewers cannot reconstruct which search produced a winner. For the stated incident, the decisive observation is compare training feature timestamp definitions with live feature retrieval windows on a matched historical request.
Question 251
In an enterprise ML workflow, the reported issue is: A model serving endpoint is created manually in the UI and every staging deployment differs subtly from production because no deployment definition is saved. Before modifying the pipeline, which evidence should be collected first?
- Inspect job ownership, compute policy and identity access after a simulated employee offboarding.
- Audit workspace bindings, job identities and catalog permissions for development workloads.
- Compare staging and production endpoint, experiment and model registry settings against source-controlled configuration.
- Map data classes, approved processing regions and cross-region feature access paths.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Compare staging and production endpoint, experiment and model registry settings against source-controlled configuration. The underlying technical issue is that declarative automation or asset-bundle definitions can version environment-specific ML assets and reproduce deployments.
Incorrect Answers
A:That diagnostic evidence is directed at a different environment architectures situation: A distributed retraining workflow uses ephemeral personal compute and fails when the original engineer's account is disabled. For the stated incident, the decisive observation is compare staging and production endpoint, experiment and model registry settings against source-controlled configuration.
B:That diagnostic evidence is directed at a different environment architectures situation: A company operates development and production models in one workspace under the same broad service account and frequently overwrites production tables during experiments. For the stated incident, the decisive observation is compare staging and production endpoint, experiment and model registry settings against source-controlled configuration.
D:That diagnostic evidence is directed at a different environment architectures situation: A multilingual prediction service must handle regional data residency, but the team copies a shared feature cache across regions without governance. For the stated incident, the decisive observation is compare staging and production endpoint, experiment and model registry settings against source-controlled configuration.
Question 252
A production ML review contains this situation: A per-city pandas UDF returns different prediction schemas for cities with absent categories, causing a distributed aggregation failure. Which statement best explains the failure mode?
- Grouped pandas functions must honor one declared consistent return schema even when groups vary.
- Horizontal replica count alone does not combine GPU memory for one model without a supported parallelization strategy.
- Concurrent study workers need a supported shared trial-storage design and correct coordination, not a fragile local-only backend.
- Ray actor/task orchestration can fit fine-grained stateful workloads that do not map naturally to relational Spark queries.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Grouped pandas functions must honor one declared consistent return schema even when groups vary. To test it, compare sample group outputs and null/type handling against the declared pandas function schema.
Incorrect Answers
B:That underlying principle is directed at a different scaling and tuning situation: An image model needs 90 GB accelerator memory for one training replica while each available GPU has 24 GB and engineers propose only adding more identical nodes. The current incident instead turns on grouped pandas functions must honor one declared consistent return schema even when groups vary.
C:That underlying principle is directed at a different scaling and tuning situation: A researcher parallelizes Optuna trials while sharing one writable SQLite file on a fragile network mount and observes duplicate reservations and storage locks. The current incident instead turns on grouped pandas functions must honor one declared consistent return schema even when groups vary.
D:That underlying principle is directed at a different scaling and tuning situation: A reinforcement-learning simulator requires stateful actors exchanging messages and irregular task graphs, but the team plans one Spark SQL groupBy per simulation step. The current incident instead turns on grouped pandas functions must honor one declared consistent return schema even when groups vary.
Question 253
Review this production model incident: A customer-risk monitoring job compares an unfiltered current high-risk segment to an all-customer historical baseline and interprets the difference as temporal drift. Which failure risk follows most closely from the unresolved behavior?
- Selection bias looks like model input deterioration even though each subgroup is stable.
- False optimism or pessimism distorts the model's apparent accuracy and promotion decisions.
- An apparent overnight model failure is only a grouping artifact and triggers costly false alerts.
- Traffic growth is mistaken for increasing risk because the raw count rises mechanically.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. Selection bias looks like model input deterioration even though each subgroup is stable. The causal constraint is: Drift claims require like-for-like cohort selection across current and reference populations.
Incorrect Answers
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A fraud model predicts instantly but labels arrive 21 days later; an inference dashboard treats unlabelled transactions as negative examples. The more immediate risk here is selection bias looks like model input deterioration even though each subgroup is stable.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A model's prediction error appears to spike at 2 a.m. local time but the monitoring table groups timestamps in UTC for a globally distributed region. The more immediate risk here is selection bias looks like model input deterioration even though each subgroup is stable.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A monitoring team wants a business metric for the share of transactions exceeding a risk threshold but records only the raw count of high-risk transactions. The more immediate risk here is selection bias looks like model input deterioration even though each subgroup is stable.
Question 254
Review this production model incident: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. What end-to-end verification is most persuasive here?
- Holdout-only categories no longer influence the training-fitted indexer and metrics are recomputed on untouched holdout records.
- Two sample sentences produce the expected token-array column from a direct transform.
- A newly emitted synthetic event is scored and appears downstream without restarting the job.
- The tracked run records each parameter map, validation metric and final chosen model before untouched test evaluation.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. Holdout-only categories no longer influence the training-fitted indexer and metrics are recomputed on untouched holdout records. It confirms whether the targeted action succeeded: split the raw observations first, then fit every learned preprocessing stage on training data only.
Incorrect Answers
B:That acceptance test is directed at a different using spark ml situation: A text pipeline repeatedly fits a Tokenizer stage even though tokenization simply converts strings into token arrays without learning a model. Here the actual success criterion is holdout-only categories no longer influence the training-fitted indexer and metrics are recomputed on untouched holdout records.
C:That acceptance test is directed at a different using spark ml situation: A streaming anomaly team uses a static read of a message directory once, believing the trained SparkML model will continuously produce scores for future arriving events. Here the actual success criterion is holdout-only categories no longer influence the training-fitted indexer and metrics are recomputed on untouched holdout records.
D:That acceptance test is directed at a different using spark ml situation: An ML engineer manually compares two logistic-regression regularization values on a single holdout but has no reusable SparkML parameter-search plan. Here the actual success criterion is holdout-only categories no longer influence the training-fitted indexer and metrics are recomputed on untouched holdout records.
Question 255
Review this production model incident: An organization has reliable model serving but never archives feature definitions or experiment parameters and cannot reproduce predictions after drift. What end-to-end verification is most persuasive here?
- The same signed release revision and dependencies run in test and production with matching fixture output.
- A clean production-like environment loads the promoted artifact and matches approved prediction fixtures.
- A replay from retained lineage recreates an example prediction within documented tolerance.
- An integration fixture confirms the promoted job reads the intended schema before traffic is moved.
Explanation
Correct Answer
C:This verification measures the required result rather than just process liveness. A replay from retained lineage recreates an example prediction within documented tolerance. It confirms whether the targeted action succeeded: record end-to-end lineage and immutable versions at each necessary lifecycle boundary.
Incorrect Answers
A:That acceptance test is directed at a different model lifecycle management situation: A regulated company changes deployed Python notebook code directly in production to correct a model transform while its test environment runs a different untracked revision. Here the actual success criterion is a replay from retained lineage recreates an example prediction within documented tolerance.
B:That acceptance test is directed at a different model lifecycle management situation: A model is promoted to production with its code unchanged but a different unpinned training library version is installed, altering prediction preprocessing. Here the actual success criterion is a replay from retained lineage recreates an example prediction within documented tolerance.
D:That acceptance test is directed at a different model lifecycle management situation: A customer segmentation project promotes a scoring job but omits a database migration required by its new feature schema. Here the actual success criterion is a replay from retained lineage recreates an example prediction within documented tolerance.
Question 256
Review this production model incident: An insurer already stores its risk features as a large Spark DataFrame but exports them into Python lists before every vector transformation and repeatedly hits out-of-memory errors. What is the central engineering constraint behind this behavior?
- A SparkML Pipeline requires transformer stages in an order that produces the input types expected by later stages.
- Training-fitted preprocessors may reject unseen labels; inference handling must be planned and tested explicitly.
- Repeated driver collection defeats the distributed processing advantages relevant to large SparkML feature pipelines.
- RMSE squares residuals and highlights large errors, while median error alone may obscure expensive outliers.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Repeated driver collection defeats the distributed processing advantages relevant to large SparkML feature pipelines. To test it, inspect DataFrame physical operations and memory spikes at each Python collection boundary.
Incorrect Answers
A:That underlying principle is directed at a different using spark ml situation: An IoT classifier passes raw text categories directly into VectorAssembler ahead of StringIndexer and OneHotEncoder, generating incompatible vector inputs. The current incident instead turns on repeated driver collection defeats the distributed processing advantages relevant to large SparkML feature pipelines.
B:That underlying principle is directed at a different using spark ml situation: An inference pipeline sends new categorical labels unseen during training to StringIndexerModel with error handling configured to reject unknown categories. The current incident instead turns on repeated driver collection defeats the distributed processing advantages relevant to large SparkML feature pipelines.
D:That underlying principle is directed at a different using spark ml situation: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. The current incident instead turns on repeated driver collection defeats the distributed processing advantages relevant to large SparkML feature pipelines.
Question 257
A production ML review contains this situation: A feature-engineering job is CPU-bound on hundreds of independent daily partitions and a team upgrades only the driver to a much larger machine. Which check shows that this particular problem has been resolved?
- Task throughput scales with added worker capacity while the driver remains within its limits.
- Peak group processing completes with bounded memory and a defined model per intended account.
- A restart retains finished trial metrics and resumes remaining work without rerunning accepted completions.
- The supported sharded model initializes and completes a forward/backward training step without OOM.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. Task throughput scales with added worker capacity while the driver remains within its limits. It confirms whether the targeted action succeeded: scale worker capacity and partition layout according to measured distributed CPU demand.
Incorrect Answers
B:That acceptance test is directed at a different scaling and tuning situation: One enterprise account holds 80% of historical events, and grouped pandas training for that account exceeds executor memory while other accounts finish. Here the actual success criterion is task throughput scales with added worker capacity while the driver remains within its limits.
C:That acceptance test is directed at a different scaling and tuning situation: An interrupted Ray hyperparameter sweep has an expensive completed-trial history but a new launch discards its experiment checkpoint. Here the actual success criterion is task throughput scales with added worker capacity while the driver remains within its limits.
D:That acceptance test is directed at a different scaling and tuning situation: A language model's weights cannot fit on one accelerator yet a team launches pure data-parallel replicas, each loading the entire model. Here the actual success criterion is task throughput scales with added worker capacity while the driver remains within its limits.
Question 258
A production ML review contains this situation: A feature function maps a null purchase amount to zero without distinguishing missing observations, and a notebook unit test covers only positive amounts. Which check shows that this particular problem has been resolved?
- The exact saved fixture is successfully scored via the deployed endpoint with expected fields.
- All boundary fixtures produce expected labeled outputs and the test suite catches the original defect.
- The integration fixture traverses every component with matching validated field names.
- Normal production execution does not run tests, while CI executes the full isolated suite.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. All boundary fixtures produce expected labeled outputs and the test suite catches the original defect. It confirms whether the targeted action succeeded: write explicit notebook unit tests for missingness and correct zero handling, then fix the transformation if needed.
Incorrect Answers
A:That acceptance test is directed at a different validation testing situation: A churn feature job succeeds and a classifier achieves acceptable F1, but the deployed endpoint returns HTTP 400 because request column names differ. Here the actual success criterion is all boundary fixtures produce expected labeled outputs and the test suite catches the original defect.
C:That acceptance test is directed at a different validation testing situation: A team unit-tests each model component independently but its feature table and serving endpoint disagree on column names. Here the actual success criterion is all boundary fixtures produce expected labeled outputs and the test suite catches the original defect.
D:That acceptance test is directed at a different validation testing situation: An ML project stores test files directly in each production notebook and execution of a deployed notebook inadvertently reruns expensive tests. Here the actual success criterion is all boundary fixtures produce expected labeled outputs and the test suite catches the original defect.
Question 259
A production ML review contains this situation: A sensor classifier silently reuses features from a year-old device after months without readings, despite a maximum allowed lookback of seven days. Which check shows that this particular problem has been resolved?
- Load tests meet request-time latency while documented fallback values remain semantically valid.
- The model input is missing or flagged stale when no eligible feature exists inside the authorized window.
- Equivalent test events produce matching approved feature values across both paths.
- Duplicated input fixtures contribute exactly once to the computed feature total.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. The model input is missing or flagged stale when no eligible feature exists inside the authorized window. It confirms whether the targeted action succeeded: enforce a supported lookback policy and explicit null handling for devices without recent data.
Incorrect Answers
A:That acceptance test is directed at a different advanced feature store concepts situation: A streaming recommender's request-time feature function calls a slow third-party API synchronously on every prediction. Here the actual success criterion is the model input is missing or flagged stale when no eligible feature exists inside the authorized window.
C:That acceptance test is directed at a different advanced feature store concepts situation: An ML pipeline computes training features in one notebook and duplicates slightly different SQL in the online pipeline for the same entities. Here the actual success criterion is the model input is missing or flagged stale when no eligible feature exists inside the authorized window.
D:That acceptance test is directed at a different advanced feature store concepts situation: A real-time feature job updates user purchase totals without checking whether upstream transaction events are delivered twice. Here the actual success criterion is the model input is missing or flagged stale when no eligible feature exists inside the authorized window.
Question 260
The engineering team encounters the following issue: A recommendation team wraps a large pandas UDF around entire customer histories, then discovers some partitions exceed worker memory while others finish quickly. What investigation would most effectively isolate this behavior?
- Inspect the Optuna trial table and compare its IDs and metrics against MLflow runs.
- Review simulator state ownership, task dependency graph and required actor lifetimes.
- Inspect per-group record counts, Arrow batch sizes and executor spill or OOM traces.
- Compare GPU inventory with the current trial task graph, resource reservations and wait times.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect per-group record counts, Arrow batch sizes and executor spill or OOM traces. The underlying technical issue is that pandas-function distribution requires bounded partition or group size, not just parallel execution of arbitrary large pandas objects.
Incorrect Answers
A:That diagnostic evidence is directed at a different scaling and tuning situation: An Optuna search runs 80 hyperparameter trials but records only the last winning score in MLflow, making comparisons impossible. For the stated incident, the decisive observation is inspect per-group record counts, Arrow batch sizes and executor spill or OOM traces.
B:That diagnostic evidence is directed at a different scaling and tuning situation: A reinforcement-learning simulator requires stateful actors exchanging messages and irregular task graphs, but the team plans one Spark SQL groupBy per simulation step. For the stated incident, the decisive observation is inspect per-group record counts, Arrow batch sizes and executor spill or OOM traces.
D:That diagnostic evidence is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. For the stated incident, the decisive observation is inspect per-group record counts, Arrow batch sizes and executor spill or OOM traces.