Databricks Certified Machine Learning Professional Practice Test 56
Question 550
Review this production model incident: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. Which failure risk follows most closely from the unresolved behavior?
- A user can access or mutate a deployed model despite restricted tracking visibility.
- Production inference can access unrelated protected data or train on unintended tables.
- The live model logic diverges from tested code and cannot be reproduced during an incident.
- Apparently identical code yields different results or fails to load under the new runtime.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. A user can access or mutate a deployed model despite restricted tracking visibility. The causal constraint is: Experiment tracking metadata is not a substitute for governed asset privileges and serving access control.
Incorrect Answers
B:That direct consequence is directed at a different model lifecycle management situation: A team trains on curated development features but deploys the same job with credentials that accidentally read unapproved production source tables. The more immediate risk here is a user can access or mutate a deployed model despite restricted tracking visibility.
C:That direct consequence is directed at a different model lifecycle management situation: A regulated company changes deployed Python notebook code directly in production to correct a model transform while its test environment runs a different untracked revision. The more immediate risk here is a user can access or mutate a deployed model despite restricted tracking visibility.
D:That direct consequence is directed at a different model lifecycle management situation: A model is promoted to production with its code unchanged but a different unpinned training library version is installed, altering prediction preprocessing. The more immediate risk here is a user can access or mutate a deployed model despite restricted tracking visibility.
Question 551
Review this production model incident: A prediction endpoint returns HTTP 503 for intermittent bursts while the model's offline accuracy remains unchanged. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Audit request identifiers, event timestamps and join cardinality in the performance computation.
- Correlate 5xx status bursts with request rate, resource use and deployment events.
- Compare historical baselines, current seasonal patterns and change approvals for the existing threshold.
- Review the metric query's output grain, fields and result-table access against data classification.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Correlate 5xx status bursts with request rate, resource use and deployment events. The underlying technical issue is that serving availability errors are distinct from model predictive accuracy and require infrastructure telemetry.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. For the stated incident, the decisive observation is correlate 5xx status bursts with request rate, resource use and deployment events.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. For the stated incident, the decisive observation is correlate 5xx status bursts with request rate, resource use and deployment events.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A monitor owner writes a custom metric that logs unaggregated account-level health data in a broadly shared results table. For the stated incident, the decisive observation is correlate 5xx status bursts with request rate, resource use and deployment events.
Question 552
The engineering team encounters the following issue: A fraud PyFunc uses a global mutable feature cache initialized by the first request and mixes data between subsequent users. Which technical principle most directly explains the root of the issue?
- MLflow run-level logging should attach parameters and evaluated metrics to the corresponding candidate run.
- Parent and nested child runs encode experiment-to-trial relationships while preserving each trial's parameters and metrics.
- A resumed search needs stable experiment and parent-child provenance across partial reruns to support comparison.
- Request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. Request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers. To test it, replay interleaved synthetic customer requests and inspect whether cache keys and outputs remain isolated.
Incorrect Answers
A:That underlying principle is directed at a different advanced mlflow usage situation: A training notebook reports F1 for several models but only logs the final model's parameters in MLflow, leaving metrics and hyperparameters disconnected. The current incident instead turns on request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
B:That underlying principle is directed at a different advanced mlflow usage situation: An experiment tunes 40 candidates but logs each trial as an unrelated root MLflow run, so reviewers cannot reconstruct which search produced a winner. The current incident instead turns on request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
C:That underlying principle is directed at a different advanced mlflow usage situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. The current incident instead turns on request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
Question 553
In an enterprise ML workflow, the reported issue is: A model version is registered but the deployment job points to an old alias after promotion and continues serving last week's model. Before modifying the pipeline, which evidence should be collected first?
- Run a tiny constant-column fixture and inspect finite-value validation after transformation.
- Identify deterministic data transformations entangled with session setup and filesystem side effects.
- Inspect test-environment evidence and the presence of function-level synthetic fixtures.
- Inspect registered model versions, alias resolution and active serving configuration after the release.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Inspect registered model versions, alias resolution and active serving configuration after the release. The underlying technical issue is that model artifact identity must be checked through registry promotion and endpoint deployment, not inferred from job success.
Incorrect Answers
A:That diagnostic evidence is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. For the stated incident, the decisive observation is inspect registered model versions, alias resolution and active serving configuration after the release.
B:That diagnostic evidence is directed at a different validation testing situation: A team hides all preprocessing logic inside one long notebook, making it hard to test independently without starting a Spark cluster. For the stated incident, the decisive observation is inspect registered model versions, alias resolution and active serving configuration after the release.
C:That diagnostic evidence is directed at a different validation testing situation: A developer changes a feature transform in a notebook and directly uses production data as the only test case. For the stated incident, the decisive observation is inspect registered model versions, alias resolution and active serving configuration after the release.
Question 554
In an enterprise ML workflow, the reported issue is: A custom PyFunc estimator loads a tokenizer file from an engineer's laptop path when called in a serving container. Before modifying the pipeline, which evidence should be collected first?
- Inspect actual test response JSON and registered output schema for a representative batch.
- Compare sandbox/prod identities, model grants and sanitized deployment authorization errors.
- Inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes.
- Inspect the model's artifact map and the load_context path resolution under an isolated inference environment.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Inspect the model's artifact map and the load_context path resolution under an isolated inference environment. The underlying technical issue is that custom Python-model artifacts need portable packaged dependencies rather than absolute developer-local file paths.
Incorrect Answers
A:That diagnostic evidence is directed at a different custom model serving situation: A serving client calls the REST endpoint expecting one number but the custom PyFunc returns a structured list of prediction and confidence fields. For the stated incident, the decisive observation is inspect the model's artifact map and the load_context path resolution under an isolated inference environment.
B:That diagnostic evidence is directed at a different custom model serving situation: A test engineer can create a model-serving endpoint in a sandbox but the production deployment fails because the automation identity lacks registered-model access. For the stated incident, the decisive observation is inspect the model's artifact map and the load_context path resolution under an isolated inference environment.
C:That diagnostic evidence is directed at a different custom model serving situation: A client invokes a registered custom ML model using an unauthenticated HTTP POST to the serving endpoint and assumes network access implies authorization. For the stated incident, the decisive observation is inspect the model's artifact map and the load_context path resolution under an isolated inference environment.
Question 555
In an enterprise ML workflow, the reported issue is: A business deploys a model whose features are computed from a stream in memory only and disappear when its compute cluster restarts. How should the engineer validate the correction?
- A restart restores feature values consistent with the pre-restart control dataset.
- The same representative raw inputs yield matching feature values in training and online requests.
- A scheduled run publishes the expected governed feature-table version with traceable input versions.
- Renaming a display label does not alter the entity's online lookup result.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. A restart restores feature values consistent with the pre-restart control dataset. It confirms whether the targeted action succeeded: publish computed features to a durable governed destination with supported streaming recovery.
Incorrect Answers
B:That acceptance test is directed at a different advanced feature store concepts situation: A customer-risk feature needs a ratio of current debt to income, but online serving recomputes it differently from the training pipeline. Here the actual success criterion is a restart restores feature values consistent with the pre-restart control dataset.
C:That acceptance test is directed at a different advanced feature store concepts situation: A team manually reruns a feature notebook each week before training; feature definitions change without an execution history or stable destination. Here the actual success criterion is a restart restores feature values consistent with the pre-restart control dataset.
D:That acceptance test is directed at a different advanced feature store concepts situation: A low-latency feature table uses external customer names as lookup keys; names change and are not unique. Here the actual success criterion is a restart restores feature values consistent with the pre-restart control dataset.
Question 556
The engineering team encounters the following issue: An engineering group duplicates a validation helper across six notebooks, and two copies enforce outdated threshold values. What corrective action should the engineer prefer?
- Centralize versioned validation functions and test behavior under all consuming environments.
- Use approved synthetic or appropriately deidentified test data with controlled production smoke verification.
- Guard zero-variance inputs and assert expected fallback or explicit failure semantics in unit tests.
- Build an end-to-end integration fixture that verifies schemas, versions and prediction outputs across stages.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Centralize versioned validation functions and test behavior under all consuming environments. The issue matters because different releases disagree on whether a model passes the same quality gate.
Incorrect Answers
B:That corrective action is directed at a different validation testing situation: An engineer uses only production replay data to validate a code change and exposes customer events in an unrestricted test workspace. The relevant corrective action here is centralize versioned validation functions and test behavior under all consuming environments.
C:That corrective action is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. The relevant corrective action here is centralize versioned validation functions and test behavior under all consuming environments.
D:That corrective action is directed at a different validation testing situation: A churn feature job succeeds and a classifier achieves acceptable F1, but the deployed endpoint returns HTTP 400 because request column names differ. The relevant corrective action here is centralize versioned validation functions and test behavior under all consuming environments.
Question 557
The engineering team encounters the following issue: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. What corrective action should the engineer prefer?
- Split the raw observations first, then fit every learned preprocessing stage on training data only.
- Evaluate the model with class-sensitive metrics and choose thresholds aligned with false-positive and missed-fraud tradeoffs.
- Select a supported distributed training framework for that model or a deliberate single-node fit rather than relabeling it SparkML.
- Build an explicit parameter grid with a compatible CrossValidator or TrainValidationSplit and preserve a final untouched test set.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Split the raw observations first, then fit every learned preprocessing stage on training data only. The issue matters because the supposedly unseen holdout can affect model inputs and inflate reported generalization.
Incorrect Answers
B:That corrective action is directed at a different using spark ml situation: A fraud detector predicts the majority no-fraud class for 99.4% of cases and advertises 99.4% accuracy while missing most positives. The relevant corrective action here is split the raw observations first, then fit every learned preprocessing stage on training data only.
C:That corrective action is directed at a different using spark ml situation: A media group has a specialized deep neural model that is not implemented as a SparkML estimator and assumes SparkML will automatically distribute its GPU training. The relevant corrective action here is split the raw observations first, then fit every learned preprocessing stage on training data only.
D:That corrective action is directed at a different using spark ml situation: An ML engineer manually compares two logistic-regression regularization values on a single holdout but has no reusable SparkML parameter-search plan. The relevant corrective action here is split the raw observations first, then fit every learned preprocessing stage on training data only.
Question 558
In an enterprise ML workflow, the reported issue is: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. What technical distinction is essential to this investigation?
- Independent per-group models are suitable for distributing grouped pandas-function work, subject to available compute and partition size.
- Ray resource declarations must match actual device use to avoid GPU oversubscription.
- Tuning runs must use equivalent evaluation data and metric definitions before ranking configurations.
- Data parallelism is often more natural than model-parallel networking for a small model with large independent data.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Ray resource declarations must match actual device use to avoid GPU oversubscription. To test it, inspect runtime GPU memory per worker alongside Ray trial resource requests.
Incorrect Answers
A:That underlying principle is directed at a different scaling and tuning situation: An ecommerce analyst fits one independent scikit-learn model per 3,000 product categories, but the driver sequentially loops through all category subsets. The current incident instead turns on ray resource declarations must match actual device use to avoid GPU oversubscription.
C:That underlying principle is directed at a different scaling and tuning situation: A hyperparameter team compares two Optuna studies with different train/validation splits and chooses the lower loss as though both metrics were directly comparable. The current incident instead turns on ray resource declarations must match actual device use to avoid GPU oversubscription.
D:That underlying principle is directed at a different scaling and tuning situation: A tabular classifier fits comfortably on one GPU and 100 million independent examples are available, but a team proposes splitting the small model's layers across hosts. The current incident instead turns on ray resource declarations must match actual device use to avoid GPU oversubscription.
Question 559
Consider the following fictional Databricks production case: A time-ordered customer-retention dataset includes events from the future, yet a developer randomly cross-validates across time and then claims a forward-looking forecast is unbiased. Which observation most directly tests the suspected failure boundary?
- Compare train and validation event timestamps and detect any fold containing observations after its test period.
- Inspect its declared output column and confirm the downstream estimator consumes a vector rather than predicted labels.
- Inspect fitted category mappings, handleInvalid behavior and a sample newly introduced category.
- Compare request latency and concurrency service-level objectives with the batch schedule and observed turnaround.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Compare train and validation event timestamps and detect any fold containing observations after its test period. The underlying technical issue is that temporal evaluation must respect what data would have been available at the forecast origin, not simply use random folds.
Incorrect Answers
B:That diagnostic evidence is directed at a different using spark ml situation: A team wants to convert several numeric inputs into one features vector but treats VectorAssembler as a clustering Estimator that should output predictions. For the stated incident, the decisive observation is compare train and validation event timestamps and detect any fold containing observations after its test period.
C:That diagnostic evidence is directed at a different using spark ml situation: An inference pipeline sends new categorical labels unseen during training to StringIndexerModel with error handling configured to reject unknown categories. For the stated incident, the decisive observation is compare train and validation event timestamps and detect any fold containing observations after its test period.
D:That diagnostic evidence is directed at a different using spark ml situation: An online pricing service requires a personalized response within 70 milliseconds per request but its team schedules a full overnight batch job as the only inference route. For the stated incident, the decisive observation is compare train and validation event timestamps and detect any fold containing observations after its test period.
Question 560
A production ML review contains this situation: A regulated company changes deployed Python notebook code directly in production to correct a model transform while its test environment runs a different untracked revision. What production-safe change best matches the evidence?
- Bind training outputs to registered immutable versions and promote them through governed jobs or serving endpoints.
- Use a supported governed model registry and controlled permissions for production model versions.
- Build and validate one immutable release artifact before promoting it through controlled environments.
- Apply the correct governance controls separately to experiments, registered models and serving endpoints.
Explanation
Correct Answer
C:This corrects the relevant failure boundary instead of merely hiding the symptom. Build and validate one immutable release artifact before promoting it through controlled environments. The issue matters because the live model logic diverges from tested code and cannot be reproduced during an incident.
Incorrect Answers
A:That corrective action is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. The relevant corrective action here is build and validate one immutable release artifact before promoting it through controlled environments.
B:That corrective action is directed at a different model lifecycle management situation: An engineer stores model weights in a personal workspace folder and treats that path as a production artifact registry with audit history. The relevant corrective action here is build and validate one immutable release artifact before promoting it through controlled environments.
D:That corrective action is directed at a different model lifecycle management situation: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. The relevant corrective action here is build and validate one immutable release artifact before promoting it through controlled environments.