Databricks Certified Machine Learning Professional Practice Test 1
Question 1
Consider the following fictional Databricks production case: A custom PyFunc's predict method assumes every request contains a numeric feature that is optional under the documented inference schema. What underlying principle determines the most appropriate interpretation?
- Custom model input contracts should validate missing and malformed values before feature transformation.
- Serving responses that are cached or audited need model-version provenance and appropriate expiration rules.
- Client integrations should parse the documented model output contract instead of assuming a scalar response.
- Serving deployment permissions must include authorization to use the intended governed model and endpoint resources.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Custom model input contracts should validate missing and malformed values before feature transformation. To test it, replay valid, missing and nonnumeric fixture fields against the PyFunc predict interface.
Incorrect Answers
B:That underlying principle is directed at a different custom model serving situation: An integration caches predictions without recording the model version and returns yesterday's values after a controlled rollback. The current incident instead turns on custom model input contracts should validate missing and malformed values before feature transformation.
C:That underlying principle is directed at a different custom model serving situation: A serving client calls the REST endpoint expecting one number but the custom PyFunc returns a structured list of prediction and confidence fields. The current incident instead turns on custom model input contracts should validate missing and malformed values before feature transformation.
D:That underlying principle is directed at a different custom model serving situation: A test engineer can create a model-serving endpoint in a sandbox but the production deployment fails because the automation identity lacks registered-model access. The current incident instead turns on custom model input contracts should validate missing and malformed values before feature transformation.
Question 2
A production ML review contains this situation: A developer changes a feature transform in a notebook and directly uses production data as the only test case. Which statement best explains the failure mode?
- Test organization should separate production orchestration from the test runner and control execution stages.
- Unit tests belong on controlled synthetic inputs before integrations, and production smoke tests do not replace them.
- Unit tests should exercise edge cases and expected output contracts, not just the happy-path fixture.
- A full ML integration test should traverse raw feature generation through training, registration, deployment and inference contracts.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Unit tests belong on controlled synthetic inputs before integrations, and production smoke tests do not replace them. To test it, inspect test-environment evidence and the presence of function-level synthetic fixtures.
Incorrect Answers
A:That underlying principle is directed at a different validation testing situation: An ML project stores test files directly in each production notebook and execution of a deployed notebook inadvertently reruns expensive tests. The current incident instead turns on unit tests belong on controlled synthetic inputs before integrations, and production smoke tests do not replace them.
C:That underlying principle is directed at a different validation testing situation: A feature function maps a null purchase amount to zero without distinguishing missing observations, and a notebook unit test covers only positive amounts. The current incident instead turns on unit tests belong on controlled synthetic inputs before integrations, and production smoke tests do not replace them.
D:That underlying principle is directed at a different validation testing situation: A churn feature job succeeds and a classifier achieves acceptable F1, but the deployed endpoint returns HTTP 400 because request column names differ. The current incident instead turns on unit tests belong on controlled synthetic inputs before integrations, and production smoke tests do not replace them.
Question 3
In an enterprise ML workflow, the reported issue is: A model's feature generation step overwrites all feature-table history daily even though time-aware training requires earlier snapshots. What likely operational or model-quality harm does this issue create?
- An apparently healthy endpoint serves outdated feature values and produces obsolete scores.
- An ancient measurement can influence a current score as if it were fresh evidence.
- Historical training datasets cannot be faithfully reconstructed after the overwrite.
- Scores after failover revert to incomplete or uninitialized feature state.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. Historical training datasets cannot be faithfully reconstructed after the overwrite. The causal constraint is: Historical feature timelines must be retained when reproducible point-in-time joins depend on prior values.
Incorrect Answers
A:That direct consequence is directed at a different advanced feature store concepts situation: A team publishes an online feature table but neglects its refresh schedule while the predictive model assumes values no older than 15 minutes. The more immediate risk here is historical training datasets cannot be faithfully reconstructed after the overwrite.
B:That direct consequence is directed at a different advanced feature store concepts situation: A sensor classifier silently reuses features from a year-old device after months without readings, despite a maximum allowed lookback of seven days. The more immediate risk here is historical training datasets cannot be faithfully reconstructed after the overwrite.
D:That direct consequence is directed at a different advanced feature store concepts situation: A business deploys a model whose features are computed from a stream in memory only and disappear when its compute cluster restarts. The more immediate risk here is historical training datasets cannot be faithfully reconstructed after the overwrite.
Question 4
The engineering team encounters the following issue: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. What direct consequence should the owner prioritize if no fix is made?
- Traffic growth is mistaken for increasing risk because the raw count rises mechanically.
- Selection bias looks like model input deterioration even though each subgroup is stable.
- A small but important region remains unavailable while overall service metrics seem healthy.
- The cold-start cohort receives poor recommendations while the combined metric remains acceptable.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. The cold-start cohort receives poor recommendations while the combined metric remains acceptable. The causal constraint is: Aggregate performance can conceal failure in important subpopulations and lifecycle stages.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A monitoring team wants a business metric for the share of transactions exceeding a risk threshold but records only the raw count of high-risk transactions. The more immediate risk here is the cold-start cohort receives poor recommendations while the combined metric remains acceptable.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A customer-risk monitoring job compares an unfiltered current high-risk segment to an all-customer historical baseline and interprets the difference as temporal drift. The more immediate risk here is the cold-start cohort receives poor recommendations while the combined metric remains acceptable.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. The more immediate risk here is the cold-start cohort receives poor recommendations while the combined metric remains acceptable.
Question 5
Review this production model incident: An inference profile groups two active model versions into one aggregate and hides a regression introduced by the new release. What end-to-end verification is most persuasive here?
- A test alert identifies the precise asset, time window, threshold and metric that caused it.
- A synthetic poor new-version cohort is distinguishable from unchanged champion predictions.
- A synthetic slow drift triggers the fixed-baseline comparison before the model exceeds tolerance.
- The shifted synthetic cohort triggers the expected numeric distribution alert and the baseline cohort does not.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. A synthetic poor new-version cohort is distinguishable from unchanged champion predictions. It confirms whether the targeted action succeeded: configure model-identity-aware inference monitoring with version-specific comparisons.
Incorrect Answers
A:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A drift alert fires but its message contains only the word anomaly without dataset, model version or baseline window. Here the actual success criterion is a synthetic poor new-version cohort is distinguishable from unchanged champion predictions.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. Here the actual success criterion is a synthetic poor new-version cohort is distinguishable from unchanged champion predictions.
D:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A loan-risk feature usually has median debt-to-income 0.34, but a new cohort has median 0.61 while the average stays almost unchanged due to outliers. Here the actual success criterion is a synthetic poor new-version cohort is distinguishable from unchanged champion predictions.
Question 6
Review this production model incident: A feature stream loses half its source events and a retraining policy treats the resulting input shift as proof that the learned concept changed. What is the central engineering constraint behind this behavior?
- Champion selection must enforce critical segment-level guardrails in addition to aggregate ranking metrics.
- Production model selection must satisfy deployment constraints as well as validation quality.
- Data pipeline outages must be distinguished from genuine population shift before retraining.
- Model candidates need evaluation on the same versioned holdout and metric definition for fair comparison.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Data pipeline outages must be distinguished from genuine population shift before retraining. To test it, compare source completeness, freshness and drift metrics across the same incident window.
Incorrect Answers
A:That underlying principle is directed at a different automated retraining situation: A new fraud classifier improves overall F1 but doubles false negatives for high-value payment accounts, and an automated policy chooses it on F1 alone. The current incident instead turns on data pipeline outages must be distinguished from genuine population shift before retraining.
B:That underlying principle is directed at a different automated retraining situation: An AutoML training job always chooses the model with the lowest training loss, even when the deployed service has a strict memory and P95 latency budget. The current incident instead turns on data pipeline outages must be distinguished from genuine population shift before retraining.
D:That underlying principle is directed at a different automated retraining situation: A retraining pipeline ranks candidate models using metrics calculated on different dataset versions, then selects the numerically highest result. The current incident instead turns on data pipeline outages must be distinguished from genuine population shift before retraining.
Question 7
In an enterprise ML workflow, the reported issue is: An ML platform moves its configuration from older Databricks Asset Bundles terminology to newer Declarative Automation Bundles references and assumes every resource schema stayed identical. What likely operational or model-quality harm does this issue create?
- A release fails when a renamed or unsupported field is assumed to work unchanged.
- A developer's experiment can alter production feature tables or serving configuration.
- Critical retraining stops when a human account or notebook workspace changes.
- One experiment starves critical retraining or inference workloads.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. A release fails when a renamed or unsupported field is assumed to work unchanged. The causal constraint is: Product naming and resource schema changes should be checked against the active Databricks CLI and documentation for the deployed version.
Incorrect Answers
B:That direct consequence is directed at a different environment architectures situation: A company operates development and production models in one workspace under the same broad service account and frequently overwrites production tables during experiments. The more immediate risk here is a release fails when a renamed or unsupported field is assumed to work unchanged.
C:That direct consequence is directed at a different environment architectures situation: A distributed retraining workflow uses ephemeral personal compute and fails when the original engineer's account is disabled. The more immediate risk here is a release fails when a renamed or unsupported field is assumed to work unchanged.
D:That direct consequence is directed at a different environment architectures situation: A production deployment scales to 30 teams but all jobs share one compute pool with no capacity limits or cost attribution. The more immediate risk here is a release fails when a renamed or unsupported field is assumed to work unchanged.
Question 8
The engineering team encounters the following issue: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. Which experiment most directly demonstrates a successful fix?
- A controlled failback restores the previous version without rebuilding model artifacts.
- Both supported request versions produce valid inference results or explicit documented compatibility responses.
- A readiness probe and known inference fixture both succeed before planned traffic movement.
- An endpoint configuration readback and sampled test traffic confirm the intended distribution.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. A controlled failback restores the previous version without rebuilding model artifacts. It confirms whether the targeted action succeeded: prepare a complete tested green environment and switch traffic after approval while retaining blue for rollback.
Incorrect Answers
B:That acceptance test is directed at a different deployment strategies situation: A rollout updates the serving model but a renamed input column breaks requests from an older mobile client. Here the actual success criterion is a controlled failback restores the previous version without rebuilding model artifacts.
C:That acceptance test is directed at a different deployment strategies situation: An endpoint update request succeeds but the newly configured model version is still loading; monitoring marks the release complete immediately. Here the actual success criterion is a controlled failback restores the previous version without rebuilding model artifacts.
D:That acceptance test is directed at a different deployment strategies situation: A Databricks Model Serving endpoint hosts champion and candidate versions, but a release config accidentally routes 100% of traffic to the candidate instead of 10%. Here the actual success criterion is a controlled failback restores the previous version without rebuilding model artifacts.
Question 9
In an enterprise ML workflow, the reported issue is: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. What technical distinction is essential to this investigation?
- Separate child run identities are required for concurrent trial provenance; a shared active run is not a concurrency-safe catchall.
- The model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
- Logged metric names require consistent definitions and units before their numerical values can be compared.
- Parent and nested child runs encode experiment-to-trial relationships while preserving each trial's parameters and metrics.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. The model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew. To test it, compare training feature timestamp definitions with live feature retrieval windows on a matched historical request.
Incorrect Answers
A:That underlying principle is directed at a different advanced mlflow usage situation: A training orchestrator starts parallel child fits but all workers accidentally log into one active MLflow run and overwrite each other's metrics. The current incident instead turns on the model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
C:That underlying principle is directed at a different advanced mlflow usage situation: A team logs a score labeled accuracy with values between 0 and 100 in some runs and between 0 and 1 in others, then ranks runs automatically. The current incident instead turns on the model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
D:That underlying principle is directed at a different advanced mlflow usage situation: An experiment tunes 40 candidates but logs each trial as an unrelated root MLflow run, so reviewers cannot reconstruct which search produced a winner. The current incident instead turns on the model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
Question 10
Consider the following fictional Databricks production case: A single-node model training run spends almost all time swapping a 40 GB feature matrix on a 16 GB VM; no distributed version exists. Which implementation is most appropriate for the stated incident?
- Increase the training host's memory within budget or change to a supported distributed estimator.
- Assign each trial a unique tracked run with a shared parent experiment and stable trial metadata.
- Adopt validated model-parallel or sharded training to fit the model and its optimizer state.
- Prepare a versioned reusable feature snapshot and schedule Ray trials against appropriately shared data.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Increase the training host's memory within budget or change to a supported distributed estimator. The issue matters because training stalls on memory pressure even while unrelated cluster nodes are underutilized.
Incorrect Answers
B:That corrective action is directed at a different scaling and tuning situation: An Optuna optimization reuses the same MLflow run ID for concurrent trials, causing overlapping parameter values and artifacts. The relevant corrective action here is increase the training host's memory within budget or change to a supported distributed estimator.
C:That corrective action is directed at a different scaling and tuning situation: A language model's weights cannot fit on one accelerator yet a team launches pure data-parallel replicas, each loading the entire model. The relevant corrective action here is increase the training host's memory within budget or change to a supported distributed estimator.
D:That corrective action is directed at a different scaling and tuning situation: A hyperparameter search operates on large Spark-managed feature tables but each candidate model is small and independent, and workers repeatedly convert the entire table for each Ray task. The relevant corrective action here is increase the training host's memory within budget or change to a supported distributed estimator.