Databricks Certified Machine Learning Professional Practice Test 10
Question 90
Consider the following fictional Databricks production case: A pipeline engineer fits a vectorizer after an estimator in the stage list and then tries to use the estimator's prediction as the vectorizer input. Which implementation is most appropriate for the stated incident?
- Retain the correctly validated single-node training path unless a measured scale requirement justifies SparkML.
- Configure selection against the intended minimization metric and rerun model comparison.
- Deploy the fitted preprocessing and estimator together as one versioned pipeline artifact.
- Topologically order vectorizers, assemblers and estimators so required feature columns exist before model fitting.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Topologically order vectorizers, assemblers and estimators so required feature columns exist before model fitting. The issue matters because pipeline fitting fails with missing-column or mismatched-schema errors.
Incorrect Answers
A:That corrective action is directed at a different using spark ml situation: A laboratory runs a linear regression every quarter on only 280 clean observations and proposes a 48-worker Spark cluster to reduce a two-second local fit. The relevant corrective action here is topologically order vectorizers, assemblers and estimators so required feature columns exist before model fitting.
B:That corrective action is directed at a different using spark ml situation: A SparkML grid-search experiment selects the model with the largest error metric even though the evaluator reports RMSE to minimize. The relevant corrective action here is topologically order vectorizers, assemblers and estimators so required feature columns exist before model fitting.
C:That corrective action is directed at a different using spark ml situation: A batch scoring service loads a classifier version from last month while using today's new feature indexer, silently changing categorical positions. The relevant corrective action here is topologically order vectorizers, assemblers and estimators so required feature columns exist before model fitting.
Question 91
In an enterprise ML workflow, the reported issue is: A streaming anomaly team uses a static read of a message directory once, believing the trained SparkML model will continuously produce scores for future arriving events. Before modifying the pipeline, which evidence should be collected first?
- Compare request latency and concurrency service-level objectives with the batch schedule and observed turnaround.
- Inspect read versus readStream setup and the active streaming query's progress and sink contract.
- Inspect class prevalence, the confusion matrix, minority recall and false-negative cost rather than headline accuracy.
- Compare train and validation event timestamps and detect any fold containing observations after its test period.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Inspect read versus readStream setup and the active streaming query's progress and sink contract. The underlying technical issue is that a fitted model can transform streaming DataFrames in a supported Structured Streaming pipeline, but a one-time static read is not a continuing source.
Incorrect Answers
A:That diagnostic evidence is directed at a different using spark ml situation: An online pricing service requires a personalized response within 70 milliseconds per request but its team schedules a full overnight batch job as the only inference route. For the stated incident, the decisive observation is inspect read versus readStream setup and the active streaming query's progress and sink contract.
C:That diagnostic evidence is directed at a different using spark ml situation: A fraud detector predicts the majority no-fraud class for 99.4% of cases and advertises 99.4% accuracy while missing most positives. For the stated incident, the decisive observation is inspect read versus readStream setup and the active streaming query's progress and sink contract.
D:That diagnostic evidence is directed at a different using spark ml situation: A time-ordered customer-retention dataset includes events from the future, yet a developer randomly cross-validates across time and then claims a forward-looking forecast is unbiased. For the stated incident, the decisive observation is inspect read versus readStream setup and the active streaming query's progress and sink contract.
Question 92
The engineering team encounters the following issue: A recommender experiment computes all evaluation metrics on training interactions and reports the result as generalization to new customers. What corrective action should the engineer prefer?
- Create an appropriate holdout design, including relevant new-user or time-split cohorts, and evaluate there.
- Fit and persist a unified PipelineModel containing scaling and model stages, then use it for inference.
- Construct a supported streaming ingestion and scoring query with checkpointing and compatible output semantics.
- Publish approved features to a supported low-latency online feature serving path and test endpoint integration.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Create an appropriate holdout design, including relevant new-user or time-split cohorts, and evaluate there. The issue matters because deployment performance may collapse on customers never seen during optimization.
Incorrect Answers
B:That corrective action is directed at a different using spark ml situation: A churn experiment separately scripts feature scaling in training but omits that scaling when the deployed prediction job loads the classifier. The relevant corrective action here is create an appropriate holdout design, including relevant new-user or time-split cohorts, and evaluate there.
C:That corrective action is directed at a different using spark ml situation: A streaming anomaly team uses a static read of a message directory once, believing the trained SparkML model will continuously produce scores for future arriving events. The relevant corrective action here is create an appropriate holdout design, including relevant new-user or time-split cohorts, and evaluate there.
D:That corrective action is directed at a different using spark ml situation: A mobile app requires thousands of concurrent low-latency scores with online features, but a team plans to query an offline historical feature Delta table for every request. The relevant corrective action here is create an appropriate holdout design, including relevant new-user or time-split cohorts, and evaluate there.
Question 93
Review this production model incident: A text pipeline repeatedly fits a Tokenizer stage even though tokenization simply converts strings into token arrays without learning a model. What end-to-end verification is most persuasive here?
- A test run completes preprocessing with bounded driver memory and matching feature values.
- A stored raw fixture produces matching transformed feature vectors and predictions in training and deployment.
- Two sample sentences produce the expected token-array column from a direct transform.
- Both known and unknown fixture categories yield the documented safe scoring behavior.
Explanation
Correct Answer
C:This verification measures the required result rather than just process liveness. Two sample sentences produce the expected token-array column from a direct transform. It confirms whether the targeted action succeeded: use Tokenizer.transform in the correct pipeline position while reserving fit for actual Estimators.
Incorrect Answers
A:That acceptance test is directed at a different using spark ml situation: An insurer already stores its risk features as a large Spark DataFrame but exports them into Python lists before every vector transformation and repeatedly hits out-of-memory errors. Here the actual success criterion is two sample sentences produce the expected token-array column from a direct transform.
B:That acceptance test is directed at a different using spark ml situation: A churn experiment separately scripts feature scaling in training but omits that scaling when the deployed prediction job loads the classifier. Here the actual success criterion is two sample sentences produce the expected token-array column from a direct transform.
D:That acceptance test is directed at a different using spark ml situation: An inference pipeline sends new categorical labels unseen during training to StringIndexerModel with error handling configured to reject unknown categories. Here the actual success criterion is two sample sentences produce the expected token-array column from a direct transform.
Question 94
The engineering team encounters the following issue: A custom drift metric is expressed as today-minus-yesterday percentage, but yesterday's eligible count can be zero. What corrective action should the engineer prefer?
- Tune the proven bottleneck and concurrency/capacity settings instead of scaling CPU blindly.
- Join labels to their originating predictions using stable IDs and a defined maturity window.
- Return a documented missing/undefined result for zero baselines rather than a misleading infinity.
- Create a dedicated approved critical-region alert with suitable minimum volume and failure-rate conditions.
Explanation
Correct Answer
C:This corrects the relevant failure boundary instead of merely hiding the symptom. Return a documented missing/undefined result for zero baselines rather than a misleading infinity. The issue matters because alerts fire on nonsensical infinite values or silently fail during low traffic.
Incorrect Answers
A:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A production endpoint scales CPU instances up when traffic rises, but request queues and P95 response time keep growing despite low CPU usage. The relevant corrective action here is return a documented missing/undefined result for zero baselines rather than a misleading infinity.
B:That corrective action is directed at a different drift detection and lakehouse monitoring situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. The relevant corrective action here is return a documented missing/undefined result for zero baselines rather than a misleading infinity.
D:That corrective action is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. The relevant corrective action here is return a documented missing/undefined result for zero baselines rather than a misleading infinity.
Question 95
Consider the following fictional Databricks production case: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. Which observation most directly tests the suspected failure boundary?
- Inspect received alert fields and the corresponding monitor query, model tag and comparison window.
- Compare labeled performance for new and established users at the same evaluation horizon.
- Inspect subgroup membership, prediction/label joins and false-negative denominator in the source table.
- Compare inference-table last-seen with metric-table last-refresh and schema errors in profiling jobs.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Compare labeled performance for new and established users at the same evaluation horizon. The underlying technical issue is that aggregate performance can conceal failure in important subpopulations and lifecycle stages.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A drift alert fires but its message contains only the word anomaly without dataset, model version or baseline window. For the stated incident, the decisive observation is compare labeled performance for new and established users at the same evaluation horizon.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A fairness reviewer needs the difference in false-negative rates between two approved cohorts, but the monitor computes an aggregate without cohort labels. For the stated incident, the decisive observation is compare labeled performance for new and established users at the same evaluation horizon.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A team has accurate inference logs but its monitoring job never refreshes computed drift metrics after a table schema change. For the stated incident, the decisive observation is compare labeled performance for new and established users at the same evaluation horizon.
Question 96
Review this production model incident: An endpoint update request succeeds but the newly configured model version is still loading; monitoring marks the release complete immediately. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Test candidate-routing behavior for protected region identifiers across many simulated requests.
- Check independent health and version identity for both active and standby deployment configurations before cutover.
- Compare serving-error, latency and outcome metrics separately for champion and canary model versions at the proposed traffic share.
- Inspect endpoint state, individual served-entity readiness and model-load errors after the update.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Inspect endpoint state, individual served-entity readiness and model-load errors after the update. The underlying technical issue is that model Serving configuration acceptance is not equivalent to endpoint/model readiness for user traffic.
Incorrect Answers
A:That diagnostic evidence is directed at a different deployment strategies situation: A model release requires all clients from a restricted region to keep the old version, but a simple percentage split sends some of those clients to the candidate. For the stated incident, the decisive observation is inspect endpoint state, individual served-entity readiness and model-load errors after the update.
B:That diagnostic evidence is directed at a different deployment strategies situation: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. For the stated incident, the decisive observation is inspect endpoint state, individual served-entity readiness and model-load errors after the update.
C:That diagnostic evidence is directed at a different deployment strategies situation: A travel site serves 30,000 bookings per minute and wants to evaluate a new fraud model while limiting customer impact during its first hour. For the stated incident, the decisive observation is inspect endpoint state, individual served-entity readiness and model-load errors after the update.
Question 97
The engineering team encounters the following issue: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. Which experiment most directly demonstrates a successful fix?
- A measured end-to-end trial batch reuses data without redundant full-table copies.
- A controlled A/B benchmark demonstrates the chosen configuration's cost-performance advantage.
- A scheduled trial set uses distinct GPU reservations and runs without memory contention.
- A distributed run trains each intended category once and meets the time budget without driver collection.
Explanation
Correct Answer
C:This verification measures the required result rather than just process liveness. A scheduled trial set uses distinct GPU reservations and runs without memory contention. It confirms whether the targeted action succeeded: declare correct per-trial GPU reservations and reduce concurrent trials until memory fits.
Incorrect Answers
A:That acceptance test is directed at a different scaling and tuning situation: A hyperparameter search operates on large Spark-managed feature tables but each candidate model is small and independent, and workers repeatedly convert the entire table for each Ray task. Here the actual success criterion is a scheduled trial set uses distinct GPU reservations and runs without memory contention.
B:That acceptance test is directed at a different scaling and tuning situation: A moderately sized dataset fits on one worker but a team provisions a huge distributed cluster and observes overhead higher than model fit. Here the actual success criterion is a scheduled trial set uses distinct GPU reservations and runs without memory contention.
D:That acceptance test is directed at a different scaling and tuning situation: An ecommerce analyst fits one independent scikit-learn model per 3,000 product categories, but the driver sequentially loops through all category subsets. Here the actual success criterion is a scheduled trial set uses distinct GPU reservations and runs without memory contention.
Question 98
A production ML review contains this situation: A training team averages gradients from multiple workers but lets each worker use a different untracked version of the model. Which check shows that this particular problem has been resolved?
- Ray reports parallel trial schedules that respect assigned GPUs and complete the planned search.
- Model artifacts and predictions are traceable to each requested store key.
- All workers converge to the same checkpoint step with documented update consistency.
- Each parallel trial resolves to distinct reproducible metrics and artifact storage.
Explanation
Correct Answer
C:This verification measures the required result rather than just process liveness. All workers converge to the same checkpoint step with documented update consistency. It confirms whether the targeted action succeeded: use framework-managed synchronized updates and reproducible checkpoints for the parallel training loop.
Incorrect Answers
A:That acceptance test is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. Here the actual success criterion is all workers converge to the same checkpoint step with documented update consistency.
B:That acceptance test is directed at a different scaling and tuning situation: A retailer needs one independent forecasting model per store but concatenates all store histories before fitting one global model. Here the actual success criterion is all workers converge to the same checkpoint step with documented update consistency.
D:That acceptance test is directed at a different scaling and tuning situation: An Optuna optimization reuses the same MLflow run ID for concurrent trials, causing overlapping parameter values and artifacts. Here the actual success criterion is all workers converge to the same checkpoint step with documented update consistency.
Question 99
A production ML review contains this situation: A monitoring team wants a business metric for the share of transactions exceeding a risk threshold but records only the raw count of high-risk transactions. Which check shows that this particular problem has been resolved?
- Undersized cohorts are appropriately suppressed while adequately powered authorized slices remain measurable.
- Injected 503 responses trigger the service-error alarm and disappear after controlled remediation.
- An injected shift remains visible against the preserved baseline after repeated monitor runs.
- Two fixtures with equal risk proportions but different volumes produce the same custom rate.
Explanation
Correct Answer
D:This verification measures the required result rather than just process liveness. Two fixtures with equal risk proportions but different volumes produce the same custom rate. It confirms whether the targeted action succeeded: define a documented custom rate metric with safe zero-denominator handling in profiling outputs.
Incorrect Answers
A:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A demographic fairness review uses a tiny sensitive subgroup with three labeled examples and reports its observed error as definitive. Here the actual success criterion is two fixtures with equal risk proportions but different volumes produce the same custom rate.
B:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A prediction endpoint returns HTTP 503 for intermittent bursts while the model's offline accuracy remains unchanged. Here the actual success criterion is two fixtures with equal risk proportions but different volumes produce the same custom rate.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A baseline table is overwritten every night with the current day's data before a drift calculation, making the comparison always look unchanged. Here the actual success criterion is two fixtures with equal risk proportions but different volumes produce the same custom rate.
Question 100
Consider the following fictional Databricks production case: An engineering group duplicates a validation helper across six notebooks, and two copies enforce outdated threshold values. What underlying principle determines the most appropriate interpretation?
- Unit tests belong on controlled synthetic inputs before integrations, and production smoke tests do not replace them.
- Unit tests should exercise edge cases and expected output contracts, not just the happy-path fixture.
- Shared reusable validation code can prevent silent divergence between independent notebook copies.
- Model artifact identity must be checked through registry promotion and endpoint deployment, not inferred from job success.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Shared reusable validation code can prevent silent divergence between independent notebook copies. To test it, compare helper source hashes and test assertions across consuming notebooks.
Incorrect Answers
A:That underlying principle is directed at a different validation testing situation: A developer changes a feature transform in a notebook and directly uses production data as the only test case. The current incident instead turns on shared reusable validation code can prevent silent divergence between independent notebook copies.
B:That underlying principle is directed at a different validation testing situation: A feature function maps a null purchase amount to zero without distinguishing missing observations, and a notebook unit test covers only positive amounts. The current incident instead turns on shared reusable validation code can prevent silent divergence between independent notebook copies.
D:That underlying principle is directed at a different validation testing situation: A model version is registered but the deployment job points to an old alias after promotion and continues serving last week's model. The current incident instead turns on shared reusable validation code can prevent silent divergence between independent notebook copies.