Databricks Certified Machine Learning Professional Practice Test 34
Question 330
The engineering team encounters the following issue: A retailer trains a demand classifier on 180 million Delta rows already partitioned across a Spark cluster, but an analyst attempts to convert the entire training set to a local pandas object. What investigation would most effectively isolate this behavior?
- Compare absolute and squared residual distributions and inspect the high-error tail on holdout data.
- Compare training-set byte size with driver memory and inspect Spark task distribution versus the pandas conversion plan.
- Inspect its declared output column and confirm the downstream estimator consumes a vector rather than predicted labels.
- Inspect fitted category mappings, handleInvalid behavior and a sample newly introduced category.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Compare training-set byte size with driver memory and inspect Spark task distribution versus the pandas conversion plan. The underlying technical issue is that distributed feature transformations and a SparkML estimator avoid mandatory collection of the full training set on one driver.
Incorrect Answers
A:That diagnostic evidence is directed at a different using spark ml situation: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. For the stated incident, the decisive observation is compare training-set byte size with driver memory and inspect Spark task distribution versus the pandas conversion plan.
C:That diagnostic evidence is directed at a different using spark ml situation: A team wants to convert several numeric inputs into one features vector but treats VectorAssembler as a clustering Estimator that should output predictions. For the stated incident, the decisive observation is compare training-set byte size with driver memory and inspect Spark task distribution versus the pandas conversion plan.
D:That diagnostic evidence is directed at a different using spark ml situation: An inference pipeline sends new categorical labels unseen during training to StringIndexerModel with error handling configured to reject unknown categories. For the stated incident, the decisive observation is compare training-set byte size with driver memory and inspect Spark task distribution versus the pandas conversion plan.
Question 331
In an enterprise ML workflow, the reported issue is: An AutoML training job always chooses the model with the lowest training loss, even when the deployed service has a strict memory and P95 latency budget. What technical distinction is essential to this investigation?
- Data pipeline outages must be distinguished from genuine population shift before retraining.
- Retraining triggers and production promotion are different stages with separate evaluation and rollout gates.
- Production model selection must satisfy deployment constraints as well as validation quality.
- Ground-truth delay constrains performance-triggered retraining; fresh unlabeled predictions cannot establish predictive accuracy.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Production model selection must satisfy deployment constraints as well as validation quality. To test it, compare candidate predictive metrics with model footprint, serving latency and expected concurrency.
Incorrect Answers
A:That underlying principle is directed at a different automated retraining situation: A feature stream loses half its source events and a retraining policy treats the resulting input shift as proof that the learned concept changed. The current incident instead turns on production model selection must satisfy deployment constraints as well as validation quality.
B:That underlying principle is directed at a different automated retraining situation: An alert from a feature monitor automatically replaces the live model with a newly trained artifact without testing or approval. The current incident instead turns on production model selection must satisfy deployment constraints as well as validation quality.
D:That underlying principle is directed at a different automated retraining situation: A credit model's true-label feedback arrives 45 days late, yet the retraining scheduler assumes yesterday's unlabeled predictions provide yesterday's observed accuracy. The current incident instead turns on production model selection must satisfy deployment constraints as well as validation quality.
Question 332
Review this production model incident: A real-time feature job updates user purchase totals without checking whether upstream transaction events are delivered twice. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Audit token scopes and endpoint access logs against the feature tables accessed by the serving model.
- Check entity/timestamp duplicate counts around job retries and writer transaction history.
- Compare source event IDs and resulting counter increments for an intentionally duplicated transaction.
- Inspect feature update timestamps against label timestamps for a sampled joined row.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Compare source event IDs and resulting counter increments for an intentionally duplicated transaction. The underlying technical issue is that streaming derived features should deduplicate at a stable business-event key before non-idempotent aggregation.
Incorrect Answers
A:That diagnostic evidence is directed at a different advanced feature store concepts situation: A model serving service uses a broadly privileged shared token to read every online feature table when it needs only one approved feature set. For the stated incident, the decisive observation is compare source event IDs and resulting counter increments for an intentionally duplicated transaction.
B:That diagnostic evidence is directed at a different advanced feature store concepts situation: A feature refresh writes duplicate records for each entity and logical timestamp after retries, breaking unique lookups. For the stated incident, the decisive observation is compare source event IDs and resulting counter increments for an intentionally duplicated transaction.
D:That diagnostic evidence is directed at a different advanced feature store concepts situation: A fraud training row is labeled on March 10 but an ordinary entity join attaches an account balance updated on March 12. For the stated incident, the decisive observation is compare source event IDs and resulting counter increments for an intentionally duplicated transaction.
Question 333
Review this production model incident: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. What is the central engineering constraint behind this behavior?
- Drift claims require like-for-like cohort selection across current and reference populations.
- Model health, performance and data drift are different failure categories that need separate investigation paths.
- Aggregate performance can conceal failure in important subpopulations and lifecycle stages.
- Subgroup metrics should enforce privacy and minimum-sample limits before strong comparative claims.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Aggregate performance can conceal failure in important subpopulations and lifecycle stages. To test it, compare labeled performance for new and established users at the same evaluation horizon.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A customer-risk monitoring job compares an unfiltered current high-risk segment to an all-customer historical baseline and interprets the difference as temporal drift. The current incident instead turns on aggregate performance can conceal failure in important subpopulations and lifecycle stages.
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: An endpoint's error rate spikes while data profiles look normal, and responders attempt to retrain the model instead of checking serving infrastructure. The current incident instead turns on aggregate performance can conceal failure in important subpopulations and lifecycle stages.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A demographic fairness review uses a tiny sensitive subgroup with three labeled examples and reports its observed error as definitive. The current incident instead turns on aggregate performance can conceal failure in important subpopulations and lifecycle stages.
Question 334
Review this production model incident: An Optuna search runs 80 hyperparameter trials but records only the last winning score in MLflow, making comparisons impossible. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Benchmark single-node and distributed alternatives using matched data, quality metrics and total compute cost.
- Compare GPU inventory with the current trial task graph, resource reservations and wait times.
- Inspect the Optuna trial table and compare its IDs and metrics against MLflow runs.
- Compare operation requirements, data conversion cost and supported query optimizations in both engines.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect the Optuna trial table and compare its IDs and metrics against MLflow runs. The underlying technical issue is that reliable tuning provenance requires parameter, metric and trial identity logging for every evaluated trial.
Incorrect Answers
A:That diagnostic evidence is directed at a different scaling and tuning situation: A moderately sized dataset fits on one worker but a team provisions a huge distributed cluster and observes overhead higher than model fit. For the stated incident, the decisive observation is inspect the Optuna trial table and compare its IDs and metrics against MLflow runs.
B:That diagnostic evidence is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. For the stated incident, the decisive observation is inspect the Optuna trial table and compare its IDs and metrics against MLflow runs.
D:That diagnostic evidence is directed at a different scaling and tuning situation: A team already has a vectorized Spark DataFrame pipeline and proposes converting everything to Ray datasets solely to perform a simple SQL aggregation. For the stated incident, the decisive observation is inspect the Optuna trial table and compare its IDs and metrics against MLflow runs.
Question 335
A production ML review contains this situation: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. What is the most material immediate risk in this situation?
- Reported performance is optimistically biased because the evaluation influenced configuration choice.
- Rare but severe forecast misses can trigger inventory shortages while average reporting looks healthy.
- The workflow fails or includes unnecessary invalid fit calls that cannot create a tokenizer model.
- Interactive scoring times out when each request launches heavy offline data scans.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. Rare but severe forecast misses can trigger inventory shortages while average reporting looks healthy. The causal constraint is: RMSE squares residuals and highlights large errors, while median error alone may obscure expensive outliers.
Incorrect Answers
A:That direct consequence is directed at a different using spark ml situation: After choosing the best parameter map, a data scientist reports the same cross-validation score as an independent final test result without any untouched data. The more immediate risk here is rare but severe forecast misses can trigger inventory shortages while average reporting looks healthy.
C:That direct consequence is directed at a different using spark ml situation: A text pipeline repeatedly fits a Tokenizer stage even though tokenization simply converts strings into token arrays without learning a model. The more immediate risk here is rare but severe forecast misses can trigger inventory shortages while average reporting looks healthy.
D:That direct consequence is directed at a different using spark ml situation: A mobile app requires thousands of concurrent low-latency scores with online features, but a team plans to query an offline historical feature Delta table for every request. The more immediate risk here is rare but severe forecast misses can trigger inventory shortages while average reporting looks healthy.
Question 336
In an enterprise ML workflow, the reported issue is: An ML engineer logs a calibration plot as a string metric, then downstream reviewers cannot inspect the visualization or its input version. Before modifying the pipeline, which evidence should be collected first?
- Inspect the run's metric keys versus artifact store and recover the expected plot file.
- Inspect logged model environment files and serving import errors for the missing package.
- Compare original search ID, resumed run tags and candidate-set coverage after restoration.
- Inspect parent_run_id tags, candidate run IDs and the experiment's tracked search configuration.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Inspect the run's metric keys versus artifact store and recover the expected plot file. The underlying technical issue is that complex objects such as plots and reports belong in versioned MLflow artifacts rather than scalar metrics.
Incorrect Answers
B:That diagnostic evidence is directed at a different advanced mlflow usage situation: A team logs a custom Python model that imports a package installed only in the author's notebook, and the serving endpoint fails when loading the artifact. For the stated incident, the decisive observation is inspect the run's metric keys versus artifact store and recover the expected plot file.
C:That diagnostic evidence is directed at a different advanced mlflow usage situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. For the stated incident, the decisive observation is inspect the run's metric keys versus artifact store and recover the expected plot file.
D:That diagnostic evidence is directed at a different advanced mlflow usage situation: An experiment tunes 40 candidates but logs each trial as an unrelated root MLflow run, so reviewers cannot reconstruct which search produced a winner. For the stated incident, the decisive observation is inspect the run's metric keys versus artifact store and recover the expected plot file.
Question 337
Review this production model incident: A monitor owner writes a custom metric that logs unaggregated account-level health data in a broadly shared results table. Which failure risk follows most closely from the unresolved behavior?
- Users receive no predictions even though the registered model metrics look excellent.
- A regression in v8 is masked by higher-volume predictions from v7.
- The monitoring pipeline either floods the SOC or ignores new material shifts.
- Observability becomes a channel for customer-level data leakage.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. Observability becomes a channel for customer-level data leakage. The causal constraint is: Custom monitoring outputs should preserve necessary aggregate signals without exposing unnecessary entity-level sensitive data.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A prediction endpoint returns HTTP 503 for intermittent bursts while the model's offline accuracy remains unchanged. The more immediate risk here is observability becomes a channel for customer-level data leakage.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. The more immediate risk here is observability becomes a channel for customer-level data leakage.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A production alert threshold is hard-coded against last year's baseline, but seasonal traffic has shifted and the owner never reviewed the threshold. The more immediate risk here is observability becomes a channel for customer-level data leakage.
Question 338
Review this production model incident: A monitor alerts whenever a ratio exceeds 0.3 for a single minute, although normal traffic produces brief spikes and operations require ten-minute persistence. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Inspect inference table version IDs and label-matched errors before and after traffic shift.
- Inspect timestamp type, event cadence and intended daily windows before choosing the monitor.
- Compare spike duration and alert time window against the approved ten-minute persistence rule.
- Inspect inference-record completeness for request, model version, inputs, outputs and timestamps.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Compare spike duration and alert time window against the approved ten-minute persistence rule. The underlying technical issue is that alert windows and persistence conditions must align with the service objective to avoid transient false positives.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. For the stated incident, the decisive observation is compare spike duration and alert time window against the approved ten-minute persistence rule.
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A fleet sensor table records measurement timestamps and analysts need daily drift trends; a snapshot-only profile erases the time dimension. For the stated incident, the decisive observation is compare spike duration and alert time window against the approved ten-minute persistence rule.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A serving endpoint appears healthy but nobody records model version, request identifier or prediction value, so a drift incident cannot be reconstructed. For the stated incident, the decisive observation is compare spike duration and alert time window against the approved ten-minute persistence rule.
Question 339
The engineering team encounters the following issue: An Optuna search runs 80 hyperparameter trials but records only the last winning score in MLflow, making comparisons impossible. Which experiment most directly demonstrates a successful fix?
- The Spark solution meets output and performance needs without intermediate Ray materialization.
- The supported sharded model initializes and completes a forward/backward training step without OOM.
- Measured model updates follow the supported synchronization contract across workers.
- Every completed trial has a linked run, parameter map, metric and deterministic winner selection.
Explanation
Correct Answer
D:This verification measures the required result rather than just process liveness. Every completed trial has a linked run, parameter map, metric and deterministic winner selection. It confirms whether the targeted action succeeded: connect supported Optuna study execution to MLflow per-trial tracking and preserve the selected objective value.
Incorrect Answers
A:That acceptance test is directed at a different scaling and tuning situation: A team already has a vectorized Spark DataFrame pipeline and proposes converting everything to Ray datasets solely to perform a simple SQL aggregation. Here the actual success criterion is every completed trial has a linked run, parameter map, metric and deterministic winner selection.
B:That acceptance test is directed at a different scaling and tuning situation: A language model's weights cannot fit on one accelerator yet a team launches pure data-parallel replicas, each loading the entire model. Here the actual success criterion is every completed trial has a linked run, parameter map, metric and deterministic winner selection.
C:That acceptance test is directed at a different scaling and tuning situation: An ML engineer tries to parallelize a stateful neural-training loop by executing independent mapPartitions calls that share one mutable global model in the driver. Here the actual success criterion is every completed trial has a linked run, parameter map, metric and deterministic winner selection.
Question 340
The engineering team encounters the following issue: A hyperparameter team compares two Optuna studies with different train/validation splits and chooses the lower loss as though both metrics were directly comparable. What corrective action should the engineer prefer?
- Tune outer concurrency based on actual cluster capacity and each trial's resource demand.
- Use supported data parallelism with appropriately batched examples and convergence checks.
- Choose a measured right-sized execution mode instead of presuming more nodes means faster fit.
- Pin shared validation protocol and data snapshot for the compared Optuna trials.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Pin shared validation protocol and data snapshot for the compared Optuna trials. The issue matters because the apparent improvement is a data-split artifact rather than a better model.
Incorrect Answers
A:That corrective action is directed at a different scaling and tuning situation: An operations model uses a SparkML estimator that itself distributes data, yet the team launches dozens of concurrent full-cluster fits on the same fixed small cluster. The relevant corrective action here is pin shared validation protocol and data snapshot for the compared Optuna trials.
B:That corrective action is directed at a different scaling and tuning situation: A tabular classifier fits comfortably on one GPU and 100 million independent examples are available, but a team proposes splitting the small model's layers across hosts. The relevant corrective action here is pin shared validation protocol and data snapshot for the compared Optuna trials.
C:That corrective action is directed at a different scaling and tuning situation: A moderately sized dataset fits on one worker but a team provisions a huge distributed cluster and observes overhead higher than model fit. The relevant corrective action here is pin shared validation protocol and data snapshot for the compared Optuna trials.