Databricks Certified Machine Learning Professional Practice Test 65
Question 640
Review this production model incident: A retail profile is created on a static materialized daily inventory table; a team expects its initial snapshot to show each prior hour as a separate trend. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Inspect whether row-level event timestamps and prediction records actually exist for the desired minute windows.
- Review rolling-window statistics alongside a fixed approved baseline for the same cohort.
- Inspect the stored snapshot rows and timestamps to determine whether historical hourly observations are retained.
- Compare null counts and prefilter event totals between baseline and current windows.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect the stored snapshot rows and timestamps to determine whether historical hourly observations are retained. The underlying technical issue is that snapshot profiles describe current table state rather than reconstruct missing hourly time-series history.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A research team captures only daily batch inference totals with no row-level timestamps but insists on per-minute live inference drift analysis. For the stated incident, the decisive observation is inspect the stored snapshot rows and timestamps to determine whether historical hourly observations are retained.
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. For the stated incident, the decisive observation is inspect the stored snapshot rows and timestamps to determine whether historical hourly observations are retained.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A nightly scoring feed drops all null-feature rows before calculating drift, making missingness jump invisible. For the stated incident, the decisive observation is inspect the stored snapshot rows and timestamps to determine whether historical hourly observations are retained.
Question 641
Consider the following fictional Databricks production case: A global model predicts late deliveries accurately overall, but all mistakes occur at one small coastal depot and the team never slices its results by site. What consequence most specifically follows from leaving the issue unchanged?
- The affected depot stays poorly served while global metrics appear stable.
- Instances restart unpredictably from memory pressure and disrupt scoring.
- Errors are assigned to the wrong predictions and reported performance becomes meaningless.
- Routine weekend traffic launches unnecessary retraining and distracts engineers.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. The affected depot stays poorly served while global metrics appear stable. The causal constraint is: Slicing on operationally meaningful locations can reveal concentrated failures hidden in the global metric.
Incorrect Answers
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A new model version uses twice the memory per replica, but capacity monitoring tracks only CPU and crash counts. The more immediate risk here is the affected depot stays poorly served while global metrics appear stable.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. The more immediate risk here is the affected depot stays poorly served while global metrics appear stable.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. The more immediate risk here is the affected depot stays poorly served while global metrics appear stable.
Question 642
The engineering team encounters the following issue: A media group has a specialized deep neural model that is not implemented as a SparkML estimator and assumes SparkML will automatically distribute its GPU training. Which experiment most directly demonstrates a successful fix?
- Every expected input route has exactly the intended output score after a distributed run with bounded driver memory.
- The training trace identifies worker-side execution and supported framework behavior for the selected model.
- The full fitted pipeline completes on a small test DataFrame and exposes the planned input and output columns.
- The selected candidate has the minimum validated RMSE among comparable parameter configurations.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. The training trace identifies worker-side execution and supported framework behavior for the selected model. It confirms whether the targeted action succeeded: select a supported distributed training framework for that model or a deliberate single-node fit rather than relabeling it SparkML.
Incorrect Answers
A:That acceptance test is directed at a different using spark ml situation: A logistics model must score 60 million daily routes already in Delta but the scoring script collects each route into Python before calling predict. Here the actual success criterion is the training trace identifies worker-side execution and supported framework behavior for the selected model.
C:That acceptance test is directed at a different using spark ml situation: A pipeline engineer fits a vectorizer after an estimator in the stage list and then tries to use the estimator's prediction as the vectorizer input. Here the actual success criterion is the training trace identifies worker-side execution and supported framework behavior for the selected model.
D:That acceptance test is directed at a different using spark ml situation: A SparkML grid-search experiment selects the model with the largest error metric even though the evaluator reports RMSE to minimize. Here the actual success criterion is the training trace identifies worker-side execution and supported framework behavior for the selected model.
Question 643
A production ML review contains this situation: After choosing the best parameter map, a data scientist reports the same cross-validation score as an independent final test result without any untouched data. Which statement best explains the failure mode?
- A fitted model can transform streaming DataFrames in a supported Structured Streaming pipeline, but a one-time static read is not a continuing source.
- Tokenizer is a Transformer and applies transform directly without a separate learned fit step.
- Hyperparameter selection uses validation evidence; the final unbiased estimate requires an independent untouched test split.
- RMSE squares residuals and highlights large errors, while median error alone may obscure expensive outliers.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Hyperparameter selection uses validation evidence; the final unbiased estimate requires an independent untouched test split. To test it, trace which observations were used by parameter selection and which produced the reported final score.
Incorrect Answers
A:That underlying principle is directed at a different using spark ml situation: A streaming anomaly team uses a static read of a message directory once, believing the trained SparkML model will continuously produce scores for future arriving events. The current incident instead turns on hyperparameter selection uses validation evidence; the final unbiased estimate requires an independent untouched test split.
B:That underlying principle is directed at a different using spark ml situation: A text pipeline repeatedly fits a Tokenizer stage even though tokenization simply converts strings into token arrays without learning a model. The current incident instead turns on hyperparameter selection uses validation evidence; the final unbiased estimate requires an independent untouched test split.
D:That underlying principle is directed at a different using spark ml situation: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. The current incident instead turns on hyperparameter selection uses validation evidence; the final unbiased estimate requires an independent untouched test split.
Question 644
In an enterprise ML workflow, the reported issue is: A production deployment scales to 30 teams but all jobs share one compute pool with no capacity limits or cost attribution. Which change addresses the real cause without undermining other requirements?
- Apply approved workload isolation, compute policies, quotas and cost attribution for training versus serving.
- Migrate or retain the configuration only after verifying current supported fields and running an explicit bundle validation.
- Define the supported model-serving and MLflow resources declaratively with validated environment targets.
- Add bundle validation, permission checks and staging smoke tests before production promotion.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Apply approved workload isolation, compute policies, quotas and cost attribution for training versus serving. The issue matters because one experiment starves critical retraining or inference workloads.
Incorrect Answers
B:That corrective action is directed at a different environment architectures situation: An ML platform moves its configuration from older Databricks Asset Bundles terminology to newer Declarative Automation Bundles references and assumes every resource schema stayed identical. The relevant corrective action here is apply approved workload isolation, compute policies, quotas and cost attribution for training versus serving.
C:That corrective action is directed at a different environment architectures situation: A model serving endpoint is created manually in the UI and every staging deployment differs subtly from production because no deployment definition is saved. The relevant corrective action here is apply approved workload isolation, compute policies, quotas and cost attribution for training versus serving.
D:That corrective action is directed at a different environment architectures situation: A continuous-delivery job deploys a notebook change through a bundle but skips validation of referenced experiment paths and job permissions. The relevant corrective action here is apply approved workload isolation, compute policies, quotas and cost attribution for training versus serving.
Question 645
Review this production model incident: One enterprise account holds 80% of historical events, and grouped pandas training for that account exceeds executor memory while other accounts finish. What is the central engineering constraint behind this behavior?
- Data parallelism is often more natural than model-parallel networking for a small model with large independent data.
- Ray Tune can schedule independent hyperparameter trials with explicit per-trial resources rather than serial execution.
- Vertical versus horizontal scaling decisions must consider dataset size, execution architecture and fixed coordination overhead.
- The largest per-group pandas object, not average group size, controls peak memory for grouped model fits.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. The largest per-group pandas object, not average group size, controls peak memory for grouped model fits. To test it, inspect per-account cardinality and worker memory/OOM traces for the failing group.
Incorrect Answers
A:That underlying principle is directed at a different scaling and tuning situation: A tabular classifier fits comfortably on one GPU and 100 million independent examples are available, but a team proposes splitting the small model's layers across hosts. The current incident instead turns on the largest per-group pandas object, not average group size, controls peak memory for grouped model fits.
B:That underlying principle is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. The current incident instead turns on the largest per-group pandas object, not average group size, controls peak memory for grouped model fits.
C:That underlying principle is directed at a different scaling and tuning situation: A moderately sized dataset fits on one worker but a team provisions a huge distributed cluster and observes overhead higher than model fit. The current incident instead turns on the largest per-group pandas object, not average group size, controls peak memory for grouped model fits.
Question 646
Consider the following fictional Databricks production case: A baseline table is overwritten every night with the current day's data before a drift calculation, making the comparison always look unchanged. What underlying principle determines the most appropriate interpretation?
- A fixed baseline must be versioned and immutable for the intended comparison period.
- Observability starts with traceable, appropriately governed prediction logging before statistical analysis.
- Subgroup metrics should enforce privacy and minimum-sample limits before strong comparative claims.
- An inference table needs correct prediction-to-label linkage for trustworthy observed performance metrics.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. A fixed baseline must be versioned and immutable for the intended comparison period. To test it, inspect baseline table versions and the order of writes relative to metric computations.
Incorrect Answers
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A serving endpoint appears healthy but nobody records model version, request identifier or prediction value, so a drift incident cannot be reconstructed. The current incident instead turns on a fixed baseline must be versioned and immutable for the intended comparison period.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A demographic fairness review uses a tiny sensitive subgroup with three labeled examples and reports its observed error as definitive. The current incident instead turns on a fixed baseline must be versioned and immutable for the intended comparison period.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. The current incident instead turns on a fixed baseline must be versioned and immutable for the intended comparison period.
Question 647
In an enterprise ML workflow, the reported issue is: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. Before modifying the pipeline, which evidence should be collected first?
- Inspect metric reference table identity and original model training-set distribution for the deployed version.
- Inspect the table schema for presence of model, prediction and timestamp columns versus the requested snapshot statistics.
- Inspect alert query configuration, notification destinations and missed alert delivery records.
- Check label availability, label delay and prediction-to-label matching IDs in the inference data.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect alert query configuration, notification destinations and missed alert delivery records. The underlying technical issue is that an alert is operationally useful only with explicit routing, threshold, urgency and escalation ownership.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A model trained on last year's customer mix is monitored only against yesterday's already shifted cohort, so its original training-population mismatch is hidden. For the stated incident, the decisive observation is inspect alert query configuration, notification destinations and missed alert delivery records.
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An analyst wants to inspect today’s static customer table for missing values and profile changes but chooses an inference monitor requiring model IDs and predictions. For the stated incident, the decisive observation is inspect alert query configuration, notification destinations and missed alert delivery records.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A model's inference table records output probabilities but omits ground-truth labels, yet reviewers expect immediate precision and recall metrics. For the stated incident, the decisive observation is inspect alert query configuration, notification destinations and missed alert delivery records.
Question 648
Review this production model incident: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Compare P95/P99 latency by request volume and concurrency at the same sampling interval.
- Inspect source timestamps and selected monitoring timezone for affected geographic cohorts.
- Compare high-risk event counts against all eligible scored records for several differing traffic volumes.
- Inspect inference table version IDs and label-matched errors before and after traffic shift.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Inspect inference table version IDs and label-matched errors before and after traffic shift. The underlying technical issue is that performance trends require model-version provenance to attribute changes to the correct release.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A custom model endpoint meets mean response time of 75 milliseconds but P99 latency reaches 1.8 seconds during peak load. For the stated incident, the decisive observation is inspect inference table version IDs and label-matched errors before and after traffic shift.
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A model's prediction error appears to spike at 2 a.m. local time but the monitoring table groups timestamps in UTC for a globally distributed region. For the stated incident, the decisive observation is inspect inference table version IDs and label-matched errors before and after traffic shift.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A monitoring team wants a business metric for the share of transactions exceeding a risk threshold but records only the raw count of high-risk transactions. For the stated incident, the decisive observation is inspect inference table version IDs and label-matched errors before and after traffic shift.
Question 649
In an enterprise ML workflow, the reported issue is: An experiment tunes 40 candidates but logs each trial as an unrelated root MLflow run, so reviewers cannot reconstruct which search produced a winner. Before modifying the pipeline, which evidence should be collected first?
- Inspect logged model environment files and serving import errors for the missing package.
- Inspect parent_run_id tags, candidate run IDs and the experiment's tracked search configuration.
- Compare training feature timestamp definitions with live feature retrieval windows on a matched historical request.
- Compare run parameter maps against logged evaluation metrics and the local experiment output.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Inspect parent_run_id tags, candidate run IDs and the experiment's tracked search configuration. The underlying technical issue is that parent and nested child runs encode experiment-to-trial relationships while preserving each trial's parameters and metrics.
Incorrect Answers
A:That diagnostic evidence is directed at a different advanced mlflow usage situation: A team logs a custom Python model that imports a package installed only in the author's notebook, and the serving endpoint fails when loading the artifact. For the stated incident, the decisive observation is inspect parent_run_id tags, candidate run IDs and the experiment's tracked search configuration.
C:That diagnostic evidence is directed at a different advanced mlflow usage situation: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. For the stated incident, the decisive observation is inspect parent_run_id tags, candidate run IDs and the experiment's tracked search configuration.
D:That diagnostic evidence is directed at a different advanced mlflow usage situation: A training notebook reports F1 for several models but only logs the final model's parameters in MLflow, leaving metrics and hyperparameters disconnected. For the stated incident, the decisive observation is inspect parent_run_id tags, candidate run IDs and the experiment's tracked search configuration.
Question 650
A production ML review contains this situation: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. Which diagnostic check provides the strongest direct evidence?
- Inspect model class and stored coefficients against the fit-versus-transform calls in the scoring job.
- Trace when the indexer fit was called relative to the train/validation split and compare observed label inventories.
- Measure the local estimator runtime against cluster initialization and task scheduling for the same sample.
- Review predicted probabilities, a precision-recall threshold curve and analyst capacity at candidate cutoffs.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Trace when the indexer fit was called relative to the train/validation split and compare observed label inventories. The underlying technical issue is that any fitted preprocessing stage can learn from holdout records and must be fitted within the training boundary.
Incorrect Answers
A:That diagnostic evidence is directed at a different using spark ml situation: An analyst loads a fitted LogisticRegressionModel but calls fit on it for every scored batch rather than transform. For the stated incident, the decisive observation is trace when the indexer fit was called relative to the train/validation split and compare observed label inventories.
C:That diagnostic evidence is directed at a different using spark ml situation: A laboratory runs a linear regression every quarter on only 280 clean observations and proposes a 48-worker Spark cluster to reduce a two-second local fit. For the stated incident, the decisive observation is trace when the indexer fit was called relative to the train/validation split and compare observed label inventories.
D:That diagnostic evidence is directed at a different using spark ml situation: A binary classifier ranks risky cases well but sends too many alerts at a default 0.5 decision threshold. For the stated incident, the decisive observation is trace when the indexer fit was called relative to the train/validation split and compare observed label inventories.