Databricks Certified Machine Learning Professional Practice Test 16
Question 150
The engineering team encounters the following issue: A research team captures only daily batch inference totals with no row-level timestamps but insists on per-minute live inference drift analysis. Which technical principle most directly explains the root of the issue?
- Input distribution and predictive outcome quality are complementary monitoring signals with different evidence needs.
- Monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence.
- Alert payloads should carry sufficient metric and lineage context for safe triage.
- Drift claims require like-for-like cohort selection across current and reference populations.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence. To test it, inspect whether row-level event timestamps and prediction records actually exist for the desired minute windows.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A risk service measures feature drift but neglects observed-label performance and continues serving an accurate-looking yet ineffective model. The current incident instead turns on monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A drift alert fires but its message contains only the word anomaly without dataset, model version or baseline window. The current incident instead turns on monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A customer-risk monitoring job compares an unfiltered current high-risk segment to an all-customer historical baseline and interprets the difference as temporal drift. The current incident instead turns on monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence.
Question 151
A production ML review contains this situation: A media group has a specialized deep neural model that is not implemented as a SparkML estimator and assumes SparkML will automatically distribute its GPU training. What is the most material immediate risk in this situation?
- Pipeline fitting fails with missing-column or mismatched-schema errors.
- The experiment may run entirely on the driver and underutilize provisioned accelerators.
- The same raw category maps to a different vector position and produces misleading predictions.
- Future information leaks into training folds and overstates forecast usefulness.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. The experiment may run entirely on the driver and underutilize provisioned accelerators. The causal constraint is: SparkML distribution does not make every external estimator a natively distributed SparkML learner.
Incorrect Answers
A:That direct consequence is directed at a different using spark ml situation: A pipeline engineer fits a vectorizer after an estimator in the stage list and then tries to use the estimator's prediction as the vectorizer input. The more immediate risk here is the experiment may run entirely on the driver and underutilize provisioned accelerators.
C:That direct consequence is directed at a different using spark ml situation: A batch scoring service loads a classifier version from last month while using today's new feature indexer, silently changing categorical positions. The more immediate risk here is the experiment may run entirely on the driver and underutilize provisioned accelerators.
D:That direct consequence is directed at a different using spark ml situation: A time-ordered customer-retention dataset includes events from the future, yet a developer randomly cross-validates across time and then claims a forward-looking forecast is unbiased. The more immediate risk here is the experiment may run entirely on the driver and underutilize provisioned accelerators.
Question 152
The engineering team encounters the following issue: A client invokes a registered custom ML model using an unauthenticated HTTP POST to the serving endpoint and assumes network access implies authorization. Which technical principle most directly explains the root of the issue?
- Databricks Model Serving REST calls require the appropriate approved authentication and endpoint permissions.
- Model artifacts should not serialize secrets; serving identities should resolve credentials securely at runtime.
- Endpoint create/update calls and operational readiness are separate lifecycle checkpoints.
- Endpoint deployment requires controlled ownership and conflict-safe update/versioning practices.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Databricks Model Serving REST calls require the appropriate approved authentication and endpoint permissions. To test it, inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes.
Incorrect Answers
B:That underlying principle is directed at a different custom model serving situation: An engineer logs a customer-specific API token inside the pickled PythonModel object before registering it to Unity Catalog. The current incident instead turns on databricks Model Serving REST calls require the appropriate approved authentication and endpoint permissions.
C:That underlying principle is directed at a different custom model serving situation: A Databricks SDK deployment script sends a model-version update but does not wait for the endpoint to reach a usable ready state before announcing success. The current incident instead turns on databricks Model Serving REST calls require the appropriate approved authentication and endpoint permissions.
D:That underlying principle is directed at a different custom model serving situation: A team edits a model endpoint through the UI while its CI pipeline concurrently applies an older configuration and reverts the intended traffic split. The current incident instead turns on databricks Model Serving REST calls require the appropriate approved authentication and endpoint permissions.
Question 153
Review this production model incident: A Databricks SDK deployment script sends a model-version update but does not wait for the endpoint to reach a usable ready state before announcing success. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Compare training import versions and logged model environment metadata with serving import failures.
- Inspect actual test response JSON and registered output schema for a representative batch.
- Inspect deployment operation status, served entity health and known inference fixture after API completion.
- Inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect deployment operation status, served entity health and known inference fixture after API completion. The underlying technical issue is that endpoint create/update calls and operational readiness are separate lifecycle checkpoints.
Incorrect Answers
A:That diagnostic evidence is directed at a different custom model serving situation: A team logs custom MLflow model weights but forgets to include required Python package versions in the model environment. For the stated incident, the decisive observation is inspect deployment operation status, served entity health and known inference fixture after API completion.
B:That diagnostic evidence is directed at a different custom model serving situation: A serving client calls the REST endpoint expecting one number but the custom PyFunc returns a structured list of prediction and confidence fields. For the stated incident, the decisive observation is inspect deployment operation status, served entity health and known inference fixture after API completion.
D:That diagnostic evidence is directed at a different custom model serving situation: A client invokes a registered custom ML model using an unauthenticated HTTP POST to the serving endpoint and assumes network access implies authorization. For the stated incident, the decisive observation is inspect deployment operation status, served entity health and known inference fixture after API completion.
Question 154
A production ML review contains this situation: A single-node model training run spends almost all time swapping a 40 GB feature matrix on a 16 GB VM; no distributed version exists. What is the most material immediate risk in this situation?
- Each replica fails device memory allocation regardless of the number of added replicas.
- A relational workaround adds expensive jobs and loses explicit actor-state semantics.
- Training stalls on memory pressure even while unrelated cluster nodes are underutilized.
- A trial's metrics and model artifacts overwrite evidence from another configuration.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. Training stalls on memory pressure even while unrelated cluster nodes are underutilized. The causal constraint is: For a single-node algorithm, adequate local RAM or a compatible distributed alternative matters more than adding idle executors.
Incorrect Answers
A:That direct consequence is directed at a different scaling and tuning situation: A language model's weights cannot fit on one accelerator yet a team launches pure data-parallel replicas, each loading the entire model. The more immediate risk here is training stalls on memory pressure even while unrelated cluster nodes are underutilized.
B:That direct consequence is directed at a different scaling and tuning situation: A reinforcement-learning simulator requires stateful actors exchanging messages and irregular task graphs, but the team plans one Spark SQL groupBy per simulation step. The more immediate risk here is training stalls on memory pressure even while unrelated cluster nodes are underutilized.
D:That direct consequence is directed at a different scaling and tuning situation: An Optuna optimization reuses the same MLflow run ID for concurrent trials, causing overlapping parameter values and artifacts. The more immediate risk here is training stalls on memory pressure even while unrelated cluster nodes are underutilized.
Question 155
A production ML review contains this situation: A distributed retraining workflow uses ephemeral personal compute and fails when the original engineer's account is disabled. Which diagnostic check provides the strongest direct evidence?
- Compare staging and production endpoint, experiment and model registry settings against source-controlled configuration.
- Compare current official supported bundle resource fields with the pinned CLI version and existing bundle schema.
- Examine bundle validation results, actual resource permissions and downstream job execution traces.
- Inspect job ownership, compute policy and identity access after a simulated employee offboarding.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Inspect job ownership, compute policy and identity access after a simulated employee offboarding. The underlying technical issue is that production ML jobs need owned service identities, durable environments and resource configuration independent of individuals.
Incorrect Answers
A:That diagnostic evidence is directed at a different environment architectures situation: A model serving endpoint is created manually in the UI and every staging deployment differs subtly from production because no deployment definition is saved. For the stated incident, the decisive observation is inspect job ownership, compute policy and identity access after a simulated employee offboarding.
B:That diagnostic evidence is directed at a different environment architectures situation: An ML platform moves its configuration from older Databricks Asset Bundles terminology to newer Declarative Automation Bundles references and assumes every resource schema stayed identical. For the stated incident, the decisive observation is inspect job ownership, compute policy and identity access after a simulated employee offboarding.
C:That diagnostic evidence is directed at a different environment architectures situation: A continuous-delivery job deploys a notebook change through a bundle but skips validation of referenced experiment paths and job permissions. For the stated incident, the decisive observation is inspect job ownership, compute policy and identity access after a simulated employee offboarding.
Question 156
A production ML review contains this situation: An ML platform moves its configuration from older Databricks Asset Bundles terminology to newer Declarative Automation Bundles references and assumes every resource schema stayed identical. Which diagnostic check provides the strongest direct evidence?
- Map data classes, approved processing regions and cross-region feature access paths.
- Inspect job ownership, compute policy and identity access after a simulated employee offboarding.
- Compare current official supported bundle resource fields with the pinned CLI version and existing bundle schema.
- Review per-team compute usage, job queuing and capacity constraints during overlapping workloads.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Compare current official supported bundle resource fields with the pinned CLI version and existing bundle schema. The underlying technical issue is that product naming and resource schema changes should be checked against the active Databricks CLI and documentation for the deployed version.
Incorrect Answers
A:That diagnostic evidence is directed at a different environment architectures situation: A multilingual prediction service must handle regional data residency, but the team copies a shared feature cache across regions without governance. For the stated incident, the decisive observation is compare current official supported bundle resource fields with the pinned CLI version and existing bundle schema.
B:That diagnostic evidence is directed at a different environment architectures situation: A distributed retraining workflow uses ephemeral personal compute and fails when the original engineer's account is disabled. For the stated incident, the decisive observation is compare current official supported bundle resource fields with the pinned CLI version and existing bundle schema.
D:That diagnostic evidence is directed at a different environment architectures situation: A production deployment scales to 30 teams but all jobs share one compute pool with no capacity limits or cost attribution. For the stated incident, the decisive observation is compare current official supported bundle resource fields with the pinned CLI version and existing bundle schema.
Question 157
Consider the following fictional Databricks production case: A customer product-category feature acquires many new category values and analysts try to detect the change using only the overall row count. What underlying principle determines the most appropriate interpretation?
- Production input should sometimes be compared to a retained training reference, not only recent windows.
- A snapshot profile suits a general current table when no prediction or event-time model-monitoring contract exists.
- Alert windows and persistence conditions must align with the service objective to avoid transient false positives.
- Categorical distribution drift needs category frequency or supported categorical distance measures, not only volume.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. Categorical distribution drift needs category frequency or supported categorical distance measures, not only volume. To test it, compare baseline and current per-category shares, unseen categories and sample counts.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A model trained on last year's customer mix is monitored only against yesterday's already shifted cohort, so its original training-population mismatch is hidden. The current incident instead turns on categorical distribution drift needs category frequency or supported categorical distance measures, not only volume.
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: An analyst wants to inspect today’s static customer table for missing values and profile changes but chooses an inference monitor requiring model IDs and predictions. The current incident instead turns on categorical distribution drift needs category frequency or supported categorical distance measures, not only volume.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A monitor alerts whenever a ratio exceeds 0.3 for a single minute, although normal traffic produces brief spikes and operations require ten-minute persistence. The current incident instead turns on categorical distribution drift needs category frequency or supported categorical distance measures, not only volume.
Question 158
Consider the following fictional Databricks production case: A churn experiment separately scripts feature scaling in training but omits that scaling when the deployed prediction job loads the classifier. What consequence most specifically follows from leaving the issue unchanged?
- Rare but severe forecast misses can trigger inventory shortages while average reporting looks healthy.
- Predictions change because the deployed estimator receives unscaled columns it was never trained on.
- Complexity and cost increase while the tiny fit becomes slower rather than faster.
- The selected hyperparameter reflects an unrepeatable or misreported single holdout trial.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. Predictions change because the deployed estimator receives unscaled columns it was never trained on. The causal constraint is: Keeping fitted preprocessing and the estimator in one saved PipelineModel prevents training-serving transformation drift.
Incorrect Answers
A:That direct consequence is directed at a different using spark ml situation: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. The more immediate risk here is predictions change because the deployed estimator receives unscaled columns it was never trained on.
C:That direct consequence is directed at a different using spark ml situation: A laboratory runs a linear regression every quarter on only 280 clean observations and proposes a 48-worker Spark cluster to reduce a two-second local fit. The more immediate risk here is predictions change because the deployed estimator receives unscaled columns it was never trained on.
D:That direct consequence is directed at a different using spark ml situation: An ML engineer manually compares two logistic-regression regularization values on a single holdout but has no reusable SparkML parameter-search plan. The more immediate risk here is predictions change because the deployed estimator receives unscaled columns it was never trained on.
Question 159
A production ML review contains this situation: A model performance dashboard compares this week's labels to last week's predictions without joining by stable request ID or consistent outcome horizon. Which statement best explains the failure mode?
- Custom monitoring outputs should preserve necessary aggregate signals without exposing unnecessary entity-level sensitive data.
- An inference table needs correct prediction-to-label linkage for trustworthy observed performance metrics.
- Serving memory headroom and resource trends are necessary to detect OOM risk in large-model releases.
- An alert is operationally useful only with explicit routing, threshold, urgency and escalation ownership.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. An inference table needs correct prediction-to-label linkage for trustworthy observed performance metrics. To test it, audit request identifiers, event timestamps and join cardinality in the performance computation.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A monitor owner writes a custom metric that logs unaggregated account-level health data in a broadly shared results table. The current incident instead turns on an inference table needs correct prediction-to-label linkage for trustworthy observed performance metrics.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A new model version uses twice the memory per replica, but capacity monitoring tracks only CPU and crash counts. The current incident instead turns on an inference table needs correct prediction-to-label linkage for trustworthy observed performance metrics.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. The current incident instead turns on an inference table needs correct prediction-to-label linkage for trustworthy observed performance metrics.
Question 160
Review this production model incident: An image model needs 90 GB accelerator memory for one training replica while each available GPU has 24 GB and engineers propose only adding more identical nodes. Which failure risk follows most closely from the unresolved behavior?
- Unnecessary model partitioning adds network latency with little memory benefit.
- The rewritten job runs only on one worker or fails during model serialization.
- Every replica hits device OOM even though aggregate cluster memory appears ample.
- Multiple trials contend for the same accelerator and fail with memory exhaustion.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. Every replica hits device OOM even though aggregate cluster memory appears ample. The causal constraint is: Horizontal replica count alone does not combine GPU memory for one model without a supported parallelization strategy.
Incorrect Answers
A:That direct consequence is directed at a different scaling and tuning situation: A tabular classifier fits comfortably on one GPU and 100 million independent examples are available, but a team proposes splitting the small model's layers across hosts. The more immediate risk here is every replica hits device OOM even though aggregate cluster memory appears ample.
B:That direct consequence is directed at a different scaling and tuning situation: A team switches a SparkML training stage to Ray without verifying that its chosen estimator supports Ray's distributed fit contracts. The more immediate risk here is every replica hits device OOM even though aggregate cluster memory appears ample.
D:That direct consequence is directed at a different scaling and tuning situation: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. The more immediate risk here is every replica hits device OOM even though aggregate cluster memory appears ample.