Databricks Certified Machine Learning Professional Practice Test 24
Question 230
Consider the following fictional Databricks production case: A production endpoint scales CPU instances up when traffic rises, but request queues and P95 response time keep growing despite low CPU usage. Which observation most directly tests the suspected failure boundary?
- Inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.
- Stratify historical samples by day type and examine whether the observed difference is expected seasonality.
- Inspect inference table version IDs and label-matched errors before and after traffic shift.
- Inspect alert query configuration, notification destinations and missed alert delivery records.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage. The underlying technical issue is that endpoint throughput can be constrained by queueing, request concurrency, external features or rate limits beyond CPU utilization.
Incorrect Answers
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. For the stated incident, the decisive observation is inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. For the stated incident, the decisive observation is inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. For the stated incident, the decisive observation is inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.
Question 231
Consider the following fictional Databricks production case: An insurer already stores its risk features as a large Spark DataFrame but exports them into Python lists before every vector transformation and repeatedly hits out-of-memory errors. Which observation most directly tests the suspected failure boundary?
- Review predicted probabilities, a precision-recall threshold curve and analyst capacity at candidate cutoffs.
- Inspect DataFrame physical operations and memory spikes at each Python collection boundary.
- Compare the original StringIndexer type with the fitted StringIndexerModel and inspect its label mapping.
- Inspect evaluator.isLargerBetter or equivalent metric semantics alongside the ordered candidate RMSE values.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Inspect DataFrame physical operations and memory spikes at each Python collection boundary. The underlying technical issue is that repeated driver collection defeats the distributed processing advantages relevant to large SparkML feature pipelines.
Incorrect Answers
A:That diagnostic evidence is directed at a different using spark ml situation: A binary classifier ranks risky cases well but sends too many alerts at a default 0.5 decision threshold. For the stated incident, the decisive observation is inspect DataFrame physical operations and memory spikes at each Python collection boundary.
C:That diagnostic evidence is directed at a different using spark ml situation: A new Spark developer calls transform on an unfitted StringIndexer instance and expects learned category labels to appear automatically. For the stated incident, the decisive observation is inspect DataFrame physical operations and memory spikes at each Python collection boundary.
D:That diagnostic evidence is directed at a different using spark ml situation: A SparkML grid-search experiment selects the model with the largest error metric even though the evaluator reports RMSE to minimize. For the stated incident, the decisive observation is inspect DataFrame physical operations and memory spikes at each Python collection boundary.
Question 232
Consider the following fictional Databricks production case: An end-to-end validation uses only a synthetic positive label and fails to detect a swapped positive/negative class mapping in inference. Which implementation is most appropriate for the stated incident?
- Extract testable pure functions into a reusable module and keep notebook orchestration thin.
- Require isolated unit tests in development followed by separate integration and production smoke gates.
- Implement explicit timezone-aware parsing and add negative tests for ambiguous inputs.
- Include paired known-label fixtures and assert prediction class semantics and output ordering.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Include paired known-label fixtures and assert prediction class semantics and output ordering. The issue matters because a valid-looking endpoint reverses class interpretation for downstream automation.
Incorrect Answers
A:That corrective action is directed at a different validation testing situation: A team hides all preprocessing logic inside one long notebook, making it hard to test independently without starting a Spark cluster. The relevant corrective action here is include paired known-label fixtures and assert prediction class semantics and output ordering.
B:That corrective action is directed at a different validation testing situation: A developer changes a feature transform in a notebook and directly uses production data as the only test case. The relevant corrective action here is include paired known-label fixtures and assert prediction class semantics and output ordering.
C:That corrective action is directed at a different validation testing situation: A date feature function treats all incoming timestamp strings as UTC even though some include explicit offsets and a unit test uses only Z-suffixed inputs. The relevant corrective action here is include paired known-label fixtures and assert prediction class semantics and output ordering.
Question 233
The engineering team encounters the following issue: A Databricks SDK deployment script sends a model-version update but does not wait for the endpoint to reach a usable ready state before announcing success. Which experiment most directly demonstrates a successful fix?
- An endpoint reaches a ready state and a test prediction succeeds before the pipeline marks the release complete.
- An artifact scan contains no credential strings while authorized serving requests still work.
- A newly registered version is discoverable under the correct catalog/schema with its tracked source run.
- A saved two-record request returns correctly ordered predictions with the expected schema.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. An endpoint reaches a ready state and a test prediction succeeds before the pipeline marks the release complete. It confirms whether the targeted action succeeded: poll or otherwise verify documented readiness, then perform a permitted smoke prediction before marking the release ready.
Incorrect Answers
B:That acceptance test is directed at a different custom model serving situation: An engineer logs a customer-specific API token inside the pickled PythonModel object before registering it to Unity Catalog. Here the actual success criterion is an endpoint reaches a ready state and a test prediction succeeds before the pipeline marks the release complete.
C:That acceptance test is directed at a different custom model serving situation: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. Here the actual success criterion is an endpoint reaches a ready state and a test prediction succeeds before the pipeline marks the release complete.
D:That acceptance test is directed at a different custom model serving situation: A notebook uses the MLflow Deployments SDK to send a column-major payload to a custom model endpoint expecting a different documented dataframe schema. Here the actual success criterion is an endpoint reaches a ready state and a test prediction succeeds before the pipeline marks the release complete.
Question 234
In an enterprise ML workflow, the reported issue is: A Databricks Model Serving endpoint hosts champion and candidate versions, but a release config accidentally routes 100% of traffic to the candidate instead of 10%. Before modifying the pipeline, which evidence should be collected first?
- Check independent health and version identity for both active and standby deployment configurations before cutover.
- Compare replica memory, P95 latency and model quality for both versions under realistic canary load.
- Read the serving endpoint's active model versions and actual traffic percentages after update.
- Review expected positive-event counts, label delays and confidence intervals for the canary duration.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Read the serving endpoint's active model versions and actual traffic percentages after update. The underlying technical issue is that traffic-routing configuration is an explicit version allocation and must match the authorized canary percentage.
Incorrect Answers
A:That diagnostic evidence is directed at a different deployment strategies situation: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. For the stated incident, the decisive observation is read the serving endpoint's active model versions and actual traffic percentages after update.
B:That diagnostic evidence is directed at a different deployment strategies situation: A new recommendation model consumes three times as much memory and the team plans a canary based solely on prediction accuracy, ignoring serving capacity. For the stated incident, the decisive observation is read the serving endpoint's active model versions and actual traffic percentages after update.
D:That diagnostic evidence is directed at a different deployment strategies situation: A candidate fraud model has near-identical average error to the champion, but an organization wants to compare rare fraudulent transactions during a brief canary. For the stated incident, the decisive observation is read the serving endpoint's active model versions and actual traffic percentages after update.
Question 235
Review this production model incident: A retailer needs one independent forecasting model per store but concatenates all store histories before fitting one global model. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Compare the business requirement and per-store holdout behavior with the single global estimator's outputs.
- Compare operation requirements, data conversion cost and supported query optimizations in both engines.
- Compare GPU inventory with the current trial task graph, resource reservations and wait times.
- Benchmark single-node and distributed alternatives using matched data, quality metrics and total compute cost.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Compare the business requirement and per-store holdout behavior with the single global estimator's outputs. The underlying technical issue is that group-specific training must preserve group identity when independent model parameters are required.
Incorrect Answers
B:That diagnostic evidence is directed at a different scaling and tuning situation: A team already has a vectorized Spark DataFrame pipeline and proposes converting everything to Ray datasets solely to perform a simple SQL aggregation. For the stated incident, the decisive observation is compare the business requirement and per-store holdout behavior with the single global estimator's outputs.
C:That diagnostic evidence is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. For the stated incident, the decisive observation is compare the business requirement and per-store holdout behavior with the single global estimator's outputs.
D:That diagnostic evidence is directed at a different scaling and tuning situation: A moderately sized dataset fits on one worker but a team provisions a huge distributed cluster and observes overhead higher than model fit. For the stated incident, the decisive observation is compare the business requirement and per-store holdout behavior with the single global estimator's outputs.
Question 236
Review this production model incident: A data scientist selects the lowest validation loss from nested runs but reports only the parent run ID as the model-producing run. What is the central engineering constraint behind this behavior?
- A parent groups trials; the specific child run that produced the selected artifact must remain identifiable.
- Request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
- A custom MLflow PyFunc's input contract and online feature lookups must be designed for request-time availability and governance.
- Experiment tracking must minimize sensitive data and enforce artifact permissions rather than treating all model logs as harmless.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. A parent groups trials; the specific child run that produced the selected artifact must remain identifiable. To test it, trace winning model artifact URI and validation metric to its actual nested child run.
Incorrect Answers
B:That underlying principle is directed at a different advanced mlflow usage situation: A fraud PyFunc uses a global mutable feature cache initialized by the first request and mixes data between subsequent users. The current incident instead turns on a parent groups trials; the specific child run that produced the selected artifact must remain identifiable.
C:That underlying principle is directed at a different advanced mlflow usage situation: A real-time churn endpoint accepts customer IDs and must fetch current allowed account features, but a custom model class instead expects the caller to supply the entire internal training row. The current incident instead turns on a parent groups trials; the specific child run that produced the selected artifact must remain identifiable.
D:That underlying principle is directed at a different advanced mlflow usage situation: A regulated analytics team uploads full customer identifiers inside MLflow model artifacts during troubleshooting. The current incident instead turns on a parent groups trials; the specific child run that produced the selected artifact must remain identifiable.
Question 237
In an enterprise ML workflow, the reported issue is: An inference profile groups two active model versions into one aggregate and hides a regression introduced by the new release. What likely operational or model-quality harm does this issue create?
- On-call teams are paged repeatedly for harmless short-lived noise.
- Configuration fails on nonexistent inference fields and delays basic table monitoring.
- Concept drift or changed label relationships degrade decisions without an input-drift alarm.
- A deteriorating new model is concealed by still-good traffic on the old model.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. A deteriorating new model is concealed by still-good traffic on the old model. The causal constraint is: Prediction monitoring requires a reliable model/version identifier to compare deployed candidates fairly.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A monitor alerts whenever a ratio exceeds 0.3 for a single minute, although normal traffic produces brief spikes and operations require ten-minute persistence. The more immediate risk here is a deteriorating new model is concealed by still-good traffic on the old model.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An analyst wants to inspect today’s static customer table for missing values and profile changes but chooses an inference monitor requiring model IDs and predictions. The more immediate risk here is a deteriorating new model is concealed by still-good traffic on the old model.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A risk service measures feature drift but neglects observed-label performance and continues serving an accurate-looking yet ineffective model. The more immediate risk here is a deteriorating new model is concealed by still-good traffic on the old model.
Question 238
The engineering team encounters the following issue: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. What corrective action should the engineer prefer?
- Record a resume lineage link and preserve distinct nested trials for both original and continued runs.
- Document and align valid online retrieval semantics with the model's training and evaluation feature contract.
- Remove unsafe global request-specific state and use explicitly keyed, governed feature resolution.
- Log the calibration chart and underlying evaluation summary as artifacts linked to the model run.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Record a resume lineage link and preserve distinct nested trials for both original and continued runs. The issue matters because duplicate or omitted trials distort comparison when two partial histories are treated as independent full searches.
Incorrect Answers
B:That corrective action is directed at a different advanced mlflow usage situation: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. The relevant corrective action here is record a resume lineage link and preserve distinct nested trials for both original and continued runs.
C:That corrective action is directed at a different advanced mlflow usage situation: A fraud PyFunc uses a global mutable feature cache initialized by the first request and mixes data between subsequent users. The relevant corrective action here is record a resume lineage link and preserve distinct nested trials for both original and continued runs.
D:That corrective action is directed at a different advanced mlflow usage situation: An ML engineer logs a calibration plot as a string metric, then downstream reviewers cannot inspect the visualization or its input version. The relevant corrective action here is record a resume lineage link and preserve distinct nested trials for both original and continued runs.
Question 239
Review this production model incident: A language model's weights cannot fit on one accelerator yet a team launches pure data-parallel replicas, each loading the entire model. What end-to-end verification is most persuasive here?
- The same training job finishes without swapping and within cost and latency limits.
- Ray reports parallel trial schedules that respect assigned GPUs and complete the planned search.
- The supported sharded model initializes and completes a forward/backward training step without OOM.
- An interrupted multiworker study resumes without duplicate trial claims or inconsistent objective history.
Explanation
Correct Answer
C:This verification measures the required result rather than just process liveness. The supported sharded model initializes and completes a forward/backward training step without OOM. It confirms whether the targeted action succeeded: adopt validated model-parallel or sharded training to fit the model and its optimizer state.
Incorrect Answers
A:That acceptance test is directed at a different scaling and tuning situation: A single-node model training run spends almost all time swapping a 40 GB feature matrix on a 16 GB VM; no distributed version exists. Here the actual success criterion is the supported sharded model initializes and completes a forward/backward training step without OOM.
B:That acceptance test is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. Here the actual success criterion is the supported sharded model initializes and completes a forward/backward training step without OOM.
D:That acceptance test is directed at a different scaling and tuning situation: A researcher parallelizes Optuna trials while sharing one writable SQLite file on a fragile network mount and observes duplicate reservations and storage locks. Here the actual success criterion is the supported sharded model initializes and completes a forward/backward training step without OOM.
Question 240
Review this production model incident: An ML platform moves its configuration from older Databricks Asset Bundles terminology to newer Declarative Automation Bundles references and assumes every resource schema stayed identical. What end-to-end verification is most persuasive here?
- Load tests show production SLAs hold while research workloads respect assigned capacity limits.
- The pinned CLI validates and deploys the requested model experiment and endpoint resources in staging.
- Region-specific test requests reach only authorized feature storage and compliant endpoints.
- The same scheduled job runs under its controlled service identity after the personal account is removed.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. The pinned CLI validates and deploys the requested model experiment and endpoint resources in staging. It confirms whether the targeted action succeeded: migrate or retain the configuration only after verifying current supported fields and running an explicit bundle validation.
Incorrect Answers
A:That acceptance test is directed at a different environment architectures situation: A production deployment scales to 30 teams but all jobs share one compute pool with no capacity limits or cost attribution. Here the actual success criterion is the pinned CLI validates and deploys the requested model experiment and endpoint resources in staging.
C:That acceptance test is directed at a different environment architectures situation: A multilingual prediction service must handle regional data residency, but the team copies a shared feature cache across regions without governance. Here the actual success criterion is the pinned CLI validates and deploys the requested model experiment and endpoint resources in staging.
D:That acceptance test is directed at a different environment architectures situation: A distributed retraining workflow uses ephemeral personal compute and fails when the original engineer's account is disabled. Here the actual success criterion is the pinned CLI validates and deploys the requested model experiment and endpoint resources in staging.