Databricks Certified Machine Learning Professional Practice Test 8
Question 70
A production ML review contains this situation: A rollout updates the serving model but a renamed input column breaks requests from an older mobile client. Which diagnostic check provides the strongest direct evidence?
- Check independent health and version identity for both active and standby deployment configurations before cutover.
- Replay stored valid requests from old and new clients against the candidate endpoint before shifting traffic.
- Review expected positive-event counts, label delays and confidence intervals for the canary duration.
- Compare serving-error, latency and outcome metrics separately for champion and canary model versions at the proposed traffic share.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Replay stored valid requests from old and new clients against the candidate endpoint before shifting traffic. The underlying technical issue is that model Serving deployments must validate backward compatibility of request schema across existing clients.
Incorrect Answers
A:That diagnostic evidence is directed at a different deployment strategies situation: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. For the stated incident, the decisive observation is replay stored valid requests from old and new clients against the candidate endpoint before shifting traffic.
C:That diagnostic evidence is directed at a different deployment strategies situation: A candidate fraud model has near-identical average error to the champion, but an organization wants to compare rare fraudulent transactions during a brief canary. For the stated incident, the decisive observation is replay stored valid requests from old and new clients against the candidate endpoint before shifting traffic.
D:That diagnostic evidence is directed at a different deployment strategies situation: A travel site serves 30,000 bookings per minute and wants to evaluate a new fraud model while limiting customer impact during its first hour. For the stated incident, the decisive observation is replay stored valid requests from old and new clients against the candidate endpoint before shifting traffic.
Question 71
Consider the following fictional Databricks production case: A model is promoted to production with its code unchanged but a different unpinned training library version is installed, altering prediction preprocessing. What underlying principle determines the most appropriate interpretation?
- Tracking, registry, jobs and serving are distinct lifecycle capabilities that must be linked by artifact identity.
- Lifecycle reproducibility spans source data, feature definitions, training configuration, model artifact and serving identity.
- Environment transitions must preserve or explicitly test library and runtime dependencies as well as source code.
- Experiment tracking metadata is not a substitute for governed asset privileges and serving access control.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Environment transitions must preserve or explicitly test library and runtime dependencies as well as source code. To test it, compare the logged training environment, lockfile and production model-load dependency versions.
Incorrect Answers
A:That underlying principle is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. The current incident instead turns on environment transitions must preserve or explicitly test library and runtime dependencies as well as source code.
B:That underlying principle is directed at a different model lifecycle management situation: An organization has reliable model serving but never archives feature definitions or experiment parameters and cannot reproduce predictions after drift. The current incident instead turns on environment transitions must preserve or explicitly test library and runtime dependencies as well as source code.
D:That underlying principle is directed at a different model lifecycle management situation: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. The current incident instead turns on environment transitions must preserve or explicitly test library and runtime dependencies as well as source code.
Question 72
Review this production model incident: A team logs a score labeled accuracy with values between 0 and 100 in some runs and between 0 and 1 in others, then ranks runs automatically. What is the central engineering constraint behind this behavior?
- The model's request-time feature contract should be compared with training-time feature semantics to avoid unintended skew.
- A resumed search needs stable experiment and parent-child provenance across partial reruns to support comparison.
- Logged metric names require consistent definitions and units before their numerical values can be compared.
- Request-scoped feature retrieval and thread-safe state are needed when the custom model serves multiple callers.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Logged metric names require consistent definitions and units before their numerical values can be compared. To test it, inspect metric computation code and raw values across runs for incompatible scale conventions.
Incorrect Answers
A:That underlying principle is directed at a different advanced mlflow usage situation: A deployed custom model fetches latest feature values during serving while offline evaluation used event-time snapshots without checking the difference. The current incident instead turns on logged metric names require consistent definitions and units before their numerical values can be compared.
B:That underlying principle is directed at a different advanced mlflow usage situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. The current incident instead turns on logged metric names require consistent definitions and units before their numerical values can be compared.
D:That underlying principle is directed at a different advanced mlflow usage situation: A fraud PyFunc uses a global mutable feature cache initialized by the first request and mixes data between subsequent users. The current incident instead turns on logged metric names require consistent definitions and units before their numerical values can be compared.
Question 73
Review this production model incident: A platform engineer creates a custom serving endpoint manually and assumes the exact settings can be recovered after an accidental deletion. Which mitigation resolves the defect rather than concealing it?
- Record compatible dependencies with the PyFunc artifact and validate them in a clean runtime.
- Use a properly scoped authorized credential and supported authenticated inference request.
- Update the caller to handle the supported structured response and validate required keys and types.
- Manage endpoint definitions through a supported declarative or API workflow with versioned parameters.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Manage endpoint definitions through a supported declarative or API workflow with versioned parameters. The issue matters because recovery requires guessing traffic splits, permissions and served model versions during an outage.
Incorrect Answers
A:That corrective action is directed at a different custom model serving situation: A team logs custom MLflow model weights but forgets to include required Python package versions in the model environment. The relevant corrective action here is manage endpoint definitions through a supported declarative or API workflow with versioned parameters.
B:That corrective action is directed at a different custom model serving situation: A client invokes a registered custom ML model using an unauthenticated HTTP POST to the serving endpoint and assumes network access implies authorization. The relevant corrective action here is manage endpoint definitions through a supported declarative or API workflow with versioned parameters.
C:That corrective action is directed at a different custom model serving situation: A serving client calls the REST endpoint expecting one number but the custom PyFunc returns a structured list of prediction and confidence fields. The relevant corrective action here is manage endpoint definitions through a supported declarative or API workflow with versioned parameters.
Question 74
Consider the following fictional Databricks production case: A team unit-tests each model component independently but its feature table and serving endpoint disagree on column names. What consequence most specifically follows from leaving the issue unchanged?
- All units pass while the production workflow fails with a schema mismatch.
- Invalid standardized values contaminate model input vectors or crash scoring.
- Operators believe a new model is live while the previous model still receives all traffic.
- The system passes isolated tests and fails immediately after deployment.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. All units pass while the production workflow fails with a schema mismatch. The causal constraint is: Integration tests verify handoffs between components that component-level unit tests cannot validate.
Incorrect Answers
B:That direct consequence is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. The more immediate risk here is all units pass while the production workflow fails with a schema mismatch.
C:That direct consequence is directed at a different validation testing situation: A model version is registered but the deployment job points to an old alias after promotion and continues serving last week's model. The more immediate risk here is all units pass while the production workflow fails with a schema mismatch.
D:That direct consequence is directed at a different validation testing situation: A test suite mocks every external interface so aggressively that even an incompatible feature table schema appears valid. The more immediate risk here is all units pass while the production workflow fails with a schema mismatch.
Question 75
Review this production model incident: A team unit-tests each model component independently but its feature table and serving endpoint disagree on column names. Which mitigation resolves the defect rather than concealing it?
- Add an environment-level integration test for table, model and serving input/output contracts.
- Write explicit notebook unit tests for missingness and correct zero handling, then fix the transformation if needed.
- Include paired known-label fixtures and assert prediction class semantics and output ordering.
- Extract testable pure functions into a reusable module and keep notebook orchestration thin.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Add an environment-level integration test for table, model and serving input/output contracts. The issue matters because all units pass while the production workflow fails with a schema mismatch.
Incorrect Answers
B:That corrective action is directed at a different validation testing situation: A feature function maps a null purchase amount to zero without distinguishing missing observations, and a notebook unit test covers only positive amounts. The relevant corrective action here is add an environment-level integration test for table, model and serving input/output contracts.
C:That corrective action is directed at a different validation testing situation: An end-to-end validation uses only a synthetic positive label and fails to detect a swapped positive/negative class mapping in inference. The relevant corrective action here is add an environment-level integration test for table, model and serving input/output contracts.
D:That corrective action is directed at a different validation testing situation: A team hides all preprocessing logic inside one long notebook, making it hard to test independently without starting a Spark cluster. The relevant corrective action here is add an environment-level integration test for table, model and serving input/output contracts.
Question 76
In an enterprise ML workflow, the reported issue is: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. What technical distinction is essential to this investigation?
- Rollout recovery must address the actual endpoint traffic configuration, not assume an alias automatically changes every active route.
- Blue-green deployment keeps two release environments available so traffic can be switched between validated states.
- An actionable rollback plan needs a retained deployable champion artifact and a tested restoration procedure.
- Model Serving configuration acceptance is not equivalent to endpoint/model readiness for user traffic.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Blue-green deployment keeps two release environments available so traffic can be switched between validated states. To test it, check independent health and version identity for both active and standby deployment configurations before cutover.
Incorrect Answers
A:That underlying principle is directed at a different deployment strategies situation: During a controlled release, the candidate model's P95 latency triples and engineers want to change the registered model alias without adjusting endpoint routing. The current incident instead turns on blue-green deployment keeps two release environments available so traffic can be switched between validated states.
C:That underlying principle is directed at a different deployment strategies situation: An ML operator promotes a new model version but its automated rollback points to a model that was deleted from the endpoint configuration. The current incident instead turns on blue-green deployment keeps two release environments available so traffic can be switched between validated states.
D:That underlying principle is directed at a different deployment strategies situation: An endpoint update request succeeds but the newly configured model version is still loading; monitoring marks the release complete immediately. The current incident instead turns on blue-green deployment keeps two release environments available so traffic can be switched between validated states.
Question 77
Consider the following fictional Databricks production case: A team already has a vectorized Spark DataFrame pipeline and proposes converting everything to Ray datasets solely to perform a simple SQL aggregation. Which observation most directly tests the suspected failure boundary?
- Compare per-replica parameter memory with device capacity and supported sharding topology.
- Compare GPU inventory with the current trial task graph, resource reservations and wait times.
- Compare operation requirements, data conversion cost and supported query optimizations in both engines.
- Inspect per-account cardinality and worker memory/OOM traces for the failing group.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Compare operation requirements, data conversion cost and supported query optimizations in both engines. The underlying technical issue is that spark SQL is often the more direct choice for distributed tabular transformations, while Ray targets flexible task/actor workloads.
Incorrect Answers
A:That diagnostic evidence is directed at a different scaling and tuning situation: A language model's weights cannot fit on one accelerator yet a team launches pure data-parallel replicas, each loading the entire model. For the stated incident, the decisive observation is compare operation requirements, data conversion cost and supported query optimizations in both engines.
B:That diagnostic evidence is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. For the stated incident, the decisive observation is compare operation requirements, data conversion cost and supported query optimizations in both engines.
D:That diagnostic evidence is directed at a different scaling and tuning situation: One enterprise account holds 80% of historical events, and grouped pandas training for that account exceeds executor memory while other accounts finish. For the stated incident, the decisive observation is compare operation requirements, data conversion cost and supported query optimizations in both engines.
Question 78
Consider the following fictional Databricks production case: A team wants to convert several numeric inputs into one features vector but treats VectorAssembler as a clustering Estimator that should output predictions. Which result is the strongest acceptance criterion after remediation?
- The assembled vector contains the ordered numeric fields and the estimator can consume it.
- A repeatable benchmark shows the chosen execution mode meets latency and validation requirements at lower total cost.
- A newly emitted synthetic event is scored and appears downstream without restarting the job.
- The full fitted pipeline completes on a small test DataFrame and exposes the planned input and output columns.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. The assembled vector contains the ordered numeric fields and the estimator can consume it. It confirms whether the targeted action succeeded: apply VectorAssembler to produce the features vector before fitting the chosen learning algorithm.
Incorrect Answers
B:That acceptance test is directed at a different using spark ml situation: A laboratory runs a linear regression every quarter on only 280 clean observations and proposes a 48-worker Spark cluster to reduce a two-second local fit. Here the actual success criterion is the assembled vector contains the ordered numeric fields and the estimator can consume it.
C:That acceptance test is directed at a different using spark ml situation: A streaming anomaly team uses a static read of a message directory once, believing the trained SparkML model will continuously produce scores for future arriving events. Here the actual success criterion is the assembled vector contains the ordered numeric fields and the estimator can consume it.
D:That acceptance test is directed at a different using spark ml situation: A pipeline engineer fits a vectorizer after an estimator in the stage list and then tries to use the estimator's prediction as the vectorizer input. Here the actual success criterion is the assembled vector contains the ordered numeric fields and the estimator can consume it.
Question 79
In an enterprise ML workflow, the reported issue is: An account-risk model's fraud recall is stable overall but unexpectedly falls for new customers in their first 30 days. What technical distinction is essential to this investigation?
- Life-cycle cohort slices can expose onboarding weaknesses that broad averages obscure.
- Global thresholds can hide local critical slices; alerting should reflect operational segmentation where justified.
- Custom monitoring outputs should preserve necessary aggregate signals without exposing unnecessary entity-level sensitive data.
- Endpoint throughput can be constrained by queueing, request concurrency, external features or rate limits beyond CPU utilization.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Life-cycle cohort slices can expose onboarding weaknesses that broad averages obscure. To test it, compute recall for new and established accounts with matched mature labels and stable cohort definitions.
Incorrect Answers
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. The current incident instead turns on life-cycle cohort slices can expose onboarding weaknesses that broad averages obscure.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A monitor owner writes a custom metric that logs unaggregated account-level health data in a broadly shared results table. The current incident instead turns on life-cycle cohort slices can expose onboarding weaknesses that broad averages obscure.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A production endpoint scales CPU instances up when traffic rises, but request queues and P95 response time keep growing despite low CPU usage. The current incident instead turns on life-cycle cohort slices can expose onboarding weaknesses that broad averages obscure.
Question 80
Review this production model incident: A feature refresh writes duplicate records for each entity and logical timestamp after retries, breaking unique lookups. Which failure risk follows most closely from the unresolved behavior?
- Invalid numeric features produce serving errors or hidden downstream score distortions.
- An ancient measurement can influence a current score as if it were fresh evidence.
- A repeated refresh doubles feature rows and multiplies downstream training joins.
- Scores after failover revert to incomplete or uninitialized feature state.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. A repeated refresh doubles feature rows and multiplies downstream training joins. The causal constraint is: Idempotent feature pipeline writes need stable entity/time keys and safe retry semantics.
Incorrect Answers
A:That direct consequence is directed at a different advanced feature store concepts situation: A payment classifier's on-demand feature division returns infinity for zero denominators and fails serialization in the serving endpoint. The more immediate risk here is a repeated refresh doubles feature rows and multiplies downstream training joins.
B:That direct consequence is directed at a different advanced feature store concepts situation: A sensor classifier silently reuses features from a year-old device after months without readings, despite a maximum allowed lookback of seven days. The more immediate risk here is a repeated refresh doubles feature rows and multiplies downstream training joins.
D:That direct consequence is directed at a different advanced feature store concepts situation: A business deploys a model whose features are computed from a stream in memory only and disappear when its compute cluster restarts. The more immediate risk here is a repeated refresh doubles feature rows and multiplies downstream training joins.