Databricks Certified Machine Learning Professional Practice Test 23
Question 220
Review this production model incident: A business deploys a model whose features are computed from a stream in memory only and disappear when its compute cluster restarts. What is the central engineering constraint behind this behavior?
- Historical feature timelines must be retained when reproducible point-in-time joins depend on prior values.
- Point-in-time correctness may require both event time and actual feature availability when ingestion latency matters.
- Production streaming features need durable governed storage or reproducible replay, not ephemeral worker variables.
- On-demand feature functions must explicitly handle invalid input and output type constraints.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Production streaming features need durable governed storage or reproducible replay, not ephemeral worker variables. To test it, inspect state store durability, output table persistence and recovery behavior under restart.
Incorrect Answers
A:That underlying principle is directed at a different advanced feature store concepts situation: A model's feature generation step overwrites all feature-table history daily even though time-aware training requires earlier snapshots. The current incident instead turns on production streaming features need durable governed storage or reproducible replay, not ephemeral worker variables.
B:That underlying principle is directed at a different advanced feature store concepts situation: A retailer's features arrive two hours late, and offline training uses event time alone even though the online system could not have seen those late arrivals at prediction time. The current incident instead turns on production streaming features need durable governed storage or reproducible replay, not ephemeral worker variables.
D:That underlying principle is directed at a different advanced feature store concepts situation: A payment classifier's on-demand feature division returns infinity for zero denominators and fails serialization in the serving endpoint. The current incident instead turns on production streaming features need durable governed storage or reproducible replay, not ephemeral worker variables.
Question 221
The engineering team encounters the following issue: A retailer needs one independent forecasting model per store but concatenates all store histories before fitting one global model. Which experiment most directly demonstrates a successful fix?
- Ray reports parallel trial schedules that respect assigned GPUs and complete the planned search.
- Model artifacts and predictions are traceable to each requested store key.
- Every completed trial has a linked run, parameter map, metric and deterministic winner selection.
- All workers converge to the same checkpoint step with documented update consistency.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. Model artifacts and predictions are traceable to each requested store key. It confirms whether the targeted action succeeded: use a supported grouped pandas function to fit a separate model for each store with stable group keys.
Incorrect Answers
A:That acceptance test is directed at a different scaling and tuning situation: A team runs independent GPU-heavy neural trials sequentially, despite having several isolated GPU workers that can fit each model. Here the actual success criterion is model artifacts and predictions are traceable to each requested store key.
C:That acceptance test is directed at a different scaling and tuning situation: An Optuna search runs 80 hyperparameter trials but records only the last winning score in MLflow, making comparisons impossible. Here the actual success criterion is model artifacts and predictions are traceable to each requested store key.
D:That acceptance test is directed at a different scaling and tuning situation: A training team averages gradients from multiple workers but lets each worker use a different untracked version of the model. Here the actual success criterion is model artifacts and predictions are traceable to each requested store key.
Question 222
In an enterprise ML workflow, the reported issue is: A client invokes a registered custom ML model using an unauthenticated HTTP POST to the serving endpoint and assumes network access implies authorization. Before modifying the pipeline, which evidence should be collected first?
- Review endpoint change history, CI deployment commits and the order of conflicting updates.
- Compare current endpoint settings with source-controlled deployment configuration and documented backup state.
- Inspect model artifact source, planned three-level Unity Catalog name and effective registration permissions.
- Inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes. The underlying technical issue is that databricks Model Serving REST calls require the appropriate approved authentication and endpoint permissions.
Incorrect Answers
A:That diagnostic evidence is directed at a different custom model serving situation: A team edits a model endpoint through the UI while its CI pipeline concurrently applies an older configuration and reverts the intended traffic split. For the stated incident, the decisive observation is inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes.
B:That diagnostic evidence is directed at a different custom model serving situation: A platform engineer creates a custom serving endpoint manually and assumes the exact settings can be recovered after an accidental deletion. For the stated incident, the decisive observation is inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes.
C:That diagnostic evidence is directed at a different custom model serving situation: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. For the stated incident, the decisive observation is inspect sanitized 401/403 response details, the caller identity and documented endpoint access scopes.
Question 223
A production ML review contains this situation: An endpoint receives a large request burst every morning and its monitoring alert reacts only to 5xx errors after autoscaling cannot keep up. Which statement best explains the failure mode?
- Aggregate performance can conceal failure in important subpopulations and lifecycle stages.
- Request-rate and capacity forecasts can provide earlier warning than error counts alone.
- Production input should sometimes be compared to a retained training reference, not only recent windows.
- Custom derived metrics need deterministic handling for undefined denominators and small samples.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Request-rate and capacity forecasts can provide earlier warning than error counts alone. To test it, compare incoming request rate, provisioned concurrency and scaling lag before observed errors.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. The current incident instead turns on request-rate and capacity forecasts can provide earlier warning than error counts alone.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A model trained on last year's customer mix is monitored only against yesterday's already shifted cohort, so its original training-population mismatch is hidden. The current incident instead turns on request-rate and capacity forecasts can provide earlier warning than error counts alone.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A custom drift metric is expressed as today-minus-yesterday percentage, but yesterday's eligible count can be zero. The current incident instead turns on request-rate and capacity forecasts can provide earlier warning than error counts alone.
Question 224
Review this production model incident: A regulated company changes deployed Python notebook code directly in production to correct a model transform while its test environment runs a different untracked revision. What is the central engineering constraint behind this behavior?
- A deploy-code strategy promotes a tested, versioned code artifact through environments rather than editing production independently.
- Tracking, registry, jobs and serving are distinct lifecycle capabilities that must be linked by artifact identity.
- Governed model registration and access controls provide traceability beyond an ad hoc personal file path.
- Experiment tracking metadata is not a substitute for governed asset privileges and serving access control.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. A deploy-code strategy promotes a tested, versioned code artifact through environments rather than editing production independently. To test it, compare release commit identifiers, approved artifacts and runtime dependency versions across dev, test and prod.
Incorrect Answers
B:That underlying principle is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. The current incident instead turns on a deploy-code strategy promotes a tested, versioned code artifact through environments rather than editing production independently.
C:That underlying principle is directed at a different model lifecycle management situation: An engineer stores model weights in a personal workspace folder and treats that path as a production artifact registry with audit history. The current incident instead turns on a deploy-code strategy promotes a tested, versioned code artifact through environments rather than editing production independently.
D:That underlying principle is directed at a different model lifecycle management situation: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. The current incident instead turns on a deploy-code strategy promotes a tested, versioned code artifact through environments rather than editing production independently.
Question 225
Review this production model incident: A test engineer can create a model-serving endpoint in a sandbox but the production deployment fails because the automation identity lacks registered-model access. What is the central engineering constraint behind this behavior?
- Serving deployment permissions must include authorization to use the intended governed model and endpoint resources.
- Serving responses that are cached or audited need model-version provenance and appropriate expiration rules.
- Governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
- Custom Python-model artifacts need portable packaged dependencies rather than absolute developer-local file paths.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Serving deployment permissions must include authorization to use the intended governed model and endpoint resources. To test it, compare sandbox/prod identities, model grants and sanitized deployment authorization errors.
Incorrect Answers
B:That underlying principle is directed at a different custom model serving situation: An integration caches predictions without recording the model version and returns yesterday's values after a controlled rollback. The current incident instead turns on serving deployment permissions must include authorization to use the intended governed model and endpoint resources.
C:That underlying principle is directed at a different custom model serving situation: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. The current incident instead turns on serving deployment permissions must include authorization to use the intended governed model and endpoint resources.
D:That underlying principle is directed at a different custom model serving situation: A custom PyFunc estimator loads a tokenizer file from an engineer's laptop path when called in a serving container. The current incident instead turns on serving deployment permissions must include authorization to use the intended governed model and endpoint resources.
Question 226
Review this production model incident: A notebook's label encoder produces a different category order depending on an unordered Python set, but tests assert only output length. What is the central engineering constraint behind this behavior?
- Test stages need appropriate data classification and isolation, not just increasingly realistic inputs.
- Test organization should separate production orchestration from the test runner and control execution stages.
- Unit tests should verify reproducible content and stable mappings rather than shape alone.
- Integration fixtures must check semantic prediction values and class labels, not only response existence.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Unit tests should verify reproducible content and stable mappings rather than shape alone. To test it, repeat the same synthetic category fixture in separate processes and compare exact mapping outputs.
Incorrect Answers
A:That underlying principle is directed at a different validation testing situation: An engineer uses only production replay data to validate a code change and exposes customer events in an unrestricted test workspace. The current incident instead turns on unit tests should verify reproducible content and stable mappings rather than shape alone.
B:That underlying principle is directed at a different validation testing situation: An ML project stores test files directly in each production notebook and execution of a deployed notebook inadvertently reruns expensive tests. The current incident instead turns on unit tests should verify reproducible content and stable mappings rather than shape alone.
D:That underlying principle is directed at a different validation testing situation: An end-to-end validation uses only a synthetic positive label and fails to detect a swapped positive/negative class mapping in inference. The current incident instead turns on unit tests should verify reproducible content and stable mappings rather than shape alone.
Question 227
Consider the following fictional Databricks production case: An online behavioral feature counter consumes a streaming source but does not retain checkpoint state after worker restarts. What consequence most specifically follows from leaving the issue unchanged?
- Restarted jobs double count behavior or lose events, corrupting online feature values.
- An ancient measurement can influence a current score as if it were fresh evidence.
- Several models change their effective input distributions without deliberate retraining or review.
- Credential compromise exposes unrelated customers' private feature data.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. Restarted jobs double count behavior or lose events, corrupting online feature values. The causal constraint is: Fault-tolerant real-time feature computation requires durable streaming progress and suitable sink idempotency.
Incorrect Answers
B:That direct consequence is directed at a different advanced feature store concepts situation: A sensor classifier silently reuses features from a year-old device after months without readings, despite a maximum allowed lookback of seven days. The more immediate risk here is restarted jobs double count behavior or lose events, corrupting online feature values.
C:That direct consequence is directed at a different advanced feature store concepts situation: A team updates a shared on-demand feature function without version control and unexpectedly changes both batch retraining and online serving behavior. The more immediate risk here is restarted jobs double count behavior or lose events, corrupting online feature values.
D:That direct consequence is directed at a different advanced feature store concepts situation: A model serving service uses a broadly privileged shared token to read every online feature table when it needs only one approved feature set. The more immediate risk here is restarted jobs double count behavior or lose events, corrupting online feature values.
Question 228
Review this production model incident: A logistics model must score 60 million daily routes already in Delta but the scoring script collects each route into Python before calling predict. Which failure risk follows most closely from the unresolved behavior?
- The supposedly unseen holdout can affect model inputs and inflate reported generalization.
- Important fraud cases remain undetected despite an impressive accuracy percentage.
- Driver memory exhaustion halts scoring and leaves most routes unprocessed.
- No training-derived category mapping exists for the requested transformation.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. Driver memory exhaustion halts scoring and leaves most routes unprocessed. The causal constraint is: Distributed batch inference can apply a saved SparkML PipelineModel directly to a DataFrame without collecting every row.
Incorrect Answers
A:That direct consequence is directed at a different using spark ml situation: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. The more immediate risk here is driver memory exhaustion halts scoring and leaves most routes unprocessed.
B:That direct consequence is directed at a different using spark ml situation: A fraud detector predicts the majority no-fraud class for 99.4% of cases and advertises 99.4% accuracy while missing most positives. The more immediate risk here is driver memory exhaustion halts scoring and leaves most routes unprocessed.
D:That direct consequence is directed at a different using spark ml situation: A new Spark developer calls transform on an unfitted StringIndexer instance and expects learned category labels to appear automatically. The more immediate risk here is driver memory exhaustion halts scoring and leaves most routes unprocessed.
Question 229
Review this production model incident: A team updates a shared on-demand feature function without version control and unexpectedly changes both batch retraining and online serving behavior. What evidence would best distinguish the actual fault from neighboring ML concerns?
- Compare feature event time, publish time and prediction decision time for a delayed fixture.
- Diff the offline and live feature transformation definitions for the same synthetic entity inputs.
- Audit token scopes and endpoint access logs against the feature tables accessed by the serving model.
- Map feature consumers, source commit IDs and before/after results for a fixed regression fixture.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Map feature consumers, source commit IDs and before/after results for a fixed regression fixture. The underlying technical issue is that shared feature logic changes should be versioned and regression-tested across all consuming models.
Incorrect Answers
A:That diagnostic evidence is directed at a different advanced feature store concepts situation: A retailer's features arrive two hours late, and offline training uses event time alone even though the online system could not have seen those late arrivals at prediction time. For the stated incident, the decisive observation is map feature consumers, source commit IDs and before/after results for a fixed regression fixture.
B:That diagnostic evidence is directed at a different advanced feature store concepts situation: An ML pipeline computes training features in one notebook and duplicates slightly different SQL in the online pipeline for the same entities. For the stated incident, the decisive observation is map feature consumers, source commit IDs and before/after results for a fixed regression fixture.
C:That diagnostic evidence is directed at a different advanced feature store concepts situation: A model serving service uses a broadly privileged shared token to read every online feature table when it needs only one approved feature set. For the stated incident, the decisive observation is map feature consumers, source commit IDs and before/after results for a fixed regression fixture.
Question 230
Consider the following fictional Databricks production case: A production endpoint scales CPU instances up when traffic rises, but request queues and P95 response time keep growing despite low CPU usage. Which observation most directly tests the suspected failure boundary?
- Inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.
- Stratify historical samples by day type and examine whether the observed difference is expected seasonality.
- Inspect inference table version IDs and label-matched errors before and after traffic shift.
- Inspect alert query configuration, notification destinations and missed alert delivery records.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage. The underlying technical issue is that endpoint throughput can be constrained by queueing, request concurrency, external features or rate limits beyond CPU utilization.
Incorrect Answers
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. For the stated incident, the decisive observation is inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. For the stated incident, the decisive observation is inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. For the stated incident, the decisive observation is inspect request queue depth, upstream feature lookup latency and rejected-request counts alongside CPU usage.