Databricks Certified Machine Learning Professional Practice Test 57
Question 560
A production ML review contains this situation: A regulated company changes deployed Python notebook code directly in production to correct a model transform while its test environment runs a different untracked revision. What production-safe change best matches the evidence?
- Bind training outputs to registered immutable versions and promote them through governed jobs or serving endpoints.
- Use a supported governed model registry and controlled permissions for production model versions.
- Build and validate one immutable release artifact before promoting it through controlled environments.
- Apply the correct governance controls separately to experiments, registered models and serving endpoints.
Explanation
Correct Answer
C:This corrects the relevant failure boundary instead of merely hiding the symptom. Build and validate one immutable release artifact before promoting it through controlled environments. The issue matters because the live model logic diverges from tested code and cannot be reproduced during an incident.
Incorrect Answers
A:That corrective action is directed at a different model lifecycle management situation: A team records experiments but has no controlled path from a selected registered model to scheduled scoring jobs. The relevant corrective action here is build and validate one immutable release artifact before promoting it through controlled environments.
B:That corrective action is directed at a different model lifecycle management situation: An engineer stores model weights in a personal workspace folder and treats that path as a production artifact registry with audit history. The relevant corrective action here is build and validate one immutable release artifact before promoting it through controlled environments.
D:That corrective action is directed at a different model lifecycle management situation: A platform team proposes using MLflow Tracking alone to enforce production endpoint authorization, ignoring Unity Catalog and service permissions. The relevant corrective action here is build and validate one immutable release artifact before promoting it through controlled environments.
Question 561
Review this production model incident: A real-time client retries every 429 rate-limit error immediately in an unbounded loop and multiplies load on the serving endpoint. What is the central engineering constraint behind this behavior?
- Governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
- Inference clients should honor rate limits and use bounded exponential backoff with idempotent retry semantics.
- Model artifacts should not serialize secrets; serving identities should resolve credentials securely at runtime.
- Repeatable endpoint deployment should preserve a controlled configuration definition and registered model identity.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Inference clients should honor rate limits and use bounded exponential backoff with idempotent retry semantics. To test it, inspect rate-limit response headers, client retry timeline and duplicate request identifiers.
Incorrect Answers
A:That underlying principle is directed at a different custom model serving situation: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. The current incident instead turns on inference clients should honor rate limits and use bounded exponential backoff with idempotent retry semantics.
C:That underlying principle is directed at a different custom model serving situation: An engineer logs a customer-specific API token inside the pickled PythonModel object before registering it to Unity Catalog. The current incident instead turns on inference clients should honor rate limits and use bounded exponential backoff with idempotent retry semantics.
D:That underlying principle is directed at a different custom model serving situation: A platform engineer creates a custom serving endpoint manually and assumes the exact settings can be recovered after an accidental deletion. The current incident instead turns on inference clients should honor rate limits and use bounded exponential backoff with idempotent retry semantics.
Question 562
Review this production model incident: A loan-risk feature usually has median debt-to-income 0.34, but a new cohort has median 0.61 while the average stays almost unchanged due to outliers. What is the central engineering constraint behind this behavior?
- Distributional drift cannot be ruled out by one stable mean; numeric quantiles and supported statistical comparisons can expose shape changes.
- Production input should sometimes be compared to a retained training reference, not only recent windows.
- Monitor creation and ongoing metric writes depend on governed resource access and configured output locations.
- Observability starts with traceable, appropriately governed prediction logging before statistical analysis.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Distributional drift cannot be ruled out by one stable mean; numeric quantiles and supported statistical comparisons can expose shape changes. To test it, compare recent and baseline quantiles plus a supported numeric distribution-difference metric at equivalent sample sizes.
Incorrect Answers
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A model trained on last year's customer mix is monitored only against yesterday's already shifted cohort, so its original training-population mismatch is hidden. The current incident instead turns on distributional drift cannot be ruled out by one stable mean; numeric quantiles and supported statistical comparisons can expose shape changes.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A monitor is pointed at a Unity Catalog table but the automation identity lacks the privileges needed to generate profile outputs. The current incident instead turns on distributional drift cannot be ruled out by one stable mean; numeric quantiles and supported statistical comparisons can expose shape changes.
D:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A serving endpoint appears healthy but nobody records model version, request identifier or prediction value, so a drift incident cannot be reconstructed. The current incident instead turns on distributional drift cannot be ruled out by one stable mean; numeric quantiles and supported statistical comparisons can expose shape changes.
Question 563
A production ML review contains this situation: A demographic fairness review uses a tiny sensitive subgroup with three labeled examples and reports its observed error as definitive. Which statement best explains the failure mode?
- Endpoint throughput can be constrained by queueing, request concurrency, external features or rate limits beyond CPU utilization.
- A custom fairness comparison needs matched labels, model predictions and governed subgroup definitions.
- Reference populations should match relevant seasonality and business context when interpreting drift.
- Subgroup metrics should enforce privacy and minimum-sample limits before strong comparative claims.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. Subgroup metrics should enforce privacy and minimum-sample limits before strong comparative claims. To test it, inspect cohort sizes, allowed sensitive fields and uncertainty around the reported subgroup error rate.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A production endpoint scales CPU instances up when traffic rises, but request queues and P95 response time keep growing despite low CPU usage. The current incident instead turns on subgroup metrics should enforce privacy and minimum-sample limits before strong comparative claims.
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A fairness reviewer needs the difference in false-negative rates between two approved cohorts, but the monitor computes an aggregate without cohort labels. The current incident instead turns on subgroup metrics should enforce privacy and minimum-sample limits before strong comparative claims.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. The current incident instead turns on subgroup metrics should enforce privacy and minimum-sample limits before strong comparative claims.
Question 564
Consider the following fictional Databricks production case: A model serving service uses a broadly privileged shared token to read every online feature table when it needs only one approved feature set. What underlying principle determines the most appropriate interpretation?
- Point-in-time training joins must select features available at or before each label observation, not the latest record irrespective of date.
- Online feature access should use least-privilege identity and scoped permissions rather than universal shared credentials.
- Production streaming features need durable governed storage or reproducible replay, not ephemeral worker variables.
- On-demand feature functions must explicitly handle invalid input and output type constraints.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. Online feature access should use least-privilege identity and scoped permissions rather than universal shared credentials. To test it, audit token scopes and endpoint access logs against the feature tables accessed by the serving model.
Incorrect Answers
A:That underlying principle is directed at a different advanced feature store concepts situation: A fraud training row is labeled on March 10 but an ordinary entity join attaches an account balance updated on March 12. The current incident instead turns on online feature access should use least-privilege identity and scoped permissions rather than universal shared credentials.
C:That underlying principle is directed at a different advanced feature store concepts situation: A business deploys a model whose features are computed from a stream in memory only and disappear when its compute cluster restarts. The current incident instead turns on online feature access should use least-privilege identity and scoped permissions rather than universal shared credentials.
D:That underlying principle is directed at a different advanced feature store concepts situation: A payment classifier's on-demand feature division returns infinity for zero denominators and fails serialization in the serving endpoint. The current incident instead turns on online feature access should use least-privilege identity and scoped permissions rather than universal shared credentials.
Question 565
Review this production model incident: A team publishes an online feature table but neglects its refresh schedule while the predictive model assumes values no older than 15 minutes. What is the central engineering constraint behind this behavior?
- On-demand feature functions must explicitly handle invalid input and output type constraints.
- An online store's existence does not guarantee its feature freshness; synchronization must be monitored.
- A time-series feature table needs a documented timestamp key and an as-of lookup to avoid multiplying labels or attaching future values.
- Streaming derived features should deduplicate at a stable business-event key before non-idempotent aggregation.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. An online store's existence does not guarantee its feature freshness; synchronization must be monitored. To test it, compare offline source update timestamps, online publication timestamps and stated staleness limits.
Incorrect Answers
A:That underlying principle is directed at a different advanced feature store concepts situation: A payment classifier's on-demand feature division returns infinity for zero denominators and fails serialization in the serving endpoint. The current incident instead turns on an online store's existence does not guarantee its feature freshness; synchronization must be monitored.
C:That underlying principle is directed at a different advanced feature store concepts situation: A training dataset combines a feature table with multiple timestamped rows per device using an equality join on device ID only. The current incident instead turns on an online store's existence does not guarantee its feature freshness; synchronization must be monitored.
D:That underlying principle is directed at a different advanced feature store concepts situation: A real-time feature job updates user purchase totals without checking whether upstream transaction events are delivered twice. The current incident instead turns on an online store's existence does not guarantee its feature freshness; synchronization must be monitored.
Question 566
A production ML review contains this situation: An ML project stores test files directly in each production notebook and execution of a deployed notebook inadvertently reruns expensive tests. Which check shows that this particular problem has been resolved?
- A constant-vector unit fixture returns a documented safe result with no NaN or infinity.
- Normal production execution does not run tests, while CI executes the full isolated suite.
- The exact saved fixture is successfully scored via the deployed endpoint with expected fields.
- The integration fixture traverses every component with matching validated field names.
Explanation
Correct Answer
B:This verification measures the required result rather than just process liveness. Normal production execution does not run tests, while CI executes the full isolated suite. It confirms whether the targeted action succeeded: keep reusable logic and tests in separate versioned units with explicit CI test stages.
Incorrect Answers
A:That acceptance test is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. Here the actual success criterion is normal production execution does not run tests, while CI executes the full isolated suite.
C:That acceptance test is directed at a different validation testing situation: A churn feature job succeeds and a classifier achieves acceptable F1, but the deployed endpoint returns HTTP 400 because request column names differ. Here the actual success criterion is normal production execution does not run tests, while CI executes the full isolated suite.
D:That acceptance test is directed at a different validation testing situation: A team unit-tests each model component independently but its feature table and serving endpoint disagree on column names. Here the actual success criterion is normal production execution does not run tests, while CI executes the full isolated suite.
Question 567
The engineering team encounters the following issue: A reinforcement-learning simulator requires stateful actors exchanging messages and irregular task graphs, but the team plans one Spark SQL groupBy per simulation step. What corrective action should the engineer prefer?
- Use an appropriate Ray actor-based framework while reserving Spark for eligible data preprocessing.
- Bind the selected model version to each group and join predictions on a unique provenance key.
- Batch or redesign microtrials and choose a proportionate degree of Ray parallelism.
- Scale worker capacity and partition layout according to measured distributed CPU demand.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Use an appropriate Ray actor-based framework while reserving Spark for eligible data preprocessing. The issue matters because a relational workaround adds expensive jobs and loses explicit actor-state semantics.
Incorrect Answers
B:That corrective action is directed at a different scaling and tuning situation: A batch scoring pipeline trains one per-region model and then joins model predictions back using region alone, duplicating rows because each region contains multiple versioned models. The relevant corrective action here is use an appropriate Ray actor-based framework while reserving Spark for eligible data preprocessing.
C:That corrective action is directed at a different scaling and tuning situation: A researcher distributes 500 tiny one-second trials across many Ray workers and sees coordination cost exceed compute time. The relevant corrective action here is use an appropriate Ray actor-based framework while reserving Spark for eligible data preprocessing.
D:That corrective action is directed at a different scaling and tuning situation: A feature-engineering job is CPU-bound on hundreds of independent daily partitions and a team upgrades only the driver to a much larger machine. The relevant corrective action here is use an appropriate Ray actor-based framework while reserving Spark for eligible data preprocessing.
Question 568
The engineering team encounters the following issue: A continuous-delivery job deploys a notebook change through a bundle but skips validation of referenced experiment paths and job permissions. What direct consequence should the owner prioritize if no fix is made?
- The deployed resources exist but cannot run due to missing experiment or model access.
- Critical retraining stops when a human account or notebook workspace changes.
- Sensitive records cross an unapproved residency boundary through convenience caching.
- One experiment starves critical retraining or inference workloads.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. The deployed resources exist but cannot run due to missing experiment or model access. The causal constraint is: Infrastructure/configuration definitions need predeployment validation and an integration check, not mere successful YAML parsing.
Incorrect Answers
B:That direct consequence is directed at a different environment architectures situation: A distributed retraining workflow uses ephemeral personal compute and fails when the original engineer's account is disabled. The more immediate risk here is the deployed resources exist but cannot run due to missing experiment or model access.
C:That direct consequence is directed at a different environment architectures situation: A multilingual prediction service must handle regional data residency, but the team copies a shared feature cache across regions without governance. The more immediate risk here is the deployed resources exist but cannot run due to missing experiment or model access.
D:That direct consequence is directed at a different environment architectures situation: A production deployment scales to 30 teams but all jobs share one compute pool with no capacity limits or cost attribution. The more immediate risk here is the deployed resources exist but cannot run due to missing experiment or model access.
Question 569
Review this production model incident: A client invokes a registered custom ML model using an unauthenticated HTTP POST to the serving endpoint and assumes network access implies authorization. Which mitigation resolves the defect rather than concealing it?
- Remove embedded secrets and configure an approved scoped secret retrieval mechanism for the serving process.
- Use a properly scoped authorized credential and supported authenticated inference request.
- Poll or otherwise verify documented readiness, then perform a permitted smoke prediction before marking the release ready.
- Route changes through a single approved configuration source and verify the active endpoint after each change.
Explanation
Correct Answer
B:This corrects the relevant failure boundary instead of merely hiding the symptom. Use a properly scoped authorized credential and supported authenticated inference request. The issue matters because unauthenticated calls are rejected or teams try to bypass endpoint controls.
Incorrect Answers
A:That corrective action is directed at a different custom model serving situation: An engineer logs a customer-specific API token inside the pickled PythonModel object before registering it to Unity Catalog. The relevant corrective action here is use a properly scoped authorized credential and supported authenticated inference request.
C:That corrective action is directed at a different custom model serving situation: A Databricks SDK deployment script sends a model-version update but does not wait for the endpoint to reach a usable ready state before announcing success. The relevant corrective action here is use a properly scoped authorized credential and supported authenticated inference request.
D:That corrective action is directed at a different custom model serving situation: A team edits a model endpoint through the UI while its CI pipeline concurrently applies an older configuration and reverts the intended traffic split. The relevant corrective action here is use a properly scoped authorized credential and supported authenticated inference request.
Question 570
In an enterprise ML workflow, the reported issue is: A research team captures only daily batch inference totals with no row-level timestamps but insists on per-minute live inference drift analysis. Before modifying the pipeline, which evidence should be collected first?
- Stratify historical samples by day type and examine whether the observed difference is expected seasonality.
- Inspect effective owner permissions, catalog/schema access and creation error logs for the target table.
- Inspect whether row-level event timestamps and prediction records actually exist for the desired minute windows.
- Compare regional event counts and failure proportions against the aggregate alert trigger.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Inspect whether row-level event timestamps and prediction records actually exist for the desired minute windows. The underlying technical issue is that monitoring granularity cannot exceed the information preserved in source rows and ingestion cadence.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An online feature monitor compares weekend shopping to weekday reference data and treats expected weekend behavior as a model failure. For the stated incident, the decisive observation is inspect whether row-level event timestamps and prediction records actually exist for the desired minute windows.
B:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A monitor is pointed at a Unity Catalog table but the automation identity lacks the privileges needed to generate profile outputs. For the stated incident, the decisive observation is inspect whether row-level event timestamps and prediction records actually exist for the desired minute windows.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. For the stated incident, the decisive observation is inspect whether row-level event timestamps and prediction records actually exist for the desired minute windows.