Databricks Certified Machine Learning Professional Practice Test 49
Question 480
The engineering team encounters the following issue: A credit reporting task needs one million records scored once per week for offline account review and requests a permanently running interactive endpoint. Which technical principle most directly explains the root of the issue?
- RMSE squares residuals and highlights large errors, while median error alone may obscure expensive outliers.
- A scheduled distributed batch inference job can better fit offline throughput and cost than always-on serving.
- Distributed batch inference can apply a saved SparkML PipelineModel directly to a DataFrame without collecting every row.
- Any fitted preprocessing stage can learn from holdout records and must be fitted within the training boundary.
Explanation
Correct Answer
B:The technical constraint is specific to the reported failure. A scheduled distributed batch inference job can better fit offline throughput and cost than always-on serving. To test it, measure the true deadline, weekly volume, endpoint idle time and batch-compute capacity.
Incorrect Answers
A:That underlying principle is directed at a different using spark ml situation: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. The current incident instead turns on a scheduled distributed batch inference job can better fit offline throughput and cost than always-on serving.
C:That underlying principle is directed at a different using spark ml situation: A logistics model must score 60 million daily routes already in Delta but the scoring script collects each route into Python before calling predict. The current incident instead turns on a scheduled distributed batch inference job can better fit offline throughput and cost than always-on serving.
D:That underlying principle is directed at a different using spark ml situation: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. The current incident instead turns on a scheduled distributed batch inference job can better fit offline throughput and cost than always-on serving.
Question 481
Review this production model incident: A recommender experiment computes all evaluation metrics on training interactions and reports the result as generalization to new customers. What end-to-end verification is most persuasive here?
- Model accuracy and coverage are reported on untouched realistic evaluation cohorts.
- A version-pinned test fixture yields the same feature vector and prediction as the approved model release.
- Weekly outputs meet completion time and completeness checks without unnecessary continuous endpoint usage.
- Two sample sentences produce the expected token-array column from a direct transform.
Explanation
Correct Answer
A:This verification measures the required result rather than just process liveness. Model accuracy and coverage are reported on untouched realistic evaluation cohorts. It confirms whether the targeted action succeeded: create an appropriate holdout design, including relevant new-user or time-split cohorts, and evaluate there.
Incorrect Answers
B:That acceptance test is directed at a different using spark ml situation: A batch scoring service loads a classifier version from last month while using today's new feature indexer, silently changing categorical positions. Here the actual success criterion is model accuracy and coverage are reported on untouched realistic evaluation cohorts.
C:That acceptance test is directed at a different using spark ml situation: A credit reporting task needs one million records scored once per week for offline account review and requests a permanently running interactive endpoint. Here the actual success criterion is model accuracy and coverage are reported on untouched realistic evaluation cohorts.
D:That acceptance test is directed at a different using spark ml situation: A text pipeline repeatedly fits a Tokenizer stage even though tokenization simply converts strings into token arrays without learning a model. Here the actual success criterion is model accuracy and coverage are reported on untouched realistic evaluation cohorts.
Question 482
In an enterprise ML workflow, the reported issue is: A team edits a model endpoint through the UI while its CI pipeline concurrently applies an older configuration and reverts the intended traffic split. What likely operational or model-quality harm does this issue create?
- Manual and automated updates race, creating unexpected model routing during production traffic.
- A malformed request crashes the entire serving worker or produces an untraceable score.
- The served model raises ImportError or changes semantics after deployment.
- The client amplifies throttling and creates a self-sustaining inference outage.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. Manual and automated updates race, creating unexpected model routing during production traffic. The causal constraint is: Endpoint deployment requires controlled ownership and conflict-safe update/versioning practices.
Incorrect Answers
B:That direct consequence is directed at a different custom model serving situation: A custom PyFunc's predict method assumes every request contains a numeric feature that is optional under the documented inference schema. The more immediate risk here is manual and automated updates race, creating unexpected model routing during production traffic.
C:That direct consequence is directed at a different custom model serving situation: A team logs custom MLflow model weights but forgets to include required Python package versions in the model environment. The more immediate risk here is manual and automated updates race, creating unexpected model routing during production traffic.
D:That direct consequence is directed at a different custom model serving situation: A real-time client retries every 429 rate-limit error immediately in an unbounded loop and multiplies load on the serving endpoint. The more immediate risk here is manual and automated updates race, creating unexpected model routing during production traffic.
Question 483
The engineering team encounters the following issue: A test engineer can create a model-serving endpoint in a sandbox but the production deployment fails because the automation identity lacks registered-model access. What direct consequence should the owner prioritize if no fix is made?
- The endpoint rejects requests or silently associates values with incorrect features.
- A malformed request crashes the entire serving worker or produces an untraceable score.
- Production changes fail late and teams are tempted to use overly broad administrator credentials.
- Reviewers cannot audit or grant precise access to a reproducible registered version.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. Production changes fail late and teams are tempted to use overly broad administrator credentials. The causal constraint is: Serving deployment permissions must include authorization to use the intended governed model and endpoint resources.
Incorrect Answers
A:That direct consequence is directed at a different custom model serving situation: A notebook uses the MLflow Deployments SDK to send a column-major payload to a custom model endpoint expecting a different documented dataframe schema. The more immediate risk here is production changes fail late and teams are tempted to use overly broad administrator credentials.
B:That direct consequence is directed at a different custom model serving situation: A custom PyFunc's predict method assumes every request contains a numeric feature that is optional under the documented inference schema. The more immediate risk here is production changes fail late and teams are tempted to use overly broad administrator credentials.
D:That direct consequence is directed at a different custom model serving situation: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. The more immediate risk here is production changes fail late and teams are tempted to use overly broad administrator credentials.
Question 484
The engineering team encounters the following issue: A fairness reviewer needs the difference in false-negative rates between two approved cohorts, but the monitor computes an aggregate without cohort labels. What direct consequence should the owner prioritize if no fix is made?
- Early-account fraud receives inadequate detection despite acceptable combined recall.
- Gradual cumulative drift becomes invisible despite material divergence from the trained population.
- A small but important region remains unavailable while overall service metrics seem healthy.
- A material disparity is hidden inside the overall average model performance.
Explanation
Correct Answer
D:This consequence follows from the specific unresolved condition. A material disparity is hidden inside the overall average model performance. The causal constraint is: A custom fairness comparison needs matched labels, model predictions and governed subgroup definitions.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An account-risk model's fraud recall is stable overall but unexpectedly falls for new customers in their first 30 days. The more immediate risk here is a material disparity is hidden inside the overall average model performance.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. The more immediate risk here is a material disparity is hidden inside the overall average model performance.
C:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. The more immediate risk here is a material disparity is hidden inside the overall average model performance.
Question 485
Consider the following fictional Databricks production case: A customer-risk monitoring job compares an unfiltered current high-risk segment to an all-customer historical baseline and interprets the difference as temporal drift. Which result is the strongest acceptance criterion after remediation?
- A deteriorating new-user fixture raises the slice metric without falsely altering established-user results.
- A synthetic request can be traced to a versioned, privacy-safe inference record and matching endpoint response.
- A deliberate new-customer regression is detected with sufficient labeled sample support.
- Matching-filter fixtures show no artificial drift while a real within-segment change remains detectable.
Explanation
Correct Answer
D:This verification measures the required result rather than just process liveness. Matching-filter fixtures show no artificial drift while a real within-segment change remains detectable. It confirms whether the targeted action succeeded: align baseline and current cohort definitions before computing temporal drift.
Incorrect Answers
A:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A recommender reports stable overall accuracy while click-through prediction quality for first-time users deteriorates sharply. Here the actual success criterion is matching-filter fixtures show no artificial drift while a real within-segment change remains detectable.
B:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: A serving endpoint appears healthy but nobody records model version, request identifier or prediction value, so a drift incident cannot be reconstructed. Here the actual success criterion is matching-filter fixtures show no artificial drift while a real within-segment change remains detectable.
C:That acceptance test is directed at a different drift detection and lakehouse monitoring situation: An account-risk model's fraud recall is stable overall but unexpectedly falls for new customers in their first 30 days. Here the actual success criterion is matching-filter fixtures show no artificial drift while a real within-segment change remains detectable.
Question 486
The engineering team encounters the following issue: A feature-refresh task writes rows with event time in local timezone while the training join assumes UTC; unit tests of each component pass. What investigation would most effectively isolate this behavior?
- Run a tiny constant-column fixture and inspect finite-value validation after transformation.
- Compare individual function tests with an end-to-end fixture crossing feature lookup and inference boundaries.
- Compare timestamp values after ingestion, feature-table write and training-set as-of join for one known event.
- Compare mock schemas with actual feature-table and serving contracts in a controlled environment.
Explanation
Correct Answer
C:This is the most discriminating check for the observed symptoms. Compare timestamp values after ingestion, feature-table write and training-set as-of join for one known event. The underlying technical issue is that cross-component tests must validate shared time semantics and feature availability throughout the ML pipeline.
Incorrect Answers
A:That diagnostic evidence is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. For the stated incident, the decisive observation is compare timestamp values after ingestion, feature-table write and training-set as-of join for one known event.
B:That diagnostic evidence is directed at a different validation testing situation: A team unit-tests each model component independently but its feature table and serving endpoint disagree on column names. For the stated incident, the decisive observation is compare timestamp values after ingestion, feature-table write and training-set as-of join for one known event.
D:That diagnostic evidence is directed at a different validation testing situation: A test suite mocks every external interface so aggressively that even an incompatible feature table schema appears valid. For the stated incident, the decisive observation is compare timestamp values after ingestion, feature-table write and training-set as-of join for one known event.
Question 487
Review this production model incident: A sensor feature pipeline computes rolling 30-minute averages by arrival time, although delayed events carry trustworthy event timestamps. Which failure risk follows most closely from the unresolved behavior?
- A repeated refresh doubles feature rows and multiplies downstream training joins.
- Late sensor readings are assigned to the wrong time window and mislead incident models.
- Scores pull the wrong customer's features or fail after name changes.
- Training-serving skew silently changes the meaning of risk thresholds.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. Late sensor readings are assigned to the wrong time window and mislead incident models. The causal constraint is: Streaming feature windows should use appropriate event-time semantics and a stated lateness policy.
Incorrect Answers
A:That direct consequence is directed at a different advanced feature store concepts situation: A feature refresh writes duplicate records for each entity and logical timestamp after retries, breaking unique lookups. The more immediate risk here is late sensor readings are assigned to the wrong time window and mislead incident models.
C:That direct consequence is directed at a different advanced feature store concepts situation: A low-latency feature table uses external customer names as lookup keys; names change and are not unique. The more immediate risk here is late sensor readings are assigned to the wrong time window and mislead incident models.
D:That direct consequence is directed at a different advanced feature store concepts situation: A customer-risk feature needs a ratio of current debt to income, but online serving recomputes it differently from the training pipeline. The more immediate risk here is late sensor readings are assigned to the wrong time window and mislead incident models.
Question 488
The engineering team encounters the following issue: An IoT classifier passes raw text categories directly into VectorAssembler ahead of StringIndexer and OneHotEncoder, generating incompatible vector inputs. What investigation would most effectively isolate this behavior?
- Inspect intermediate stage schemas and the inputCol/outputCol contract immediately before vector assembly.
- Inspect the estimator ParamGridBuilder maps, evaluator metric and fold configuration used for the claimed search.
- Compare training customer IDs with disjoint evaluation users or future interactions and inspect overlap.
- Measure the local estimator runtime against cluster initialization and task scheduling for the same sample.
Explanation
Correct Answer
A:This is the most discriminating check for the observed symptoms. Inspect intermediate stage schemas and the inputCol/outputCol contract immediately before vector assembly. The underlying technical issue is that a SparkML Pipeline requires transformer stages in an order that produces the input types expected by later stages.
Incorrect Answers
B:That diagnostic evidence is directed at a different using spark ml situation: An ML engineer manually compares two logistic-regression regularization values on a single holdout but has no reusable SparkML parameter-search plan. For the stated incident, the decisive observation is inspect intermediate stage schemas and the inputCol/outputCol contract immediately before vector assembly.
C:That diagnostic evidence is directed at a different using spark ml situation: A recommender experiment computes all evaluation metrics on training interactions and reports the result as generalization to new customers. For the stated incident, the decisive observation is inspect intermediate stage schemas and the inputCol/outputCol contract immediately before vector assembly.
D:That diagnostic evidence is directed at a different using spark ml situation: A laboratory runs a linear regression every quarter on only 280 clean observations and proposes a 48-worker Spark cluster to reduce a two-second local fit. For the stated incident, the decisive observation is inspect intermediate stage schemas and the inputCol/outputCol contract immediately before vector assembly.
Question 489
Review this production model incident: A production deployment scales to 30 teams but all jobs share one compute pool with no capacity limits or cost attribution. Which failure risk follows most closely from the unresolved behavior?
- The deployed resources exist but cannot run due to missing experiment or model access.
- A test release mutates production model aliases despite otherwise successful deployment.
- One experiment starves critical retraining or inference workloads.
- Manual drift creates unreproducible endpoint permissions or traffic settings.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. One experiment starves critical retraining or inference workloads. The causal constraint is: Multi-team ML environments need enforceable resource policies and observability to prevent noisy-neighbor failure.
Incorrect Answers
A:That direct consequence is directed at a different environment architectures situation: A continuous-delivery job deploys a notebook change through a bundle but skips validation of referenced experiment paths and job permissions. The more immediate risk here is one experiment starves critical retraining or inference workloads.
B:That direct consequence is directed at a different environment architectures situation: A bundle references a model registry name without qualifying its environment, and a development deployment accidentally updates production registered-model aliases. The more immediate risk here is one experiment starves critical retraining or inference workloads.
D:That direct consequence is directed at a different environment architectures situation: A model serving endpoint is created manually in the UI and every staging deployment differs subtly from production because no deployment definition is saved. The more immediate risk here is one experiment starves critical retraining or inference workloads.
Question 490
Consider the following fictional Databricks production case: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. Which observation most directly tests the suspected failure boundary?
- Compare current endpoint settings with source-controlled deployment configuration and documented backup state.
- Compare response creation time, endpoint model version and cache keys before and after rollback.
- Compare sandbox/prod identities, model grants and sanitized deployment authorization errors.
- Inspect model artifact source, planned three-level Unity Catalog name and effective registration permissions.
Explanation
Correct Answer
D:This is the most discriminating check for the observed symptoms. Inspect model artifact source, planned three-level Unity Catalog name and effective registration permissions. The underlying technical issue is that governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
Incorrect Answers
A:That diagnostic evidence is directed at a different custom model serving situation: A platform engineer creates a custom serving endpoint manually and assumes the exact settings can be recovered after an accidental deletion. For the stated incident, the decisive observation is inspect model artifact source, planned three-level Unity Catalog name and effective registration permissions.
B:That diagnostic evidence is directed at a different custom model serving situation: An integration caches predictions without recording the model version and returns yesterday's values after a controlled rollback. For the stated incident, the decisive observation is inspect model artifact source, planned three-level Unity Catalog name and effective registration permissions.
C:That diagnostic evidence is directed at a different custom model serving situation: A test engineer can create a model-serving endpoint in a sandbox but the production deployment fails because the automation identity lacks registered-model access. For the stated incident, the decisive observation is inspect model artifact source, planned three-level Unity Catalog name and effective registration permissions.