Databricks Certified Machine Learning Professional Practice Test 62
Question 610
The engineering team encounters the following issue: A training notebook reports F1 for several models but only logs the final model's parameters in MLflow, leaving metrics and hyperparameters disconnected. What investigation would most effectively isolate this behavior?
- Compare original search ID, resumed run tags and candidate-set coverage after restoration.
- Compare run parameter maps against logged evaluation metrics and the local experiment output.
- Inspect logged model environment files and serving import errors for the missing package.
- Compare actual serving request schema, feature lookup capabilities and the model's predict signature.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Compare run parameter maps against logged evaluation metrics and the local experiment output. The underlying technical issue is that mLflow run-level logging should attach parameters and evaluated metrics to the corresponding candidate run.
Incorrect Answers
A:That diagnostic evidence is directed at a different advanced mlflow usage situation: A research team restarts a long hyperparameter sweep and merges all newly executed trials into a new experiment without connecting them to the original parent. For the stated incident, the decisive observation is compare run parameter maps against logged evaluation metrics and the local experiment output.
C:That diagnostic evidence is directed at a different advanced mlflow usage situation: A team logs a custom Python model that imports a package installed only in the author's notebook, and the serving endpoint fails when loading the artifact. For the stated incident, the decisive observation is compare run parameter maps against logged evaluation metrics and the local experiment output.
D:That diagnostic evidence is directed at a different advanced mlflow usage situation: A real-time churn endpoint accepts customer IDs and must fetch current allowed account features, but a custom model class instead expects the caller to supply the entire internal training row. For the stated incident, the decisive observation is compare run parameter maps against logged evaluation metrics and the local experiment output.
Question 611
In an enterprise ML workflow, the reported issue is: A custom model endpoint meets mean response time of 75 milliseconds but P99 latency reaches 1.8 seconds during peak load. Before modifying the pipeline, which evidence should be collected first?
- Review rolling-window statistics alongside a fixed approved baseline for the same cohort.
- Compare P95/P99 latency by request volume and concurrency at the same sampling interval.
- Inspect inference table version IDs and label-matched errors before and after traffic shift.
- Compare regional event counts and failure proportions against the aggregate alert trigger.
Explanation
Correct Answer
B:This is the most discriminating check for the observed symptoms. Compare P95/P99 latency by request volume and concurrency at the same sampling interval. The underlying technical issue is that tail latency can violate user experience even when mean latency appears healthy.
Incorrect Answers
A:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A streaming payments monitor always compares today with the previous hour, and a slow week-long distribution drift escapes because adjacent hours differ only slightly. For the stated incident, the decisive observation is compare P95/P99 latency by request volume and concurrency at the same sampling interval.
C:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: A scoring endpoint quietly moves from model v7 to v8 during a rollout but an inference trend aggregates both versions in one error curve. For the stated incident, the decisive observation is compare P95/P99 latency by request volume and concurrency at the same sampling interval.
D:That diagnostic evidence is directed at a different drift detection and lakehouse monitoring situation: An alert query uses one global transaction-failure count and masks a complete outage in a small high-value region. For the stated incident, the decisive observation is compare P95/P99 latency by request volume and concurrency at the same sampling interval.
Question 612
A production ML review contains this situation: A researcher parallelizes Optuna trials while sharing one writable SQLite file on a fragile network mount and observes duplicate reservations and storage locks. What production-safe change best matches the evidence?
- Use an appropriate concurrency-safe shared storage/backend and bounded worker count with reproducible MLflow logging.
- Use an appropriate Ray actor-based framework while reserving Spark for eligible data preprocessing.
- Declare correct per-trial GPU reservations and reduce concurrent trials until memory fits.
- Use framework-managed synchronized updates and reproducible checkpoints for the parallel training loop.
Explanation
Correct Answer
A:This corrects the relevant failure boundary instead of merely hiding the symptom. Use an appropriate concurrency-safe shared storage/backend and bounded worker count with reproducible MLflow logging. The issue matters because trials are lost or repeated even when compute nodes appear healthy.
Incorrect Answers
B:That corrective action is directed at a different scaling and tuning situation: A reinforcement-learning simulator requires stateful actors exchanging messages and irregular task graphs, but the team plans one Spark SQL groupBy per simulation step. The relevant corrective action here is use an appropriate concurrency-safe shared storage/backend and bounded worker count with reproducible MLflow logging.
C:That corrective action is directed at a different scaling and tuning situation: A Ray Tune training function requests zero GPU resources but initializes a large GPU model inside every trial. The relevant corrective action here is use an appropriate concurrency-safe shared storage/backend and bounded worker count with reproducible MLflow logging.
D:That corrective action is directed at a different scaling and tuning situation: A training team averages gradients from multiple workers but lets each worker use a different untracked version of the model. The relevant corrective action here is use an appropriate concurrency-safe shared storage/backend and bounded worker count with reproducible MLflow logging.
Question 613
Consider the following fictional Databricks production case: A binary model's average predicted probability remains 0.4, but outcomes show that 0.8 predictions are now correct only half the time. What underlying principle determines the most appropriate interpretation?
- An alert is operationally useful only with explicit routing, threshold, urgency and escalation ownership.
- Serving availability errors are distinct from model predictive accuracy and require infrastructure telemetry.
- Time-of-day slices require explicit timezone interpretation and region-specific boundaries.
- Calibration drift requires comparing predicted probabilities with observed outcome frequencies, not just prediction means.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. Calibration drift requires comparing predicted probabilities with observed outcome frequencies, not just prediction means. To test it, bucket predictions by confidence and compare matured empirical positive rates for each bucket.
Incorrect Answers
A:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A fraud-feature drift monitor crosses its threshold but the alert has no assigned recipient and no incident ownership. The current incident instead turns on calibration drift requires comparing predicted probabilities with observed outcome frequencies, not just prediction means.
B:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A prediction endpoint returns HTTP 503 for intermittent bursts while the model's offline accuracy remains unchanged. The current incident instead turns on calibration drift requires comparing predicted probabilities with observed outcome frequencies, not just prediction means.
C:That underlying principle is directed at a different drift detection and lakehouse monitoring situation: A model's prediction error appears to spike at 2 a.m. local time but the monitoring table groups timestamps in UTC for a globally distributed region. The current incident instead turns on calibration drift requires comparing predicted probabilities with observed outcome frequencies, not just prediction means.
Question 614
Review this production model incident: A team wants to convert several numeric inputs into one features vector but treats VectorAssembler as a clustering Estimator that should output predictions. Which failure risk follows most closely from the unresolved behavior?
- A predictor is given nonexistent features or operators expect labels from a nonpredictive stage.
- Pipeline fitting fails with missing-column or mismatched-schema errors.
- The single driver can exhaust memory before training begins despite ample executor capacity.
- The same raw category maps to a different vector position and produces misleading predictions.
Explanation
Correct Answer
A:This consequence follows from the specific unresolved condition. A predictor is given nonexistent features or operators expect labels from a nonpredictive stage. The causal constraint is: VectorAssembler is a Transformer that creates a vector column, not a trainable predictor.
Incorrect Answers
B:That direct consequence is directed at a different using spark ml situation: A pipeline engineer fits a vectorizer after an estimator in the stage list and then tries to use the estimator's prediction as the vectorizer input. The more immediate risk here is a predictor is given nonexistent features or operators expect labels from a nonpredictive stage.
C:That direct consequence is directed at a different using spark ml situation: A retailer trains a demand classifier on 180 million Delta rows already partitioned across a Spark cluster, but an analyst attempts to convert the entire training set to a local pandas object. The more immediate risk here is a predictor is given nonexistent features or operators expect labels from a nonpredictive stage.
D:That direct consequence is directed at a different using spark ml situation: A batch scoring service loads a classifier version from last month while using today's new feature indexer, silently changing categorical positions. The more immediate risk here is a predictor is given nonexistent features or operators expect labels from a nonpredictive stage.
Question 615
The engineering team encounters the following issue: A rollout updates the serving model but a renamed input column breaks requests from an older mobile client. What corrective action should the engineer prefer?
- Use a supported routing design or keep the protected region on the approved model until scoped rollout is available.
- Prepare a complete tested green environment and switch traffic after approval while retaining blue for rollback.
- Send a small approved percentage of eligible live requests to the candidate and expand only after defined gates pass.
- Add a version-compatible request adapter or coordinate client migration with explicit acceptance gates.
Explanation
Correct Answer
D:This corrects the relevant failure boundary instead of merely hiding the symptom. Add a version-compatible request adapter or coordinate client migration with explicit acceptance gates. The issue matters because older clients receive validation errors even though new client requests succeed.
Incorrect Answers
A:That corrective action is directed at a different deployment strategies situation: A model release requires all clients from a restricted region to keep the old version, but a simple percentage split sends some of those clients to the candidate. The relevant corrective action here is add a version-compatible request adapter or coordinate client migration with explicit acceptance gates.
B:That corrective action is directed at a different deployment strategies situation: A bank requires an immediate rollback after deployment and wants the prior serving environment intact until the new environment passes tests. The relevant corrective action here is add a version-compatible request adapter or coordinate client migration with explicit acceptance gates.
C:That corrective action is directed at a different deployment strategies situation: A travel site serves 30,000 bookings per minute and wants to evaluate a new fraud model while limiting customer impact during its first hour. The relevant corrective action here is add a version-compatible request adapter or coordinate client migration with explicit acceptance gates.
Question 616
A production ML review contains this situation: A serving client calls the REST endpoint expecting one number but the custom PyFunc returns a structured list of prediction and confidence fields. Which statement best explains the failure mode?
- Client integrations should parse the documented model output contract instead of assuming a scalar response.
- Custom Python-model artifacts need portable packaged dependencies rather than absolute developer-local file paths.
- Endpoint deployment requires controlled ownership and conflict-safe update/versioning practices.
- Governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
Explanation
Correct Answer
A:The technical constraint is specific to the reported failure. Client integrations should parse the documented model output contract instead of assuming a scalar response. To test it, inspect actual test response JSON and registered output schema for a representative batch.
Incorrect Answers
B:That underlying principle is directed at a different custom model serving situation: A custom PyFunc estimator loads a tokenizer file from an engineer's laptop path when called in a serving container. The current incident instead turns on client integrations should parse the documented model output contract instead of assuming a scalar response.
C:That underlying principle is directed at a different custom model serving situation: A team edits a model endpoint through the UI while its CI pipeline concurrently applies an older configuration and reverts the intended traffic split. The current incident instead turns on client integrations should parse the documented model output contract instead of assuming a scalar response.
D:That underlying principle is directed at a different custom model serving situation: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. The current incident instead turns on client integrations should parse the documented model output contract instead of assuming a scalar response.
Question 617
In an enterprise ML workflow, the reported issue is: An end-to-end validation uses only a synthetic positive label and fails to detect a swapped positive/negative class mapping in inference. What technical distinction is essential to this investigation?
- Separating pure functions from orchestration can make unit tests faster while retaining notebook-driven workflows.
- Integration tests verify handoffs between components that component-level unit tests cannot validate.
- Integration fixtures must check semantic prediction values and class labels, not only response existence.
- Deterministic function tests should include degenerate data distributions that violate assumed mathematical denominators.
Explanation
Correct Answer
C:The technical constraint is specific to the reported failure. Integration fixtures must check semantic prediction values and class labels, not only response existence. To test it, compare class-index mappings and positive/negative fixture outputs from training to serving.
Incorrect Answers
A:That underlying principle is directed at a different validation testing situation: A team hides all preprocessing logic inside one long notebook, making it hard to test independently without starting a Spark cluster. The current incident instead turns on integration fixtures must check semantic prediction values and class labels, not only response existence.
B:That underlying principle is directed at a different validation testing situation: A team unit-tests each model component independently but its feature table and serving endpoint disagree on column names. The current incident instead turns on integration fixtures must check semantic prediction values and class labels, not only response existence.
D:That underlying principle is directed at a different validation testing situation: A preprocessing function divides by a standard deviation and returns infinity for a constant numeric column. The current incident instead turns on integration fixtures must check semantic prediction values and class labels, not only response existence.
Question 618
Review this production model incident: A research group wants to register a custom classification model in Unity Catalog but uses a local path as its complete model identity and has no governed model name. What is the central engineering constraint behind this behavior?
- Serving deployment permissions must include authorization to use the intended governed model and endpoint resources.
- Serving inference payloads must conform to the endpoint and model's accepted request/response schema.
- Repeatable endpoint deployment should preserve a controlled configuration definition and registered model identity.
- Governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
Explanation
Correct Answer
D:The technical constraint is specific to the reported failure. Governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions. To test it, inspect model artifact source, planned three-level Unity Catalog name and effective registration permissions.
Incorrect Answers
A:That underlying principle is directed at a different custom model serving situation: A test engineer can create a model-serving endpoint in a sandbox but the production deployment fails because the automation identity lacks registered-model access. The current incident instead turns on governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
B:That underlying principle is directed at a different custom model serving situation: A notebook uses the MLflow Deployments SDK to send a column-major payload to a custom model endpoint expecting a different documented dataframe schema. The current incident instead turns on governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
C:That underlying principle is directed at a different custom model serving situation: A platform engineer creates a custom serving endpoint manually and assumes the exact settings can be recovered after an accidental deletion. The current incident instead turns on governed custom-model registration requires a model artifact plus catalog/schema/model identity and appropriate permissions.
Question 619
In an enterprise ML workflow, the reported issue is: A customer product-category feature acquires many new category values and analysts try to detect the change using only the overall row count. What likely operational or model-quality harm does this issue create?
- Real drift is erased because the comparator silently changes with the observed sample.
- Temporal drift is hidden when a cumulative snapshot averages past and current behavior.
- The classifier faces unseen or redistributed categories while ingestion volume looks healthy.
- The dashboard reports yesterday's metrics as current and misses a real shift.
Explanation
Correct Answer
C:This consequence follows from the specific unresolved condition. The classifier faces unseen or redistributed categories while ingestion volume looks healthy. The causal constraint is: Categorical distribution drift needs category frequency or supported categorical distance measures, not only volume.
Incorrect Answers
A:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A baseline table is overwritten every night with the current day's data before a drift calculation, making the comparison always look unchanged. The more immediate risk here is the classifier faces unseen or redistributed categories while ingestion volume looks healthy.
B:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A fleet sensor table records measurement timestamps and analysts need daily drift trends; a snapshot-only profile erases the time dimension. The more immediate risk here is the classifier faces unseen or redistributed categories while ingestion volume looks healthy.
D:That direct consequence is directed at a different drift detection and lakehouse monitoring situation: A team has accurate inference logs but its monitoring job never refreshes computed drift metrics after a table schema change. The more immediate risk here is the classifier faces unseen or redistributed categories while ingestion volume looks healthy.
Question 620
Consider the following fictional Databricks production case: A laboratory runs a linear regression every quarter on only 280 clean observations and proposes a 48-worker Spark cluster to reduce a two-second local fit. What consequence most specifically follows from leaving the issue unchanged?
- Rare but severe forecast misses can trigger inventory shortages while average reporting looks healthy.
- Complexity and cost increase while the tiny fit becomes slower rather than faster.
- A predictor is given nonexistent features or operators expect labels from a nonpredictive stage.
- The supposedly unseen holdout can affect model inputs and inflate reported generalization.
Explanation
Correct Answer
B:This consequence follows from the specific unresolved condition. Complexity and cost increase while the tiny fit becomes slower rather than faster. The causal constraint is: Small workloads may run more efficiently on one node because Spark startup, scheduling and serialization have fixed costs.
Incorrect Answers
A:That direct consequence is directed at a different using spark ml situation: A forecasting model has a few catastrophic numeric errors hidden by a low median absolute error and operations care about large misses. The more immediate risk here is complexity and cost increase while the tiny fit becomes slower rather than faster.
C:That direct consequence is directed at a different using spark ml situation: A team wants to convert several numeric inputs into one features vector but treats VectorAssembler as a clustering Estimator that should output predictions. The more immediate risk here is complexity and cost increase while the tiny fit becomes slower rather than faster.
D:That direct consequence is directed at a different using spark ml situation: A credit-risk pipeline fits StringIndexer on the entire dataset before splitting train and holdout, and rare holdout labels influence its learned category mapping. The more immediate risk here is complexity and cost increase while the tiny fit becomes slower rather than faster.