Amazon AWS Certified Machine Learning Engineer - Associate MLA-C01 Practice Test 26
Question 250
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A missing-value imputer calculates its median on the complete labeled dataset before train/test splitting, inflating apparent validation performance. The dataset contract must remain reproducible for a backfill. Which investigation provides the most decisive FIRST evidence for the underlying problem?
- Inspect sampler placement, duplicate neighbor IDs, fold membership, and class-conditioned metric changes.
- Measure file-size distribution, object request count, Spark task scheduling overhead, and read throughput.
- Inspect feature scale distributions, model sensitivity, and whether fitted scaler statistics are shared across serving and training.
- Inspect preprocessing fit order, imputation statistics, dataset split boundaries, and pipeline serialization.
Explanation
Correct Answer
D:Inspect preprocessing fit order, imputation statistics, dataset split boundaries, and pipeline serialization. This directly tests median imputation fitted on training split in the reported AWS ML workflow. Preprocessing parameters learned from held-out records leak information into model selection. It distinguishes the problem before choosing a change. The related targeted response is: Split first; fit the imputer only on the training partition and apply those learned statistics to validation and test data. Reported signal: A missing-value imputer calculates its median on the complete labeled dataset before train/test splitting, inflating apparent validation performance. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
A:Inspect sampler placement, duplicate neighbor IDs, fold membership, and class-conditioned metric changes. This is a possible diagnostic for class imbalance handled inside cross-validation, but it does not directly test the median imputation fitted on training split failure reported here. Prioritize: Inspect preprocessing fit order, imputation statistics, dataset split boundaries, and pipeline serialization. The relevant case is: A missing-value imputer calculates its median on the complete labeled dataset before train/test splitting, inflating apparent validation performance. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
B:Measure file-size distribution, object request count, Spark task scheduling overhead, and read throughput. This is a possible diagnostic for columnar file compaction for small objects, but it does not directly test the median imputation fitted on training split failure reported here. Prioritize: Inspect preprocessing fit order, imputation statistics, dataset split boundaries, and pipeline serialization. The relevant case is: A missing-value imputer calculates its median on the complete labeled dataset before train/test splitting, inflating apparent validation performance. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Inspect feature scale distributions, model sensitivity, and whether fitted scaler statistics are shared across serving and training. This is a possible diagnostic for standardization of numerical features, but it does not directly test the median imputation fitted on training split failure reported here. Prioritize: Inspect preprocessing fit order, imputation statistics, dataset split boundaries, and pipeline serialization. The relevant case is: A missing-value imputer calculates its median on the complete labeled dataset before train/test splitting, inflating apparent validation performance. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 251
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Change control requires measurable endpoint or pipeline acceptance evidence. What should the ML engineer inspect FIRST to distinguish this failure from unrelated issues?
- Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage.
- Inspect preprocessing version drift, input/output contracts, and the selected hosting container topology.
- Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories.
- Inspect container startup logs, listening port, health endpoint, MIME types, and inference response contract.
Explanation
Correct Answer
C:Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories. This directly tests eventbridge conditional retraining trigger in the reported AWS ML workflow. Event-driven automation needs selective event patterns and readiness checks; every object-write event is not a retraining signal. It distinguishes the problem before choosing a change. The related targeted response is: Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete. Reported signal: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
A:Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage. This is a possible diagnostic for sagemaker pipelines parameterized retraining, but it does not directly test the eventbridge conditional retraining trigger failure reported here. Prioritize: Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories. The relevant case is: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
B:Inspect preprocessing version drift, input/output contracts, and the selected hosting container topology. This is a possible diagnostic for multi-container inference pipeline responsibilities, but it does not directly test the eventbridge conditional retraining trigger failure reported here. Prioritize: Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories. The relevant case is: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
D:Inspect container startup logs, listening port, health endpoint, MIME types, and inference response contract. This is a possible diagnostic for bring-your-own-inference-container contract, but it does not directly test the eventbridge conditional retraining trigger failure reported here. Prioritize: Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories. The relevant case is: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Question 252
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Change control requires measurable endpoint or pipeline acceptance evidence. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Provide an approved S3 gateway or interface endpoint path and least-privilege bucket/role permissions without unnecessarily opening internet egress.
- Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs.
- Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete.
- Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages.
- Inspect preprocessing version drift, input/output contracts, and the selected hosting container topology.
Explanation
Correct Answers
B:Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. This directly tests model registry version promotion provenance in the reported AWS ML workflow. Immutable package versions and approval metadata connect evaluation with deployment; mutable S3 nicknames do not. It distinguishes the problem before choosing a change. The related targeted response is: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
D:Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. This is the appropriate controlled remedy for model registry version promotion provenance. Immutable package versions and approval metadata connect evaluation with deployment; mutable S3 nicknames do not. Verify the cause with: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. Then confirm: The production endpoint matches the reviewed model package and auditor can trace its evaluation and approver. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
A:Provide an approved S3 gateway or interface endpoint path and least-privilege bucket/role permissions without unnecessarily opening internet egress. This selection concerns sagemaker vpc endpoint access for private artifacts rather than the required diagnostic-and-remedy pair for model registry version promotion provenance. Use evidence: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. and the matching correction: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. Signal: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
C:Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete. This selection concerns eventbridge conditional retraining trigger rather than the required diagnostic-and-remedy pair for model registry version promotion provenance. Use evidence: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. and the matching correction: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. Signal: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
E:Inspect preprocessing version drift, input/output contracts, and the selected hosting container topology. This selection concerns multi-container inference pipeline responsibilities rather than the required diagnostic-and-remedy pair for model registry version promotion provenance. Use evidence: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. and the matching correction: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. Signal: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Question 253
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Change control requires measurable endpoint or pipeline acceptance evidence. Which observation most directly identifies the root cause of the reported behavior?
- Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories.
- Inspect target processor, supported Neo compiler configurations, operator compatibility, and deployment artifact format.
- Inspect invocation cadence, acceptable completion window, per-record latency requirement, and idle endpoint cost.
- Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs.
Explanation
Correct Answer
D:Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. This directly tests model registry version promotion provenance in the reported AWS ML workflow. Immutable package versions and approval metadata connect evaluation with deployment; mutable S3 nicknames do not. It distinguishes the problem before choosing a change. The related targeted response is: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. Reported signal: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
A:Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories. This is a possible diagnostic for eventbridge conditional retraining trigger, but it does not directly test the model registry version promotion provenance failure reported here. Prioritize: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. The relevant case is: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
B:Inspect target processor, supported Neo compiler configurations, operator compatibility, and deployment artifact format. This is a possible diagnostic for sagemaker neo edge compiler compatibility, but it does not directly test the model registry version promotion provenance failure reported here. Prioritize: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. The relevant case is: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
C:Inspect invocation cadence, acceptable completion window, per-record latency requirement, and idle endpoint cost. This is a possible diagnostic for real-time versus batch inference decision, but it does not directly test the model registry version promotion provenance failure reported here. Prioritize: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. The relevant case is: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Question 254
During an approved rollout of an AWS ML inference workload, the following condition is observed: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. The team must meet its documented availability and rollback constraints. What postchange observation provides the strongest acceptance evidence?
- Large inputs complete without client request timeout and clients receive tracked result locations or failure notifications.
- Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs.
- A deliberately poor candidate halts before production while a passing approved candidate can proceed.
- Distinct requested model IDs return correct predictions with acceptable cold load latency and lower aggregate idle cost.
Explanation
Correct Answer
B:Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. This is the acceptance evidence for the described sagemaker pipelines parameterized retraining scenario. Pipelines make dependencies explicit; manually ordered notebook operations are difficult to repeat or audit. The corrective implementation it validates is: Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts. Reported signal: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an approved rollout of an AWS ML inference workload.
Incorrect Answers
A:Large inputs complete without client request timeout and clients receive tracked result locations or failure notifications. This could validate asynchronous endpoint for large payloads, but it does not establish that sagemaker pipelines parameterized retraining is resolved for this workload. Confirm instead: Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. Observed problem: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an approved rollout of an AWS ML inference workload.
C:A deliberately poor candidate halts before production while a passing approved candidate can proceed. This could validate codepipeline approval gate on model metrics, but it does not establish that sagemaker pipelines parameterized retraining is resolved for this workload. Confirm instead: Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. Observed problem: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an approved rollout of an AWS ML inference workload.
D:Distinct requested model IDs return correct predictions with acceptable cold load latency and lower aggregate idle cost. This could validate multi-model endpoint hosting tradeoff, but it does not establish that sagemaker pipelines parameterized retraining is resolved for this workload. Confirm instead: Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. Observed problem: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an approved rollout of an AWS ML inference workload.
Question 255
At a production AWS ML data lake supporting customer-facing predictions, the following condition is observed: A customer-support labeling dataset includes raw email addresses and account identifiers that external annotators do not need. Engineers must avoid changing unrelated source partitions. Which TWO steps best establish the evidence and correct the identified root cause? Select TWO.
- Split by stable patient identity before training and reserve a disjoint held-out group for evaluation.
- Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions.
- Compact the partitioned dataset into appropriately sized Parquet objects without changing the record keys or schema.
- Compare online Feature Store record values and offline event-time keyed feature history for the same entity and model snapshot.
- Remove or pseudonymize unnecessary PII before annotation and restrict access to the token mapping under separate controls.
Explanation
Correct Answers
B:Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions. This directly tests pii masking before model labeling in the reported AWS ML workflow. Data minimization applies before external labeling; encryption in transit does not remove unnecessary sensitive content from the task. It distinguishes the problem before choosing a change. The related targeted response is: Remove or pseudonymize unnecessary PII before annotation and restrict access to the token mapping under separate controls. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A customer-support labeling dataset includes raw email addresses and account identifiers that external annotators do not need. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
E:Remove or pseudonymize unnecessary PII before annotation and restrict access to the token mapping under separate controls. This is the appropriate controlled remedy for pii masking before model labeling. Data minimization applies before external labeling; encryption in transit does not remove unnecessary sensitive content from the task. Verify the cause with: Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions. Then confirm: Annotation workers can label required content without viewing direct identifiers and authorized audits can trace approved mappings. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A customer-support labeling dataset includes raw email addresses and account identifiers that external annotators do not need. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Incorrect Answers
A:Split by stable patient identity before training and reserve a disjoint held-out group for evaluation. This selection concerns group-leakage train and validation split rather than the required diagnostic-and-remedy pair for pii masking before model labeling. Use evidence: Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions. and the matching correction: Remove or pseudonymize unnecessary PII before annotation and restrict access to the token mapping under separate controls. Signal: A customer-support labeling dataset includes raw email addresses and account identifiers that external annotators do not need. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
C:Compact the partitioned dataset into appropriately sized Parquet objects without changing the record keys or schema. This selection concerns columnar file compaction for small objects rather than the required diagnostic-and-remedy pair for pii masking before model labeling. Use evidence: Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions. and the matching correction: Remove or pseudonymize unnecessary PII before annotation and restrict access to the token mapping under separate controls. Signal: A customer-support labeling dataset includes raw email addresses and account identifiers that external annotators do not need. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
D:Compare online Feature Store record values and offline event-time keyed feature history for the same entity and model snapshot. This selection concerns sagemaker feature store online offline consistency rather than the required diagnostic-and-remedy pair for pii masking before model labeling. Use evidence: Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions. and the matching correction: Remove or pseudonymize unnecessary PII before annotation and restrict access to the token mapping under separate controls. Signal: A customer-support labeling dataset includes raw email addresses and account identifiers that external annotators do not need. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Question 256
As a regional data-pipeline rollout is validated against approved AWS storage and streams, the following condition is observed: A daily S3 dataset consists of millions of tiny Parquet files, and training startup spends more time opening objects than scanning data. The data team must preserve event-time meaning and lineage. Which verification result BEST demonstrates that the issue is resolved?
- Training read time and object operations decline while the resulting row count and checksums match.
- A controlled out-of-order replay yields the expected window totals and bounded late-event treatment.
- Two ordinary runs ingest only newly eligible objects while a separately invoked backfill handles prior intervals.
- Memory use stays within limits and previously unseen categories are processed without dimension mismatch.
Explanation
Correct Answer
A:Training read time and object operations decline while the resulting row count and checksums match. This is the acceptance evidence for the described columnar file compaction for small objects scenario. Excessive small files create metadata and request overhead even with a columnar data format. The corrective implementation it validates is: Compact the partitioned dataset into appropriately sized Parquet objects without changing the record keys or schema. Reported signal: A daily S3 dataset consists of millions of tiny Parquet files, and training startup spends more time opening objects than scanning data. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Incorrect Answers
B:A controlled out-of-order replay yields the expected window totals and bounded late-event treatment. This could validate streaming feature rolling window correctness, but it does not establish that columnar file compaction for small objects is resolved for this workload. Confirm instead: Training read time and object operations decline while the resulting row count and checksums match. Observed problem: A daily S3 dataset consists of millions of tiny Parquet files, and training startup spends more time opening objects than scanning data. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
C:Two ordinary runs ingest only newly eligible objects while a separately invoked backfill handles prior intervals. This could validate glue job bookmark state error, but it does not establish that columnar file compaction for small objects is resolved for this workload. Confirm instead: Training read time and object operations decline while the resulting row count and checksums match. Observed problem: A daily S3 dataset consists of millions of tiny Parquet files, and training startup spends more time opening objects than scanning data. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
D:Memory use stays within limits and previously unseen categories are processed without dimension mismatch. This could validate high-cardinality categorical encoding, but it does not establish that columnar file compaction for small objects is resolved for this workload. Confirm instead: Training read time and object operations decline while the resulting row count and checksums match. Observed problem: A daily S3 dataset consists of millions of tiny Parquet files, and training startup spends more time opening objects than scanning data. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Question 257
While reproducing the alert in an isolated AWS ML integration account, the following condition is observed: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. The proposed change must preserve logging and least privilege. Which TWO steps best establish the evidence and correct the identified root cause? Select TWO.
- Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed.
- Grant only the needed S3 and KMS actions for the approved data prefix and ensure appropriate role trust.
- Inspect S3 object encryption, key policy, grant permissions, IAM role, and denied KMS decrypt events.
- Compare probability calibration plots, Brier score and event rates across recent labeled cohorts.
- Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis.
Explanation
Correct Answers
A:Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. This is the appropriate controlled remedy for serving output confidence calibration drift. A good ranking metric does not guarantee calibrated probabilities; calibration must be verified against observed outcomes. Verify the cause with: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. Then confirm: Predicted probability buckets align with observed frequencies on held-out recent labeled cohorts. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
D:Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. This directly tests serving output confidence calibration drift in the reported AWS ML workflow. A good ranking metric does not guarantee calibrated probabilities; calibration must be verified against observed outcomes. It distinguishes the problem before choosing a change. The related targeted response is: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Incorrect Answers
B:Grant only the needed S3 and KMS actions for the approved data prefix and ensure appropriate role trust. This selection concerns sagemaker execution role s3 least privilege rather than the required diagnostic-and-remedy pair for serving output confidence calibration drift. Use evidence: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. and the matching correction: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. Signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
C:Inspect S3 object encryption, key policy, grant permissions, IAM role, and denied KMS decrypt events. This selection concerns s3 sse-kms cross-account model artifact access rather than the required diagnostic-and-remedy pair for serving output confidence calibration drift. Use evidence: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. and the matching correction: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. Signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
E:Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis. This selection concerns model monitor ground truth association rather than the required diagnostic-and-remedy pair for serving output confidence calibration drift. Use evidence: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. and the matching correction: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. Signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Question 258
During production evaluation of a SageMaker AI model candidate, the following condition is observed: A custom SageMaker training image exits before fitting because its entrypoint does not understand the job's expected channel paths and arguments. The release is blocked until the underlying model behavior is understood. Which measured outcome would confirm that the correction worked as intended?
- The model no longer relies on future-only flags and held-out results reflect features available at inference.
- Training stops before extended overfitting and the retained model corresponds to the best validated epoch.
- A rolling-origin backtest improves multi-step forecast error and respects temporal order.
- The managed training job starts the intended script and produces a usable model artifact in the output location.
Explanation
Correct Answer
D:The managed training job starts the intended script and produces a usable model artifact in the output location. This is the acceptance evidence for the described training image compatibility for byoc scenario. Custom containers must implement the hosting or training contract; building a valid Docker image alone is insufficient. The corrective implementation it validates is: Adapt the custom training container to SageMaker training job conventions and verify locally against a representative channel layout. Reported signal: A custom SageMaker training image exits before fitting because its entrypoint does not understand the job's expected channel paths and arguments. Operational setting: During production evaluation of a SageMaker AI model candidate.
Incorrect Answers
A:The model no longer relies on future-only flags and held-out results reflect features available at inference. This could validate clarify feature attribution against target leakage, but it does not establish that training image compatibility for byoc is resolved for this workload. Confirm instead: The managed training job starts the intended script and produces a usable model artifact in the output location. Observed problem: A custom SageMaker training image exits before fitting because its entrypoint does not understand the job's expected channel paths and arguments. Operational setting: During production evaluation of a SageMaker AI model candidate.
B:Training stops before extended overfitting and the retained model corresponds to the best validated epoch. This could validate early stopping on validation plateau, but it does not establish that training image compatibility for byoc is resolved for this workload. Confirm instead: The managed training job starts the intended script and produces a usable model artifact in the output location. Observed problem: A custom SageMaker training image exits before fitting because its entrypoint does not understand the job's expected channel paths and arguments. Operational setting: During production evaluation of a SageMaker AI model candidate.
C:A rolling-origin backtest improves multi-step forecast error and respects temporal order. This could validate time-series forecast horizon and seasonality, but it does not establish that training image compatibility for byoc is resolved for this workload. Confirm instead: The managed training job starts the intended script and produces a usable model artifact in the output location. Observed problem: A custom SageMaker training image exits before fitting because its entrypoint does not understand the job's expected channel paths and arguments. Operational setting: During production evaluation of a SageMaker AI model candidate.
Question 259
While reproducing the alert in an isolated AWS ML integration account, the following condition is observed: An A/B model test routes consecutive requests from the same user to different variants, contaminating the comparison with unstable user experience. The proposed change must preserve logging and least privilege. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect routing hashes, experiment assignments, variant weights, and request identity consistency.
- Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable.
- Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis.
- Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family.
- Inspect caller identity, pass-role resource ARN, service constraint, and denied event attributes.
Explanation
Correct Answers
A:Inspect routing hashes, experiment assignments, variant weights, and request identity consistency. This directly tests champion challenger a/b assignment consistency in the reported AWS ML workflow. Random assignment per request can mix treatments for one entity and bias longitudinal online experiments. It distinguishes the problem before choosing a change. The related targeted response is: Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An A/B model test routes consecutive requests from the same user to different variants, contaminating the comparison with unstable user experience. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
B:Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable. This is the appropriate controlled remedy for champion challenger a/b assignment consistency. Random assignment per request can mix treatments for one entity and bias longitudinal online experiments. Verify the cause with: Inspect routing hashes, experiment assignments, variant weights, and request identity consistency. Then confirm: Repeated requests for one test user reach the same assigned variant and measured outcomes remain comparable. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An A/B model test routes consecutive requests from the same user to different variants, contaminating the comparison with unstable user experience. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Incorrect Answers
C:Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis. This selection concerns model monitor ground truth association rather than the required diagnostic-and-remedy pair for champion challenger a/b assignment consistency. Use evidence: Inspect routing hashes, experiment assignments, variant weights, and request identity consistency. and the matching correction: Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable. Signal: An A/B model test routes consecutive requests from the same user to different variants, contaminating the comparison with unstable user experience. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
D:Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family. This selection concerns inference recommender benchmarking instance family rather than the required diagnostic-and-remedy pair for champion challenger a/b assignment consistency. Use evidence: Inspect routing hashes, experiment assignments, variant weights, and request identity consistency. and the matching correction: Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable. Signal: An A/B model test routes consecutive requests from the same user to different variants, contaminating the comparison with unstable user experience. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
E:Inspect caller identity, pass-role resource ARN, service constraint, and denied event attributes. This selection concerns iam passrole constraint for model deployment rather than the required diagnostic-and-remedy pair for champion challenger a/b assignment consistency. Use evidence: Inspect routing hashes, experiment assignments, variant weights, and request identity consistency. and the matching correction: Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable. Signal: An A/B model test routes consecutive requests from the same user to different variants, contaminating the comparison with unstable user experience. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Question 260
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: An overnight ML export runs expensive full-table queries on a production RDS database and causes application transaction latency spikes. The dataset contract must remain reproducible for a backfill. Which verification result BEST demonstrates that the issue is resolved?
- No synthetic or duplicated minority instance leaks into held-out folds, and minority-class recall reflects genuine unseen cases.
- Agreement on a held-out gold set rises without silently discarding rare legitimate classes.
- Training read time and object operations decline while the resulting row count and checksums match.
- Application p95 latency returns to target while exported records meet a measured completeness and staleness objective.
Explanation
Correct Answer
D:Application p95 latency returns to target while exported records meet a measured completeness and staleness objective. This is the acceptance evidence for the described rds extraction overload into ml scenario. Extract jobs contend with transactional queries; moving or bounding reads can isolate production workloads. The corrective implementation it validates is: Use controlled incremental extraction or a suitable read replica and throttle export jobs within the approved data freshness target. Reported signal: An overnight ML export runs expensive full-table queries on a production RDS database and causes application transaction latency spikes. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
A:No synthetic or duplicated minority instance leaks into held-out folds, and minority-class recall reflects genuine unseen cases. This could validate class imbalance handled inside cross-validation, but it does not establish that rds extraction overload into ml is resolved for this workload. Confirm instead: Application p95 latency returns to target while exported records meet a measured completeness and staleness objective. Observed problem: An overnight ML export runs expensive full-table queries on a production RDS database and causes application transaction latency spikes. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
B:Agreement on a held-out gold set rises without silently discarding rare legitimate classes. This could validate ground truth labeling agreement review, but it does not establish that rds extraction overload into ml is resolved for this workload. Confirm instead: Application p95 latency returns to target while exported records meet a measured completeness and staleness objective. Observed problem: An overnight ML export runs expensive full-table queries on a production RDS database and causes application transaction latency spikes. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Training read time and object operations decline while the resulting row count and checksums match. This could validate columnar file compaction for small objects, but it does not establish that rds extraction overload into ml is resolved for this workload. Confirm instead: Application p95 latency returns to target while exported records meet a measured completeness and staleness objective. Observed problem: An overnight ML export runs expensive full-table queries on a production RDS database and causes application transaction latency spikes. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
