Amazon AWS Certified Machine Learning Engineer - Associate MLA-C01 Practice Test 29
Question 280
During production evaluation of a SageMaker AI model candidate, the following condition is observed: A neural network continues training for many epochs after validation loss begins increasing while training loss keeps falling. The release is blocked until the underlying model behavior is understood. Which investigation provides the most decisive FIRST evidence for the underlying problem?
- Assess tabular feature structure, sample size, baseline metrics, inference constraints, and model interpretability requirements.
- Inspect feature importance stability, variance across folds, regularization and model dimensionality.
- Inspect routing mode, shadow traffic duplication, inference latency, output divergence, and payload compatibility.
- Inspect epoch-level training and validation curves, stopping criteria, and the best checkpoint location.
Explanation
Correct Answer
D:Inspect epoch-level training and validation curves, stopping criteria, and the best checkpoint location. This directly tests early stopping on validation plateau in the reported AWS ML workflow. More epochs are not always better; validation loss can worsen while training loss continues to decrease. It distinguishes the problem before choosing a change. The related targeted response is: Enable patience-based early stopping and restore the checkpoint with the best validation metric. Reported signal: A neural network continues training for many epochs after validation loss begins increasing while training loss keeps falling. Operational setting: During production evaluation of a SageMaker AI model candidate.
Incorrect Answers
A:Assess tabular feature structure, sample size, baseline metrics, inference constraints, and model interpretability requirements. This is a possible diagnostic for xgboost tabular classification baseline, but it does not directly test the early stopping on validation plateau failure reported here. Prioritize: Inspect epoch-level training and validation curves, stopping criteria, and the best checkpoint location. The relevant case is: A neural network continues training for many epochs after validation loss begins increasing while training loss keeps falling. Operational setting: During production evaluation of a SageMaker AI model candidate.
B:Inspect feature importance stability, variance across folds, regularization and model dimensionality. This is a possible diagnostic for feature selection to reduce overparameterization, but it does not directly test the early stopping on validation plateau failure reported here. Prioritize: Inspect epoch-level training and validation curves, stopping criteria, and the best checkpoint location. The relevant case is: A neural network continues training for many epochs after validation loss begins increasing while training loss keeps falling. Operational setting: During production evaluation of a SageMaker AI model candidate.
C:Inspect routing mode, shadow traffic duplication, inference latency, output divergence, and payload compatibility. This is a possible diagnostic for shadow deployment model evaluation without response change, but it does not directly test the early stopping on validation plateau failure reported here. Prioritize: Inspect epoch-level training and validation curves, stopping criteria, and the best checkpoint location. The relevant case is: A neural network continues training for many epochs after validation loss begins increasing while training loss keeps falling. Operational setting: During production evaluation of a SageMaker AI model candidate.
Question 281
As a regional data-pipeline rollout is validated against approved AWS storage and streams, the following condition is observed: A near-real-time account feature sums transactions using processing time, so late-arriving events land in the wrong window and produce inconsistent scores. The data team must preserve event-time meaning and lineage. What postchange observation provides the strongest acceptance evidence?
- Application p95 latency returns to target while exported records meet a measured completeness and staleness objective.
- A controlled out-of-order replay yields the expected window totals and bounded late-event treatment.
- The recovery run reads a complete expected interval from retained stream records or durable archive without silent gaps.
- Neighboring times across midnight map close in cyclic feature space and validation improves without leakage.
Explanation
Correct Answer
B:A controlled out-of-order replay yields the expected window totals and bounded late-event treatment. This is the acceptance evidence for the described streaming feature rolling window correctness scenario. Processing-time windows depend on arrival timing; event-time windows need explicit watermarks and late-event policy. The corrective implementation it validates is: Define event-time windows with appropriate lateness handling and a documented revision policy for late records. Reported signal: A near-real-time account feature sums transactions using processing time, so late-arriving events land in the wrong window and produce inconsistent scores. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Incorrect Answers
A:Application p95 latency returns to target while exported records meet a measured completeness and staleness objective. This could validate rds extraction overload into ml, but it does not establish that streaming feature rolling window correctness is resolved for this workload. Confirm instead: A controlled out-of-order replay yields the expected window totals and bounded late-event treatment. Observed problem: A near-real-time account feature sums transactions using processing time, so late-arriving events land in the wrong window and produce inconsistent scores. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
C:The recovery run reads a complete expected interval from retained stream records or durable archive without silent gaps. This could validate kinesis consumer retention gap, but it does not establish that streaming feature rolling window correctness is resolved for this workload. Confirm instead: A controlled out-of-order replay yields the expected window totals and bounded late-event treatment. Observed problem: A near-real-time account feature sums transactions using processing time, so late-arriving events land in the wrong window and produce inconsistent scores. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
D:Neighboring times across midnight map close in cyclic feature space and validation improves without leakage. This could validate time-series cyclical calendar feature, but it does not establish that streaming feature rolling window correctness is resolved for this workload. Confirm instead: A controlled out-of-order replay yields the expected window totals and bounded late-event treatment. Observed problem: A near-real-time account feature sums transactions using processing time, so late-arriving events land in the wrong window and produce inconsistent scores. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Question 282
As a regional data-pipeline rollout is validated against approved AWS storage and streams, the following condition is observed: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. The data team must preserve event-time meaning and lineage. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving.
- Define point-in-time offline joins, freshness SLAs, and aligned feature version logic for online retrieval.
- Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale.
- Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions.
- Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports.
Explanation
Correct Answers
A:Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. This is the appropriate controlled remedy for skewed numerical feature log transformation. A monotonic log transform can reduce heavy-tail influence, but invalid domains and train-serving parity must be handled. Verify the cause with: Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. Then confirm: The transformed distribution is less skewed and inference uses identical guarded transformations. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
C:Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. This directly tests skewed numerical feature log transformation in the reported AWS ML workflow. A monotonic log transform can reduce heavy-tail influence, but invalid domains and train-serving parity must be handled. It distinguishes the problem before choosing a change. The related targeted response is: Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Incorrect Answers
B:Define point-in-time offline joins, freshness SLAs, and aligned feature version logic for online retrieval. This selection concerns sagemaker feature store online offline consistency rather than the required diagnostic-and-remedy pair for skewed numerical feature log transformation. Use evidence: Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. and the matching correction: Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. Signal: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
D:Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions. This selection concerns pii masking before model labeling rather than the required diagnostic-and-remedy pair for skewed numerical feature log transformation. Use evidence: Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. and the matching correction: Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. Signal: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
E:Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports. This selection concerns model training data residency boundary rather than the required diagnostic-and-remedy pair for skewed numerical feature log transformation. Use evidence: Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. and the matching correction: Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. Signal: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Question 283
As a regulated ML workload prepares for an automated regional rollout, the following condition is observed: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. The team must demonstrate a repeatable monitoring or security control. What targeted change should be implemented after verifying the stated cause?
- Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks.
- Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting.
- Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes.
- Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it.
Explanation
Correct Answer
A:Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. This is the appropriate controlled remedy for sagemaker model monitor feature drift baseline. Endpoint uptime does not demonstrate feature-distribution stability; monitoring compares current inputs with expected statistics. Verify the cause with: Compare incoming captured feature statistics against a stored training baseline and review SageMaker Model Monitor drift alerts. Then confirm: Changed feature distributions produce attributable violations even while endpoint availability remains healthy. Reported signal: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Incorrect Answers
B:Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. This addresses a different issue involving tag-based ml training cost attribution. It does not correct the stated sagemaker model monitor feature drift baseline mechanism. The targeted response is: Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. Case evidence: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
C:Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes. This addresses a different issue involving clarify postdeployment group bias monitoring. It does not correct the stated sagemaker model monitor feature drift baseline mechanism. The targeted response is: Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. Case evidence: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
D:Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. This addresses a different issue involving spot training cost versus interruption recovery. It does not correct the stated sagemaker model monitor feature drift baseline mechanism. The targeted response is: Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. Case evidence: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Question 284
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A regulated ML pipeline copies EU-only customer records into an unapproved cross-region training bucket for convenience. The dataset contract must remain reproducible for a backfill. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports.
- Inspect the stream retention setting, consumer checkpoint timestamps, and the oldest available sequence or time position.
- Use rolling-origin or chronological validation with strictly past-derived features and appropriate forecast horizons.
- Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features.
- Inspect S3 replication rules, region-specific bucket policies, Glue job destinations, KMS keys, and applicable residency controls.
Explanation
Correct Answers
A:Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports. This is the appropriate controlled remedy for model training data residency boundary. Encrypting copied data does not automatically satisfy geographic residency constraints; placement and access must be controlled. Verify the cause with: Inspect S3 replication rules, region-specific bucket policies, Glue job destinations, KMS keys, and applicable residency controls. Then confirm: A test batch remains in the approved region with enforceable policy and audit evidence for all derived artifacts. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A regulated ML pipeline copies EU-only customer records into an unapproved cross-region training bucket for convenience. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
E:Inspect S3 replication rules, region-specific bucket policies, Glue job destinations, KMS keys, and applicable residency controls. This directly tests model training data residency boundary in the reported AWS ML workflow. Encrypting copied data does not automatically satisfy geographic residency constraints; placement and access must be controlled. It distinguishes the problem before choosing a change. The related targeted response is: Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A regulated ML pipeline copies EU-only customer records into an unapproved cross-region training bucket for convenience. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
B:Inspect the stream retention setting, consumer checkpoint timestamps, and the oldest available sequence or time position. This selection concerns kinesis consumer retention gap rather than the required diagnostic-and-remedy pair for model training data residency boundary. Use evidence: Inspect S3 replication rules, region-specific bucket policies, Glue job destinations, KMS keys, and applicable residency controls. and the matching correction: Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports. Signal: A regulated ML pipeline copies EU-only customer records into an unapproved cross-region training bucket for convenience. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Use rolling-origin or chronological validation with strictly past-derived features and appropriate forecast horizons. This selection concerns chronological split for forecast evaluation rather than the required diagnostic-and-remedy pair for model training data residency boundary. Use evidence: Inspect S3 replication rules, region-specific bucket policies, Glue job destinations, KMS keys, and applicable residency controls. and the matching correction: Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports. Signal: A regulated ML pipeline copies EU-only customer records into an unapproved cross-region training bucket for convenience. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
D:Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features. This selection concerns glue schema evolution with mixed input types rather than the required diagnostic-and-remedy pair for model training data residency boundary. Use evidence: Inspect S3 replication rules, region-specific bucket policies, Glue job destinations, KMS keys, and applicable residency controls. and the matching correction: Keep restricted records and dependent training artifacts in approved regions and deny unauthorized replication or cross-region exports. Signal: A regulated ML pipeline copies EU-only customer records into an unapproved cross-region training bucket for convenience. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 285
During an approved rollout of an AWS ML inference workload, the following condition is observed: A pipeline pushes a newly trained model directly to production even though its evaluated recall has fallen below the business threshold. The team must meet its documented availability and rollback constraints. Which corrective action most directly addresses this ML workflow problem?
- Provide an approved S3 gateway or interface endpoint path and least-privilege bucket/role permissions without unnecessarily opening internet egress.
- Use a SageMaker Batch Transform or batch inference job for the scheduled bulk scoring workflow.
- Grant the build role only required object and key operations for its specific artifact prefix and encryption key.
- Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds.
Explanation
Correct Answer
D:Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds. This is the appropriate controlled remedy for codepipeline approval gate on model metrics. A CI/CD stage that records evaluation without enforcing its result is not a release control. Verify the cause with: Inspect pipeline stages, evaluation artifacts, test result propagation, registry approval state, and deployment dependencies. Then confirm: A deliberately poor candidate halts before production while a passing approved candidate can proceed. Reported signal: A pipeline pushes a newly trained model directly to production even though its evaluated recall has fallen below the business threshold. Operational setting: During an approved rollout of an AWS ML inference workload.
Incorrect Answers
A:Provide an approved S3 gateway or interface endpoint path and least-privilege bucket/role permissions without unnecessarily opening internet egress. This addresses a different issue involving sagemaker vpc endpoint access for private artifacts. It does not correct the stated codepipeline approval gate on model metrics mechanism. The targeted response is: Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds. Case evidence: A pipeline pushes a newly trained model directly to production even though its evaluated recall has fallen below the business threshold. Operational setting: During an approved rollout of an AWS ML inference workload.
B:Use a SageMaker Batch Transform or batch inference job for the scheduled bulk scoring workflow. This addresses a different issue involving real-time versus batch inference decision. It does not correct the stated codepipeline approval gate on model metrics mechanism. The targeted response is: Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds. Case evidence: A pipeline pushes a newly trained model directly to production even though its evaluated recall has fallen below the business threshold. Operational setting: During an approved rollout of an AWS ML inference workload.
C:Grant the build role only required object and key operations for its specific artifact prefix and encryption key. This addresses a different issue involving codebuild iam least-privilege artifact upload. It does not correct the stated codepipeline approval gate on model metrics mechanism. The targeted response is: Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds. Case evidence: A pipeline pushes a newly trained model directly to production even though its evaluated recall has fallen below the business threshold. Operational setting: During an approved rollout of an AWS ML inference workload.
Question 286
At the model-governance gate before promotion of a new ML artifact, the following condition is observed: Two notebooks deploy different model artifacts under the same informal name, so the team cannot reproduce which code and data generated production predictions. The team needs a defensible technical decision rather than an untracked workaround. Which TWO complementary choices provide the decisive diagnosis AND corrective response? Select TWO.
- Evaluate Amazon Textract form and table extraction first, adding custom processing only for unsupported fields.
- Register immutable model package versions with lineage and require explicit approval before promotion to production.
- Inspect forecast horizon, frequency, calendar patterns, historical covariates, and temporal train/test partitions.
- Prefer data parallel training with synchronized gradients when the model fits per device and the bottleneck is dataset throughput.
- Inspect model package version, artifact URI, training metadata, data snapshot, and registry approval state.
Explanation
Correct Answers
B:Register immutable model package versions with lineage and require explicit approval before promotion to production. This is the appropriate controlled remedy for sagemaker model registry candidate approval. A human-readable model nickname is not a reproducible deployment identifier; versioned registry artifacts support controlled promotion. Verify the cause with: Inspect model package version, artifact URI, training metadata, data snapshot, and registry approval state. Then confirm: A production endpoint resolves to one approved model package version with traceable training lineage. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: Two notebooks deploy different model artifacts under the same informal name, so the team cannot reproduce which code and data generated production predictions. Operational setting: At the model-governance gate before promotion of a new ML artifact.
E:Inspect model package version, artifact URI, training metadata, data snapshot, and registry approval state. This directly tests sagemaker model registry candidate approval in the reported AWS ML workflow. A human-readable model nickname is not a reproducible deployment identifier; versioned registry artifacts support controlled promotion. It distinguishes the problem before choosing a change. The related targeted response is: Register immutable model package versions with lineage and require explicit approval before promotion to production. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: Two notebooks deploy different model artifacts under the same informal name, so the team cannot reproduce which code and data generated production predictions. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Incorrect Answers
A:Evaluate Amazon Textract form and table extraction first, adding custom processing only for unsupported fields. This selection concerns managed ai service versus custom ocr model rather than the required diagnostic-and-remedy pair for sagemaker model registry candidate approval. Use evidence: Inspect model package version, artifact URI, training metadata, data snapshot, and registry approval state. and the matching correction: Register immutable model package versions with lineage and require explicit approval before promotion to production. Signal: Two notebooks deploy different model artifacts under the same informal name, so the team cannot reproduce which code and data generated production predictions. Operational setting: At the model-governance gate before promotion of a new ML artifact.
C:Inspect forecast horizon, frequency, calendar patterns, historical covariates, and temporal train/test partitions. This selection concerns time-series forecast horizon and seasonality rather than the required diagnostic-and-remedy pair for sagemaker model registry candidate approval. Use evidence: Inspect model package version, artifact URI, training metadata, data snapshot, and registry approval state. and the matching correction: Register immutable model package versions with lineage and require explicit approval before promotion to production. Signal: Two notebooks deploy different model artifacts under the same informal name, so the team cannot reproduce which code and data generated production predictions. Operational setting: At the model-governance gate before promotion of a new ML artifact.
D:Prefer data parallel training with synchronized gradients when the model fits per device and the bottleneck is dataset throughput. This selection concerns data parallelism versus model parallelism rather than the required diagnostic-and-remedy pair for sagemaker model registry candidate approval. Use evidence: Inspect model package version, artifact URI, training metadata, data snapshot, and registry approval state. and the matching correction: Register immutable model package versions with lineage and require explicit approval before promotion to production. Signal: Two notebooks deploy different model artifacts under the same informal name, so the team cannot reproduce which code and data generated production predictions. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Question 287
While reproducing a training incident in an independent SageMaker experiment, the following condition is observed: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. The experiment must be compared fairly against its prior baseline. Which observation most directly identifies the root cause of the reported behavior?
- Inspect feature importance stability, variance across folds, regularization and model dimensionality.
- Inspect target types, business loss function, sample labels, and whether the required output is continuous or categorical.
- Inspect Clarify attribution results, feature timestamp semantics, and whether the flag is created after the target event.
- Measure class prevalence, precision-recall curve, ROC curve, and alert volume at the intended operating threshold.
Explanation
Correct Answer
D:Measure class prevalence, precision-recall curve, ROC curve, and alert volume at the intended operating threshold. This directly tests roc-auc versus pr-auc for rare events in the reported AWS ML workflow. With rare positive classes, precision and operational false-alert counts can be more revealing than ROC-AUC alone. It distinguishes the problem before choosing a change. The related targeted response is: Evaluate PR-AUC and precision at required recall alongside ROC-AUC, and select a deployment threshold for actual prevalence. Reported signal: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Incorrect Answers
A:Inspect feature importance stability, variance across folds, regularization and model dimensionality. This is a possible diagnostic for feature selection to reduce overparameterization, but it does not directly test the roc-auc versus pr-auc for rare events failure reported here. Prioritize: Measure class prevalence, precision-recall curve, ROC curve, and alert volume at the intended operating threshold. The relevant case is: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
B:Inspect target types, business loss function, sample labels, and whether the required output is continuous or categorical. This is a possible diagnostic for regression versus classification problem formulation, but it does not directly test the roc-auc versus pr-auc for rare events failure reported here. Prioritize: Measure class prevalence, precision-recall curve, ROC curve, and alert volume at the intended operating threshold. The relevant case is: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
C:Inspect Clarify attribution results, feature timestamp semantics, and whether the flag is created after the target event. This is a possible diagnostic for clarify feature attribution against target leakage, but it does not directly test the roc-auc versus pr-auc for rare events failure reported here. Prioritize: Measure class prevalence, precision-recall curve, ROC curve, and alert volume at the intended operating threshold. The relevant case is: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Question 288
While reproducing a training incident in an independent SageMaker experiment, the following condition is observed: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. The experiment must be compared fairly against its prior baseline. Which engineering response resolves the actual issue without masking the symptoms?
- Use tracked SageMaker Experiments runs with fixed comparable evaluation splits and a documented metric contract.
- Configure a supported SageMaker checkpoint location and resume training state on managed Spot interruption recovery.
- Register immutable model package versions with lineage and require explicit approval before promotion to production.
- Evaluate PR-AUC and precision at required recall alongside ROC-AUC, and select a deployment threshold for actual prevalence.
Explanation
Correct Answer
D:Evaluate PR-AUC and precision at required recall alongside ROC-AUC, and select a deployment threshold for actual prevalence. This is the appropriate controlled remedy for roc-auc versus pr-auc for rare events. With rare positive classes, precision and operational false-alert counts can be more revealing than ROC-AUC alone. Verify the cause with: Measure class prevalence, precision-recall curve, ROC curve, and alert volume at the intended operating threshold. Then confirm: The tested alert rate and precision match the production workload rather than an optimistic single ROC figure. Reported signal: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Incorrect Answers
A:Use tracked SageMaker Experiments runs with fixed comparable evaluation splits and a documented metric contract. This addresses a different issue involving reproducible sagemaker experiments comparisons. It does not correct the stated roc-auc versus pr-auc for rare events mechanism. The targeted response is: Evaluate PR-AUC and precision at required recall alongside ROC-AUC, and select a deployment threshold for actual prevalence. Case evidence: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
B:Configure a supported SageMaker checkpoint location and resume training state on managed Spot interruption recovery. This addresses a different issue involving managed spot training checkpoint recovery. It does not correct the stated roc-auc versus pr-auc for rare events mechanism. The targeted response is: Evaluate PR-AUC and precision at required recall alongside ROC-AUC, and select a deployment threshold for actual prevalence. Case evidence: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
C:Register immutable model package versions with lineage and require explicit approval before promotion to production. This addresses a different issue involving sagemaker model registry candidate approval. It does not correct the stated roc-auc versus pr-auc for rare events mechanism. The targeted response is: Evaluate PR-AUC and precision at required recall alongside ROC-AUC, and select a deployment threshold for actual prevalence. Case evidence: A rare equipment-failure classifier reports a high ROC-AUC but its operational queue contains mostly false alarms. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Question 289
At the model-governance gate before promotion of a new ML artifact, the following condition is observed: A fraud-screening model has high global accuracy because most purchases are legitimate, while expensive fraud cases are often missed. The team needs a defensible technical decision rather than an untracked workaround. Which observation most directly identifies the root cause of the reported behavior?
- Inspect experiment run lineage, dataset/version hashes, code revisions, random seeds, and metric definitions.
- Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints.
- Inspect interruption events, checkpoint directory mapping, restoration logic, and persistent S3 checkpoint artifacts.
- Inspect confusion matrix, class prevalence, false-negative cost, and precision-recall curves across thresholds.
Explanation
Correct Answer
D:Inspect confusion matrix, class prevalence, false-negative cost, and precision-recall curves across thresholds. This directly tests precision versus recall fraud threshold in the reported AWS ML workflow. Accuracy can conceal poor minority-class detection; threshold selection trades recall against precision. It distinguishes the problem before choosing a change. The related targeted response is: Select a decision threshold using recall or cost-aware precision/recall targets and measure on the untouched holdout. Reported signal: A fraud-screening model has high global accuracy because most purchases are legitimate, while expensive fraud cases are often missed. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Incorrect Answers
A:Inspect experiment run lineage, dataset/version hashes, code revisions, random seeds, and metric definitions. This is a possible diagnostic for reproducible sagemaker experiments comparisons, but it does not directly test the precision versus recall fraud threshold failure reported here. Prioritize: Inspect confusion matrix, class prevalence, false-negative cost, and precision-recall curves across thresholds. The relevant case is: A fraud-screening model has high global accuracy because most purchases are legitimate, while expensive fraud cases are often missed. Operational setting: At the model-governance gate before promotion of a new ML artifact.
B:Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. This is a possible diagnostic for hyperparameter bayesian tuning strategy, but it does not directly test the precision versus recall fraud threshold failure reported here. Prioritize: Inspect confusion matrix, class prevalence, false-negative cost, and precision-recall curves across thresholds. The relevant case is: A fraud-screening model has high global accuracy because most purchases are legitimate, while expensive fraud cases are often missed. Operational setting: At the model-governance gate before promotion of a new ML artifact.
C:Inspect interruption events, checkpoint directory mapping, restoration logic, and persistent S3 checkpoint artifacts. This is a possible diagnostic for managed spot training checkpoint recovery, but it does not directly test the precision versus recall fraud threshold failure reported here. Prioritize: Inspect confusion matrix, class prevalence, false-negative cost, and precision-recall curves across thresholds. The relevant case is: A fraud-screening model has high global accuracy because most purchases are legitimate, while expensive fraud cases are often missed. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Question 290
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A demand model treats hour 23 and hour 0 as far apart when integer hour is used as a linear continuous feature. The dataset contract must remain reproducible for a backfill. Which corrective action most directly addresses this ML workflow problem?
- Encode hour with sine and cosine cyclic features and validate with a time-aware split.
- Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving.
- Use rolling-origin or chronological validation with strictly past-derived features and appropriate forecast horizons.
- Compact the partitioned dataset into appropriately sized Parquet objects without changing the record keys or schema.
Explanation
Correct Answer
A:Encode hour with sine and cosine cyclic features and validate with a time-aware split. This is the appropriate controlled remedy for time-series cyclical calendar feature. Time-of-day has periodic topology; raw integer distance does not represent wraparound. Verify the cause with: Inspect prediction errors around midnight and feature representations for the periodic hour variable. Then confirm: Neighboring times across midnight map close in cyclic feature space and validation improves without leakage. Reported signal: A demand model treats hour 23 and hour 0 as far apart when integer hour is used as a linear continuous feature. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
B:Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. This addresses a different issue involving skewed numerical feature log transformation. It does not correct the stated time-series cyclical calendar feature mechanism. The targeted response is: Encode hour with sine and cosine cyclic features and validate with a time-aware split. Case evidence: A demand model treats hour 23 and hour 0 as far apart when integer hour is used as a linear continuous feature. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Use rolling-origin or chronological validation with strictly past-derived features and appropriate forecast horizons. This addresses a different issue involving chronological split for forecast evaluation. It does not correct the stated time-series cyclical calendar feature mechanism. The targeted response is: Encode hour with sine and cosine cyclic features and validate with a time-aware split. Case evidence: A demand model treats hour 23 and hour 0 as far apart when integer hour is used as a linear continuous feature. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
D:Compact the partitioned dataset into appropriately sized Parquet objects without changing the record keys or schema. This addresses a different issue involving columnar file compaction for small objects. It does not correct the stated time-series cyclical calendar feature mechanism. The targeted response is: Encode hour with sine and cosine cyclic features and validate with a time-aware split. Case evidence: A demand model treats hour 23 and hour 0 as far apart when integer hour is used as a linear continuous feature. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.