Amazon AWS Certified Machine Learning Engineer - Associate MLA-C01 Practice Test 118
Question 1170
During a production MLOps reliability and security review, the following condition is observed: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. The owner needs an auditable resolution without broadening privileges. Which engineering response resolves the actual issue without masking the symptoms?
- Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it.
- Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family.
- Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes.
- Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks.
Explanation
Correct Answer
D:Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. This is the appropriate controlled remedy for sagemaker model monitor feature drift baseline. Endpoint uptime does not demonstrate feature-distribution stability; monitoring compares current inputs with expected statistics. Verify the cause with: Compare incoming captured feature statistics against a stored training baseline and review SageMaker Model Monitor drift alerts. Then confirm: Changed feature distributions produce attributable violations even while endpoint availability remains healthy. Reported signal: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: During a production MLOps reliability and security review.
Incorrect Answers
A:Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. This addresses a different issue involving spot training cost versus interruption recovery. It does not correct the stated sagemaker model monitor feature drift baseline mechanism. The targeted response is: Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. Case evidence: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: During a production MLOps reliability and security review.
B:Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family. This addresses a different issue involving inference recommender benchmarking instance family. It does not correct the stated sagemaker model monitor feature drift baseline mechanism. The targeted response is: Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. Case evidence: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: During a production MLOps reliability and security review.
C:Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes. This addresses a different issue involving clarify postdeployment group bias monitoring. It does not correct the stated sagemaker model monitor feature drift baseline mechanism. The targeted response is: Configure data capture and an appropriate Model Monitor baseline with scheduled data-quality checks. Case evidence: A deployed credit-risk model begins receiving applicants with income distributions unlike training data, but dashboard operators only track endpoint HTTP 200 rates. Operational setting: During a production MLOps reliability and security review.
Question 1171
While reproducing the alert in an isolated AWS ML integration account, the following condition is observed: A real-time recommender fetches cached customer history updated hourly while offline training used a daily snapshot with different aggregation boundaries. The proposed change must preserve logging and least privilege. Which observation most directly identifies the root cause of the reported behavior?
- Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution.
- Inspect routing hashes, experiment assignments, variant weights, and request identity consistency.
- Inspect async inference backlog size, age of oldest request, worker capacity and completion/error metrics.
- Inspect online versus offline feature computation code, event-time cutoffs, freshness, and sampled entity values.
Explanation
Correct Answer
D:Inspect online versus offline feature computation code, event-time cutoffs, freshness, and sampled entity values. This directly tests training-serving feature skew incident in the reported AWS ML workflow. Mismatched feature windows and update cadence can degrade production predictions even if training accuracy is stable. It distinguishes the problem before choosing a change. The related targeted response is: Use a consistent versioned feature definition and test offline/online values at the same observation cutoff. Reported signal: A real-time recommender fetches cached customer history updated hourly while offline training used a daily snapshot with different aggregation boundaries. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Incorrect Answers
A:Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution. This is a possible diagnostic for cloudwatch 5xxerrors endpoint saturation, but it does not directly test the training-serving feature skew incident failure reported here. Prioritize: Inspect online versus offline feature computation code, event-time cutoffs, freshness, and sampled entity values. The relevant case is: A real-time recommender fetches cached customer history updated hourly while offline training used a daily snapshot with different aggregation boundaries. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
B:Inspect routing hashes, experiment assignments, variant weights, and request identity consistency. This is a possible diagnostic for champion challenger a/b assignment consistency, but it does not directly test the training-serving feature skew incident failure reported here. Prioritize: Inspect online versus offline feature computation code, event-time cutoffs, freshness, and sampled entity values. The relevant case is: A real-time recommender fetches cached customer history updated hourly while offline training used a daily snapshot with different aggregation boundaries. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
C:Inspect async inference backlog size, age of oldest request, worker capacity and completion/error metrics. This is a possible diagnostic for asynchronous inference queue backlog alert, but it does not directly test the training-serving feature skew incident failure reported here. Prioritize: Inspect online versus offline feature computation code, event-time cutoffs, freshness, and sampled entity values. The relevant case is: A real-time recommender fetches cached customer history updated hourly while offline training used a daily snapshot with different aggregation boundaries. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Question 1172
At the model-governance gate before promotion of a new ML artifact, the following condition is observed: A supply-chain demand forecast ignores weekly seasonality and uses an ordinary row-shuffled classifier for a 14-day continuous forecast. The team needs a defensible technical decision rather than an untracked workaround. What targeted change should be implemented after verifying the stated cause?
- Use an appropriate text classification model or supported SageMaker BlazingText supervised approach with validated labels.
- Use justified feature selection or sparsity constraints fitted on training folds before retraining.
- Use a time-series forecasting approach with past context and seasonal features, evaluating all intended future horizons.
- Read the configured SageMaker training channel path or environment variable and fail clearly on missing input.
Explanation
Correct Answer
C:Use a time-series forecasting approach with past context and seasonal features, evaluating all intended future horizons. This is the appropriate controlled remedy for time-series forecast horizon and seasonality. Forecasting requires horizon-aware validation and temporal structure, not a shuffled categorical target. Verify the cause with: Inspect forecast horizon, frequency, calendar patterns, historical covariates, and temporal train/test partitions. Then confirm: A rolling-origin backtest improves multi-step forecast error and respects temporal order. Reported signal: A supply-chain demand forecast ignores weekly seasonality and uses an ordinary row-shuffled classifier for a 14-day continuous forecast. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Incorrect Answers
A:Use an appropriate text classification model or supported SageMaker BlazingText supervised approach with validated labels. This addresses a different issue involving sagemaker built-in blazingtext choice. It does not correct the stated time-series forecast horizon and seasonality mechanism. The targeted response is: Use a time-series forecasting approach with past context and seasonal features, evaluating all intended future horizons. Case evidence: A supply-chain demand forecast ignores weekly seasonality and uses an ordinary row-shuffled classifier for a 14-day continuous forecast. Operational setting: At the model-governance gate before promotion of a new ML artifact.
B:Use justified feature selection or sparsity constraints fitted on training folds before retraining. This addresses a different issue involving feature selection to reduce overparameterization. It does not correct the stated time-series forecast horizon and seasonality mechanism. The targeted response is: Use a time-series forecasting approach with past context and seasonal features, evaluating all intended future horizons. Case evidence: A supply-chain demand forecast ignores weekly seasonality and uses an ordinary row-shuffled classifier for a 14-day continuous forecast. Operational setting: At the model-governance gate before promotion of a new ML artifact.
D:Read the configured SageMaker training channel path or environment variable and fail clearly on missing input. This addresses a different issue involving sagemaker script-mode data channel mismatch. It does not correct the stated time-series forecast horizon and seasonality mechanism. The targeted response is: Use a time-series forecasting approach with past context and seasonal features, evaluating all intended future horizons. Case evidence: A supply-chain demand forecast ignores weekly seasonality and uses an ordinary row-shuffled classifier for a 14-day continuous forecast. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Question 1173
During production evaluation of a SageMaker AI model candidate, the following condition is observed: A credit-risk pilot selects a high-performing opaque model even though reviewers must justify material prediction features to an auditor. The release is blocked until the underlying model behavior is understood. Which engineering response resolves the actual issue without masking the symptoms?
- Read the configured SageMaker training channel path or environment variable and fail clearly on missing input.
- Compare a more interpretable model or add tested SageMaker Clarify explainability to the decision process.
- Use an unsupervised clustering approach such as k-means after appropriate feature preparation and assess whether groups are meaningful.
- Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses.
Explanation
Correct Answer
B:Compare a more interpretable model or add tested SageMaker Clarify explainability to the decision process. This is the appropriate controlled remedy for interpretability for high-stakes prediction. Feature attribution can support explanation, but model choice must reflect transparency requirements and attribution is not causality. Verify the cause with: Inspect documented explanation requirements, candidate feature contributions, and available post-hoc interpretation tests. Then confirm: Reviewers can identify stable feature drivers for representative decisions without misleading causal claims. Reported signal: A credit-risk pilot selects a high-performing opaque model even though reviewers must justify material prediction features to an auditor. Operational setting: During production evaluation of a SageMaker AI model candidate.
Incorrect Answers
A:Read the configured SageMaker training channel path or environment variable and fail clearly on missing input. This addresses a different issue involving sagemaker script-mode data channel mismatch. It does not correct the stated interpretability for high-stakes prediction mechanism. The targeted response is: Compare a more interpretable model or add tested SageMaker Clarify explainability to the decision process. Case evidence: A credit-risk pilot selects a high-performing opaque model even though reviewers must justify material prediction features to an auditor. Operational setting: During production evaluation of a SageMaker AI model candidate.
C:Use an unsupervised clustering approach such as k-means after appropriate feature preparation and assess whether groups are meaningful. This addresses a different issue involving clustering for unlabeled customer segments. It does not correct the stated interpretability for high-stakes prediction mechanism. The targeted response is: Compare a more interpretable model or add tested SageMaker Clarify explainability to the decision process. Case evidence: A credit-risk pilot selects a high-performing opaque model even though reviewers must justify material prediction features to an auditor. Operational setting: During production evaluation of a SageMaker AI model candidate.
D:Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses. This addresses a different issue involving shadow deployment model evaluation without response change. It does not correct the stated interpretability for high-stakes prediction mechanism. The targeted response is: Compare a more interpretable model or add tested SageMaker Clarify explainability to the decision process. Case evidence: A credit-risk pilot selects a high-performing opaque model even though reviewers must justify material prediction features to an auditor. Operational setting: During production evaluation of a SageMaker AI model candidate.
Question 1174
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Change control requires measurable endpoint or pipeline acceptance evidence. What targeted change should be implemented after verifying the stated cause?
- Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete.
- Define the endpoint resources and parameters in versioned CloudFormation or CDK code, review a change set, and reconcile drift.
- Compile the supported model for the specific edge target using SageMaker Neo or choose a compatible conversion route.
- Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds.
Explanation
Correct Answer
A:Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete. This is the appropriate controlled remedy for eventbridge conditional retraining trigger. Event-driven automation needs selective event patterns and readiness checks; every object-write event is not a retraining signal. Verify the cause with: Inspect EventBridge rule patterns, source events, object prefixes, and pipeline execution histories. Then confirm: Only the intended completed data batch launches one controlled training workflow, while unrelated objects do not. Reported signal: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
B:Define the endpoint resources and parameters in versioned CloudFormation or CDK code, review a change set, and reconcile drift. This addresses a different issue involving cloudformation stack parameter drift. It does not correct the stated eventbridge conditional retraining trigger mechanism. The targeted response is: Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete. Case evidence: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
C:Compile the supported model for the specific edge target using SageMaker Neo or choose a compatible conversion route. This addresses a different issue involving sagemaker neo edge compiler compatibility. It does not correct the stated eventbridge conditional retraining trigger mechanism. The targeted response is: Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete. Case evidence: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
D:Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds. This addresses a different issue involving codepipeline approval gate on model metrics. It does not correct the stated eventbridge conditional retraining trigger mechanism. The targeted response is: Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete. Case evidence: An ML system retrains on every S3 object event even when the object is a debug file, wasting resources and changing model versions unexpectedly. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Question 1175
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A raw CSV field contains quoted commas and a header is sometimes missing, so naïve delimiter splitting shifts the label into the wrong column. The dataset contract must remain reproducible for a backfill. Which investigation provides the most decisive FIRST evidence for the underlying problem?
- Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions.
- Inspect sample raw records, quoting rules, expected headers, and parsed column counts in the ingestion job.
- Compare the query predicates with S3 object prefixes, partition metadata, and query plans to confirm partitions are not being pruned.
- Compare online Feature Store record values and offline event-time keyed feature history for the same entity and model snapshot.
Explanation
Correct Answer
B:Inspect sample raw records, quoting rules, expected headers, and parsed column counts in the ingestion job. This directly tests csv schema ambiguity during ingestion in the reported AWS ML workflow. CSV parsing must respect quoting and schema; splitting on every comma corrupts valid quoted fields. It distinguishes the problem before choosing a change. The related targeted response is: Use a CSV parser with explicit quoting and schema validation; quarantine malformed records before training. Reported signal: A raw CSV field contains quoted commas and a header is sometimes missing, so naïve delimiter splitting shifts the label into the wrong column. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
A:Inspect source fields, sensitivity classification, Ground Truth task input, and downstream annotation permissions. This is a possible diagnostic for pii masking before model labeling, but it does not directly test the csv schema ambiguity during ingestion failure reported here. Prioritize: Inspect sample raw records, quoting rules, expected headers, and parsed column counts in the ingestion job. The relevant case is: A raw CSV field contains quoted commas and a header is sometimes missing, so naïve delimiter splitting shifts the label into the wrong column. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Compare the query predicates with S3 object prefixes, partition metadata, and query plans to confirm partitions are not being pruned. This is a possible diagnostic for partition pruning in s3 parquet training input, but it does not directly test the csv schema ambiguity during ingestion failure reported here. Prioritize: Inspect sample raw records, quoting rules, expected headers, and parsed column counts in the ingestion job. The relevant case is: A raw CSV field contains quoted commas and a header is sometimes missing, so naïve delimiter splitting shifts the label into the wrong column. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
D:Compare online Feature Store record values and offline event-time keyed feature history for the same entity and model snapshot. This is a possible diagnostic for sagemaker feature store online offline consistency, but it does not directly test the csv schema ambiguity during ingestion failure reported here. Prioritize: Inspect sample raw records, quoting rules, expected headers, and parsed column counts in the ingestion job. The relevant case is: A raw CSV field contains quoted commas and a header is sometimes missing, so naïve delimiter splitting shifts the label into the wrong column. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 1176
As a regulated ML workload prepares for an automated regional rollout, the following condition is observed: A recommendation endpoint drops peak requests with HTTP 5xx as invocations per instance rise, while the dashboard averages latency across an entire day. The team must demonstrate a repeatable monitoring or security control. What should the ML engineer inspect FIRST to distinguish this failure from unrelated issues?
- Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics.
- Measure memory, throughput, p95 latency and cost across supported candidate endpoint configurations.
- Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution.
- Compare probability calibration plots, Brier score and event rates across recent labeled cohorts.
Explanation
Correct Answer
C:Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution. This directly tests cloudwatch 5xxerrors endpoint saturation in the reported AWS ML workflow. Daily averages can conceal short capacity failures; monitoring must use an incident-relevant interval. It distinguishes the problem before choosing a change. The related targeted response is: Configure actionable alarms, capacity, and supported scaling or backpressure based on the burst window. Reported signal: A recommendation endpoint drops peak requests with HTTP 5xx as invocations per instance rise, while the dashboard averages latency across an entire day. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Incorrect Answers
A:Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. This is a possible diagnostic for target-label delay in production quality monitoring, but it does not directly test the cloudwatch 5xxerrors endpoint saturation failure reported here. Prioritize: Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution. The relevant case is: A recommendation endpoint drops peak requests with HTTP 5xx as invocations per instance rise, while the dashboard averages latency across an entire day. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
B:Measure memory, throughput, p95 latency and cost across supported candidate endpoint configurations. This is a possible diagnostic for inference recommender benchmarking instance family, but it does not directly test the cloudwatch 5xxerrors endpoint saturation failure reported here. Prioritize: Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution. The relevant case is: A recommendation endpoint drops peak requests with HTTP 5xx as invocations per instance rise, while the dashboard averages latency across an entire day. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
D:Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. This is a possible diagnostic for serving output confidence calibration drift, but it does not directly test the cloudwatch 5xxerrors endpoint saturation failure reported here. Prioritize: Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution. The relevant case is: A recommendation endpoint drops peak requests with HTTP 5xx as invocations per instance rise, while the dashboard averages latency across an entire day. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Question 1177
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: Three SageMaker Ground Truth annotators assign conflicting labels to borderline product photos, but a training job assumes all majority votes are gold. The dataset contract must remain reproducible for a backfill. Which TWO steps best establish the evidence and correct the identified root cause? Select TWO.
- Inspect AWS Glue Data Quality rule evaluations, missing-key rates, source partitions, and prior accepted baselines.
- Use rolling-origin or chronological validation with strictly past-derived features and appropriate forecast horizons.
- Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set.
- Package the training tokenizer and normalization configuration with the model artifact and require serving compatibility checks.
- Revise instructions, route ambiguous samples for expert adjudication, and monitor label quality per class.
Explanation
Correct Answers
C:Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set. This directly tests ground truth labeling agreement review in the reported AWS ML workflow. Majority vote does not guarantee semantic correctness when instructions are ambiguous or a class is rare. It distinguishes the problem before choosing a change. The related targeted response is: Revise instructions, route ambiguous samples for expert adjudication, and monitor label quality per class. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: Three SageMaker Ground Truth annotators assign conflicting labels to borderline product photos, but a training job assumes all majority votes are gold. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
E:Revise instructions, route ambiguous samples for expert adjudication, and monitor label quality per class. This is the appropriate controlled remedy for ground truth labeling agreement review. Majority vote does not guarantee semantic correctness when instructions are ambiguous or a class is rare. Verify the cause with: Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set. Then confirm: Agreement on a held-out gold set rises without silently discarding rare legitimate classes. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: Three SageMaker Ground Truth annotators assign conflicting labels to borderline product photos, but a training job assumes all majority votes are gold. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
A:Inspect AWS Glue Data Quality rule evaluations, missing-key rates, source partitions, and prior accepted baselines. This selection concerns glue data quality non-null contract rather than the required diagnostic-and-remedy pair for ground truth labeling agreement review. Use evidence: Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set. and the matching correction: Revise instructions, route ambiguous samples for expert adjudication, and monitor label quality per class. Signal: Three SageMaker Ground Truth annotators assign conflicting labels to borderline product photos, but a training job assumes all majority votes are gold. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
B:Use rolling-origin or chronological validation with strictly past-derived features and appropriate forecast horizons. This selection concerns chronological split for forecast evaluation rather than the required diagnostic-and-remedy pair for ground truth labeling agreement review. Use evidence: Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set. and the matching correction: Revise instructions, route ambiguous samples for expert adjudication, and monitor label quality per class. Signal: Three SageMaker Ground Truth annotators assign conflicting labels to borderline product photos, but a training job assumes all majority votes are gold. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
D:Package the training tokenizer and normalization configuration with the model artifact and require serving compatibility checks. This selection concerns text tokenization mismatch in serving rather than the required diagnostic-and-remedy pair for ground truth labeling agreement review. Use evidence: Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set. and the matching correction: Revise instructions, route ambiguous samples for expert adjudication, and monitor label quality per class. Signal: Three SageMaker Ground Truth annotators assign conflicting labels to borderline product photos, but a training job assumes all majority votes are gold. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 1178
At a production AWS ML data lake supporting customer-facing predictions, the following condition is observed: A hiring-model dataset has very different positive-label rates across sensitive groups, but the team compares only global accuracy. Engineers must avoid changing unrelated source partitions. Which corrective action most directly addresses this ML workflow problem?
- Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving.
- Split by stable patient identity before training and reserve a disjoint held-out group for evaluation.
- Measure label distribution differences with an appropriate pretraining bias metric and improve data representation before training.
- Package the training tokenizer and normalization configuration with the model artifact and require serving compatibility checks.
Explanation
Correct Answer
C:Measure label distribution differences with an appropriate pretraining bias metric and improve data representation before training. This is the appropriate controlled remedy for clarify label imbalance measurement. Aggregate accuracy can conceal unequal source-label distributions, which pretraining metrics assess before model fitting. Verify the cause with: Inspect group sample sizes, label-positive fractions, and the appropriate SageMaker Clarify pretraining bias report. Then confirm: The report quantifies class or group imbalance with reproducible group counts and documents remediation assumptions. Reported signal: A hiring-model dataset has very different positive-label rates across sensitive groups, but the team compares only global accuracy. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Incorrect Answers
A:Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. This addresses a different issue involving skewed numerical feature log transformation. It does not correct the stated clarify label imbalance measurement mechanism. The targeted response is: Measure label distribution differences with an appropriate pretraining bias metric and improve data representation before training. Case evidence: A hiring-model dataset has very different positive-label rates across sensitive groups, but the team compares only global accuracy. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
B:Split by stable patient identity before training and reserve a disjoint held-out group for evaluation. This addresses a different issue involving group-leakage train and validation split. It does not correct the stated clarify label imbalance measurement mechanism. The targeted response is: Measure label distribution differences with an appropriate pretraining bias metric and improve data representation before training. Case evidence: A hiring-model dataset has very different positive-label rates across sensitive groups, but the team compares only global accuracy. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
D:Package the training tokenizer and normalization configuration with the model artifact and require serving compatibility checks. This addresses a different issue involving text tokenization mismatch in serving. It does not correct the stated clarify label imbalance measurement mechanism. The targeted response is: Measure label distribution differences with an appropriate pretraining bias metric and improve data representation before training. Case evidence: A hiring-model dataset has very different positive-label rates across sensitive groups, but the team compares only global accuracy. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Question 1179
At the model-governance gate before promotion of a new ML artifact, the following condition is observed: A model reaches near-perfect training accuracy but substantially worse holdout performance as feature coefficients grow unusually large. The team needs a defensible technical decision rather than an untracked workaround. What postchange observation provides the strongest acceptance evidence?
- A rolling-origin backtest improves multi-step forecast error and respects temporal order.
- Held-out messages receive the intended class predictions and macro-averaged quality is measurable.
- The train-validation gap narrows without losing acceptable validation performance.
- The fine-tuned model improves task accuracy without collapsing pretrained representations.
Explanation
Correct Answer
C:The train-validation gap narrows without losing acceptable validation performance. This is the acceptance evidence for the described overfitting reduced with l2 regularization scenario. L2 penalizes large weights and can reduce variance but excessive regularization also causes underfitting. The corrective implementation it validates is: Increase justified L2 regularization or reduce model capacity and revalidate with an untouched holdout. Reported signal: A model reaches near-perfect training accuracy but substantially worse holdout performance as feature coefficients grow unusually large. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Incorrect Answers
A:A rolling-origin backtest improves multi-step forecast error and respects temporal order. This could validate time-series forecast horizon and seasonality, but it does not establish that overfitting reduced with l2 regularization is resolved for this workload. Confirm instead: The train-validation gap narrows without losing acceptable validation performance. Observed problem: A model reaches near-perfect training accuracy but substantially worse holdout performance as feature coefficients grow unusually large. Operational setting: At the model-governance gate before promotion of a new ML artifact.
B:Held-out messages receive the intended class predictions and macro-averaged quality is measurable. This could validate sagemaker built-in blazingtext choice, but it does not establish that overfitting reduced with l2 regularization is resolved for this workload. Confirm instead: The train-validation gap narrows without losing acceptable validation performance. Observed problem: A model reaches near-perfect training accuracy but substantially worse holdout performance as feature coefficients grow unusually large. Operational setting: At the model-governance gate before promotion of a new ML artifact.
D:The fine-tuned model improves task accuracy without collapsing pretrained representations. This could validate fine-tuning pretrained model learning-rate stability, but it does not establish that overfitting reduced with l2 regularization is resolved for this workload. Confirm instead: The train-validation gap narrows without losing acceptable validation performance. Observed problem: A model reaches near-perfect training accuracy but substantially worse holdout performance as feature coefficients grow unusually large. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Question 1180
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: A latency-sensitive transformer endpoint runs on a small CPU instance with severe token-processing latency and no cost comparison. Change control requires measurable endpoint or pipeline acceptance evidence. What should the ML engineer inspect FIRST to distinguish this failure from unrelated issues?
- Profile model operations, memory footprint, CPU saturation, request latency, and GPU throughput under representative load.
- Compare invocation count per instance, model latency, GPU utilization, and the behavior of current scaling policy.
- Inspect CodeBuild execution-role policy, bucket policy, KMS permissions, and denied CloudTrail events.
- Profile request arrival intervals, cold-start tolerance, memory needs, and compatible serverless inference limits.
Explanation
Correct Answer
A:Profile model operations, memory footprint, CPU saturation, request latency, and GPU throughput under representative load. This directly tests gpu versus cpu inference fit in the reported AWS ML workflow. Accelerators can improve matrix-heavy inference, but service choice should be measured against workload size and economics. It distinguishes the problem before choosing a change. The related targeted response is: Benchmark supported accelerated inference instances and choose GPU capacity only if latency and throughput gains justify cost. Reported signal: A latency-sensitive transformer endpoint runs on a small CPU instance with severe token-processing latency and no cost comparison. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
B:Compare invocation count per instance, model latency, GPU utilization, and the behavior of current scaling policy. This is a possible diagnostic for sagemaker endpoint auto scaling metric selection, but it does not directly test the gpu versus cpu inference fit failure reported here. Prioritize: Profile model operations, memory footprint, CPU saturation, request latency, and GPU throughput under representative load. The relevant case is: A latency-sensitive transformer endpoint runs on a small CPU instance with severe token-processing latency and no cost comparison. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
C:Inspect CodeBuild execution-role policy, bucket policy, KMS permissions, and denied CloudTrail events. This is a possible diagnostic for codebuild iam least-privilege artifact upload, but it does not directly test the gpu versus cpu inference fit failure reported here. Prioritize: Profile model operations, memory footprint, CPU saturation, request latency, and GPU throughput under representative load. The relevant case is: A latency-sensitive transformer endpoint runs on a small CPU instance with severe token-processing latency and no cost comparison. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
D:Profile request arrival intervals, cold-start tolerance, memory needs, and compatible serverless inference limits. This is a possible diagnostic for serverless inference for intermittent requests, but it does not directly test the gpu versus cpu inference fit failure reported here. Prioritize: Profile model operations, memory footprint, CPU saturation, request latency, and GPU throughput under representative load. The relevant case is: A latency-sensitive transformer endpoint runs on a small CPU instance with severe token-processing latency and no cost comparison. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.