Amazon AWS Certified Machine Learning Engineer - Associate MLA-C01 Practice Test 1
Question 1
While testing a second region for recoverable deployment of the same ML service, the following condition is observed: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. The deployment must be reproducible without manual console-only fixes. Which measured outcome would confirm that the correction worked as intended?
- The build uploads signed artifacts successfully while unauthorized buckets and prefixes remain denied.
- The compiled artifact runs on the target device with validated outputs and acceptable memory and latency.
- Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs.
- Identical deployment revisions resolve to the same container image digest and environment dependencies.
Explanation
Correct Answer
C:Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. This is the acceptance evidence for the described sagemaker pipelines parameterized retraining scenario. Pipelines make dependencies explicit; manually ordered notebook operations are difficult to repeat or audit. The corrective implementation it validates is: Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts. Reported signal: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: While testing a second region for recoverable deployment of the same ML service.
Incorrect Answers
A:The build uploads signed artifacts successfully while unauthorized buckets and prefixes remain denied. This could validate codebuild iam least-privilege artifact upload, but it does not establish that sagemaker pipelines parameterized retraining is resolved for this workload. Confirm instead: Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. Observed problem: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: While testing a second region for recoverable deployment of the same ML service.
B:The compiled artifact runs on the target device with validated outputs and acceptable memory and latency. This could validate sagemaker neo edge compiler compatibility, but it does not establish that sagemaker pipelines parameterized retraining is resolved for this workload. Confirm instead: Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. Observed problem: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: While testing a second region for recoverable deployment of the same ML service.
D:Identical deployment revisions resolve to the same container image digest and environment dependencies. This could validate ecr immutable image digest for inference, but it does not establish that sagemaker pipelines parameterized retraining is resolved for this workload. Confirm instead: Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. Observed problem: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: While testing a second region for recoverable deployment of the same ML service.
Question 2
As a regulated ML workload prepares for an automated regional rollout, the following condition is observed: A training fleet runs many jobs and operators manually search thousands of unstructured log files to locate recurring data-channel failures. The team must demonstrate a repeatable monitoring or security control. Which TWO complementary choices provide the decisive diagnosis AND corrective response? Select TWO.
- Inspect CloudTrail management events, data event selectors, S3 bucket coverage, and log retention controls.
- Use a consistent versioned feature definition and test offline/online values at the same observation cutoff.
- Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time.
- Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family.
- Inspect consistent log group names, job identifiers, timestamps, and failure codes in CloudWatch Logs.
Explanation
Correct Answers
C:Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time. This is the appropriate controlled remedy for cloudwatch logs insights error correlation. Querying structured centralized logs scales better than visual browsing of unrelated service output. Verify the cause with: Inspect consistent log group names, job identifiers, timestamps, and failure codes in CloudWatch Logs. Then confirm: A query isolates the failing job versions and recurring error class within the incident interval. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A training fleet runs many jobs and operators manually search thousands of unstructured log files to locate recurring data-channel failures. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
E:Inspect consistent log group names, job identifiers, timestamps, and failure codes in CloudWatch Logs. This directly tests cloudwatch logs insights error correlation in the reported AWS ML workflow. Querying structured centralized logs scales better than visual browsing of unrelated service output. It distinguishes the problem before choosing a change. The related targeted response is: Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A training fleet runs many jobs and operators manually search thousands of unstructured log files to locate recurring data-channel failures. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Incorrect Answers
A:Inspect CloudTrail management events, data event selectors, S3 bucket coverage, and log retention controls. This selection concerns cloudtrail training data access audit coverage rather than the required diagnostic-and-remedy pair for cloudwatch logs insights error correlation. Use evidence: Inspect consistent log group names, job identifiers, timestamps, and failure codes in CloudWatch Logs. and the matching correction: Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time. Signal: A training fleet runs many jobs and operators manually search thousands of unstructured log files to locate recurring data-channel failures. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
B:Use a consistent versioned feature definition and test offline/online values at the same observation cutoff. This selection concerns training-serving feature skew incident rather than the required diagnostic-and-remedy pair for cloudwatch logs insights error correlation. Use evidence: Inspect consistent log group names, job identifiers, timestamps, and failure codes in CloudWatch Logs. and the matching correction: Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time. Signal: A training fleet runs many jobs and operators manually search thousands of unstructured log files to locate recurring data-channel failures. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
D:Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family. This selection concerns inference recommender benchmarking instance family rather than the required diagnostic-and-remedy pair for cloudwatch logs insights error correlation. Use evidence: Inspect consistent log group names, job identifiers, timestamps, and failure codes in CloudWatch Logs. and the matching correction: Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time. Signal: A training fleet runs many jobs and operators manually search thousands of unstructured log files to locate recurring data-channel failures. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Question 3
During a production MLOps reliability and security review, the following condition is observed: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. The owner needs an auditable resolution without broadening privileges. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family.
- Inspect resource tags, cost allocation tag activation, Cost Explorer grouping, and unallocated resource types.
- Inspect image layers, environment configuration, secret source, runtime role permissions, and rotation logs.
- Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting.
- Authorize the specific serving role to decrypt with the relevant KMS key under a restricted key policy and S3 access.
Explanation
Correct Answers
B:Inspect resource tags, cost allocation tag activation, Cost Explorer grouping, and unallocated resource types. This directly tests tag-based ml training cost attribution in the reported AWS ML workflow. Cost Explorer grouping requires supported activated resource tags; naming resources does not automatically allocate charges. It distinguishes the problem before choosing a change. The related targeted response is: Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: During a production MLOps reliability and security review.
D:Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. This is the appropriate controlled remedy for tag-based ml training cost attribution. Cost Explorer grouping requires supported activated resource tags; naming resources does not automatically allocate charges. Verify the cause with: Inspect resource tags, cost allocation tag activation, Cost Explorer grouping, and unallocated resource types. Then confirm: Cost reports attribute the majority of in-scope ML spend to the correct team without altering access controls. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: During a production MLOps reliability and security review.
Incorrect Answers
A:Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family. This selection concerns inference recommender benchmarking instance family rather than the required diagnostic-and-remedy pair for tag-based ml training cost attribution. Use evidence: Inspect resource tags, cost allocation tag activation, Cost Explorer grouping, and unallocated resource types. and the matching correction: Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. Signal: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: During a production MLOps reliability and security review.
C:Inspect image layers, environment configuration, secret source, runtime role permissions, and rotation logs. This selection concerns secret rotation for inference container credentials rather than the required diagnostic-and-remedy pair for tag-based ml training cost attribution. Use evidence: Inspect resource tags, cost allocation tag activation, Cost Explorer grouping, and unallocated resource types. and the matching correction: Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. Signal: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: During a production MLOps reliability and security review.
E:Authorize the specific serving role to decrypt with the relevant KMS key under a restricted key policy and S3 access. This selection concerns s3 sse-kms cross-account model artifact access rather than the required diagnostic-and-remedy pair for tag-based ml training cost attribution. Use evidence: Inspect resource tags, cost allocation tag activation, Cost Explorer grouping, and unallocated resource types. and the matching correction: Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. Signal: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: During a production MLOps reliability and security review.
Question 4
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Change control requires measurable endpoint or pipeline acceptance evidence. Which engineering response resolves the actual issue without masking the symptoms?
- Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages.
- Use SageMaker asynchronous inference with S3 input/output and suitable queue and notification settings.
- Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds.
- Implement the SageMaker inference container
/ping and /invocations expectations or use a supported serving toolkit.
Explanation
Correct Answer
A:Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. This is the appropriate controlled remedy for model registry version promotion provenance. Immutable package versions and approval metadata connect evaluation with deployment; mutable S3 nicknames do not. Verify the cause with: Inspect model registry package versions, approval metadata, artifact hashes, and pipeline deployment inputs. Then confirm: The production endpoint matches the reviewed model package and auditor can trace its evaluation and approver. Reported signal: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
B:Use SageMaker asynchronous inference with S3 input/output and suitable queue and notification settings. This addresses a different issue involving asynchronous endpoint for large payloads. It does not correct the stated model registry version promotion provenance mechanism. The targeted response is: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. Case evidence: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
C:Require model evaluation and explicit approval gates before production deployment, blocking failed metric thresholds. This addresses a different issue involving codepipeline approval gate on model metrics. It does not correct the stated model registry version promotion provenance mechanism. The targeted response is: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. Case evidence: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
D:Implement the SageMaker inference container /ping and /invocations expectations or use a supported serving toolkit. This addresses a different issue involving bring-your-own-inference-container contract. It does not correct the stated model registry version promotion provenance mechanism. The targeted response is: Promote an explicitly approved SageMaker Model Registry package version through repeatable pipeline stages. Case evidence: A pipeline deploys a model directly from an unversioned notebook S3 path and cannot prove which evaluated candidate was approved. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Question 5
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: A real-time ML service is configured for very low baseline capacity but traffic can surge tenfold within seconds, exhausting available instance capacity. Change control requires measurable endpoint or pipeline acceptance evidence. Which observation most directly identifies the root cause of the reported behavior?
- Inspect production variant weights, guardrail metrics, rollback procedure, and representativeness of staged exposure.
- Inspect container startup logs, listening port, health endpoint, MIME types, and inference response contract.
- Inspect service quotas, minimum and maximum endpoint capacity, scaling cooldowns, startup time, and burst size.
- Inspect model sizes, framework compatibility, per-model request rates, load latency, and isolation requirements.
Explanation
Correct Answer
C:Inspect service quotas, minimum and maximum endpoint capacity, scaling cooldowns, startup time, and burst size. This directly tests endpoint scale-to-zero cold capacity planning in the reported AWS ML workflow. Reactive scaling takes time and is bounded by quotas; zero spare capacity may not absorb an instantaneous burst. It distinguishes the problem before choosing a change. The related targeted response is: Set adequate minimum capacity, scaling limits, alarms, and request backpressure for the expected traffic ramp. Reported signal: A real-time ML service is configured for very low baseline capacity but traffic can surge tenfold within seconds, exhausting available instance capacity. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
A:Inspect production variant weights, guardrail metrics, rollback procedure, and representativeness of staged exposure. This is a possible diagnostic for canary variant rollout for model changes, but it does not directly test the endpoint scale-to-zero cold capacity planning failure reported here. Prioritize: Inspect service quotas, minimum and maximum endpoint capacity, scaling cooldowns, startup time, and burst size. The relevant case is: A real-time ML service is configured for very low baseline capacity but traffic can surge tenfold within seconds, exhausting available instance capacity. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
B:Inspect container startup logs, listening port, health endpoint, MIME types, and inference response contract. This is a possible diagnostic for bring-your-own-inference-container contract, but it does not directly test the endpoint scale-to-zero cold capacity planning failure reported here. Prioritize: Inspect service quotas, minimum and maximum endpoint capacity, scaling cooldowns, startup time, and burst size. The relevant case is: A real-time ML service is configured for very low baseline capacity but traffic can surge tenfold within seconds, exhausting available instance capacity. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
D:Inspect model sizes, framework compatibility, per-model request rates, load latency, and isolation requirements. This is a possible diagnostic for multi-model endpoint hosting tradeoff, but it does not directly test the endpoint scale-to-zero cold capacity planning failure reported here. Prioritize: Inspect service quotas, minimum and maximum endpoint capacity, scaling cooldowns, startup time, and burst size. The relevant case is: A real-time ML service is configured for very low baseline capacity but traffic can surge tenfold within seconds, exhausting available instance capacity. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Question 6
While testing a second region for recoverable deployment of the same ML service, the following condition is observed: A spiky serverless ML endpoint exceeds customer latency targets on cold starts even though average compute utilization is low. The deployment must be reproducible without manual console-only fixes. Which engineering response resolves the actual issue without masking the symptoms?
- Set adequate minimum capacity, scaling limits, alarms, and request backpressure for the expected traffic ramp.
- Use a compatible SageMaker multi-model endpoint with managed model loading and monitor cache/eviction behavior.
- Configure supported provisioned concurrency for the latency-critical interval or switch to provisioned real-time hosting if justified.
- Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete.
Explanation
Correct Answer
C:Configure supported provisioned concurrency for the latency-critical interval or switch to provisioned real-time hosting if justified. This is the appropriate controlled remedy for sagemaker endpoint provisioned concurrency tradeoff. Provisioned concurrency can reduce cold-start effects but adds reserved cost and must match supported endpoint mode. Verify the cause with: Inspect serverless invocation frequency, cold-start latency, supported provisioned concurrency settings, and cost. Then confirm: Cold-start p95 improves during planned demand windows while capacity cost remains measured. Reported signal: A spiky serverless ML endpoint exceeds customer latency targets on cold starts even though average compute utilization is low. Operational setting: While testing a second region for recoverable deployment of the same ML service.
Incorrect Answers
A:Set adequate minimum capacity, scaling limits, alarms, and request backpressure for the expected traffic ramp. This addresses a different issue involving endpoint scale-to-zero cold capacity planning. It does not correct the stated sagemaker endpoint provisioned concurrency tradeoff mechanism. The targeted response is: Configure supported provisioned concurrency for the latency-critical interval or switch to provisioned real-time hosting if justified. Case evidence: A spiky serverless ML endpoint exceeds customer latency targets on cold starts even though average compute utilization is low. Operational setting: While testing a second region for recoverable deployment of the same ML service.
B:Use a compatible SageMaker multi-model endpoint with managed model loading and monitor cache/eviction behavior. This addresses a different issue involving multi-model endpoint hosting tradeoff. It does not correct the stated sagemaker endpoint provisioned concurrency tradeoff mechanism. The targeted response is: Configure supported provisioned concurrency for the latency-critical interval or switch to provisioned real-time hosting if justified. Case evidence: A spiky serverless ML endpoint exceeds customer latency targets on cold starts even though average compute utilization is low. Operational setting: While testing a second region for recoverable deployment of the same ML service.
D:Filter appropriate validated data-ready events and trigger retraining only after the required dataset manifest is complete. This addresses a different issue involving eventbridge conditional retraining trigger. It does not correct the stated sagemaker endpoint provisioned concurrency tradeoff mechanism. The targeted response is: Configure supported provisioned concurrency for the latency-critical interval or switch to provisioned real-time hosting if justified. Case evidence: A spiky serverless ML endpoint exceeds customer latency targets on cold starts even though average compute utilization is low. Operational setting: While testing a second region for recoverable deployment of the same ML service.
Question 7
As a regional data-pipeline rollout is validated against approved AWS storage and streams, the following condition is observed: Several distributed workers repeatedly read small shared checkpoints with POSIX file semantics, but the pipeline treats object-storage paths as a mutable shared filesystem. The data team must preserve event-time meaning and lineage. Which verification result BEST demonstrates that the issue is resolved?
- Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path.
- Agreement on a held-out gold set rises without silently discarding rare legitimate classes.
- No synthetic or duplicated minority instance leaks into held-out folds, and minority-class recall reflects genuine unseen cases.
- Concurrent workers observe consistent supported shared-file operations and training artifact delivery remains durable.
Explanation
Correct Answer
D:Concurrent workers observe consistent supported shared-file operations and training artifact delivery remains durable. This is the acceptance evidence for the described s3 versus efs training storage choice scenario. S3 object semantics differ from mounted shared filesystems; choose storage for workload access behavior. The corrective implementation it validates is: Place shared POSIX-style working files on suitable EFS or FSx storage and keep immutable large artifacts in S3. Reported signal: Several distributed workers repeatedly read small shared checkpoints with POSIX file semantics, but the pipeline treats object-storage paths as a mutable shared filesystem. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Incorrect Answers
A:Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path. This could validate glue schema evolution with mixed input types, but it does not establish that s3 versus efs training storage choice is resolved for this workload. Confirm instead: Concurrent workers observe consistent supported shared-file operations and training artifact delivery remains durable. Observed problem: Several distributed workers repeatedly read small shared checkpoints with POSIX file semantics, but the pipeline treats object-storage paths as a mutable shared filesystem. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
B:Agreement on a held-out gold set rises without silently discarding rare legitimate classes. This could validate ground truth labeling agreement review, but it does not establish that s3 versus efs training storage choice is resolved for this workload. Confirm instead: Concurrent workers observe consistent supported shared-file operations and training artifact delivery remains durable. Observed problem: Several distributed workers repeatedly read small shared checkpoints with POSIX file semantics, but the pipeline treats object-storage paths as a mutable shared filesystem. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
C:No synthetic or duplicated minority instance leaks into held-out folds, and minority-class recall reflects genuine unseen cases. This could validate class imbalance handled inside cross-validation, but it does not establish that s3 versus efs training storage choice is resolved for this workload. Confirm instead: Concurrent workers observe consistent supported shared-file operations and training artifact delivery remains durable. Observed problem: Several distributed workers repeatedly read small shared checkpoints with POSIX file semantics, but the pipeline treats object-storage paths as a mutable shared filesystem. Operational setting: As a regional data-pipeline rollout is validated against approved AWS storage and streams.
Question 8
While reproducing the alert in an isolated AWS ML integration account, the following condition is observed: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. The proposed change must preserve logging and least privilege. What targeted change should be implemented after verifying the stated cause?
- Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed.
- Enable appropriate CloudTrail S3 data events for the sensitive dataset with protected storage and retention.
- Grant only the needed S3 and KMS actions for the approved data prefix and ensure appropriate role trust.
- Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes.
Explanation
Correct Answer
D:Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes. This is the appropriate controlled remedy for clarify postdeployment group bias monitoring. Aggregate performance can mask group-specific shifts; fairness monitoring must examine appropriate group metrics. Verify the cause with: Inspect group-level prediction and label distributions, subgroup sample sizes, and configured bias metrics. Then confirm: Auditors see reproducible subgroup statistics and alarms for material disparities rather than only global totals. Reported signal: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Incorrect Answers
A:Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. This addresses a different issue involving serving output confidence calibration drift. It does not correct the stated clarify postdeployment group bias monitoring mechanism. The targeted response is: Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes. Case evidence: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
B:Enable appropriate CloudTrail S3 data events for the sensitive dataset with protected storage and retention. This addresses a different issue involving cloudtrail training data access audit coverage. It does not correct the stated clarify postdeployment group bias monitoring mechanism. The targeted response is: Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes. Case evidence: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
C:Grant only the needed S3 and KMS actions for the approved data prefix and ensure appropriate role trust. This addresses a different issue involving sagemaker execution role s3 least privilege. It does not correct the stated clarify postdeployment group bias monitoring mechanism. The targeted response is: Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes. Case evidence: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Question 9
As a regulated ML workload prepares for an automated regional rollout, the following condition is observed: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. The team must demonstrate a repeatable monitoring or security control. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect caller identity, pass-role resource ARN, service constraint, and denied event attributes.
- Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time.
- Use a consistent versioned feature definition and test offline/online values at the same observation cutoff.
- Compare probability calibration plots, Brier score and event rates across recent labeled cohorts.
- Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed.
Explanation
Correct Answers
D:Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. This directly tests serving output confidence calibration drift in the reported AWS ML workflow. A good ranking metric does not guarantee calibrated probabilities; calibration must be verified against observed outcomes. It distinguishes the problem before choosing a change. The related targeted response is: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
E:Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. This is the appropriate controlled remedy for serving output confidence calibration drift. A good ranking metric does not guarantee calibrated probabilities; calibration must be verified against observed outcomes. Verify the cause with: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. Then confirm: Predicted probability buckets align with observed frequencies on held-out recent labeled cohorts. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Incorrect Answers
A:Inspect caller identity, pass-role resource ARN, service constraint, and denied event attributes. This selection concerns iam passrole constraint for model deployment rather than the required diagnostic-and-remedy pair for serving output confidence calibration drift. Use evidence: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. and the matching correction: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. Signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
B:Use structured logging and scoped CloudWatch Logs Insights queries to aggregate failures by training job and time. This selection concerns cloudwatch logs insights error correlation rather than the required diagnostic-and-remedy pair for serving output confidence calibration drift. Use evidence: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. and the matching correction: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. Signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
C:Use a consistent versioned feature definition and test offline/online values at the same observation cutoff. This selection concerns training-serving feature skew incident rather than the required diagnostic-and-remedy pair for serving output confidence calibration drift. Use evidence: Compare probability calibration plots, Brier score and event rates across recent labeled cohorts. and the matching correction: Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. Signal: A binary risk model continues producing apparently confident scores after a market regime change, but realized outcomes show the scores no longer correspond to observed event frequency. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Question 10
At the model-governance gate before promotion of a new ML artifact, the following condition is observed: A text classifier includes thousands of weak correlated features and becomes unstable across repeated training runs. The team needs a defensible technical decision rather than an untracked workaround. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect tensor summaries, gradient magnitudes, learning-rate schedule, and training loss anomalies with supported debugging hooks.
- Use a smaller learning rate, optionally freeze early layers, and assess staged unfreezing against a held-out set.
- Use justified feature selection or sparsity constraints fitted on training folds before retraining.
- Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses.
- Inspect feature importance stability, variance across folds, regularization and model dimensionality.
Explanation
Correct Answers
C:Use justified feature selection or sparsity constraints fitted on training folds before retraining. This is the appropriate controlled remedy for feature selection to reduce overparameterization. Unnecessary high-dimensional inputs can increase variance and compute cost; selection itself must avoid evaluation leakage. Verify the cause with: Inspect feature importance stability, variance across folds, regularization and model dimensionality. Then confirm: Holdout quality stays acceptable while model size and run-to-run instability decrease. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A text classifier includes thousands of weak correlated features and becomes unstable across repeated training runs. Operational setting: At the model-governance gate before promotion of a new ML artifact.
E:Inspect feature importance stability, variance across folds, regularization and model dimensionality. This directly tests feature selection to reduce overparameterization in the reported AWS ML workflow. Unnecessary high-dimensional inputs can increase variance and compute cost; selection itself must avoid evaluation leakage. It distinguishes the problem before choosing a change. The related targeted response is: Use justified feature selection or sparsity constraints fitted on training folds before retraining. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A text classifier includes thousands of weak correlated features and becomes unstable across repeated training runs. Operational setting: At the model-governance gate before promotion of a new ML artifact.
Incorrect Answers
A:Inspect tensor summaries, gradient magnitudes, learning-rate schedule, and training loss anomalies with supported debugging hooks. This selection concerns model debugger exploding gradient diagnosis rather than the required diagnostic-and-remedy pair for feature selection to reduce overparameterization. Use evidence: Inspect feature importance stability, variance across folds, regularization and model dimensionality. and the matching correction: Use justified feature selection or sparsity constraints fitted on training folds before retraining. Signal: A text classifier includes thousands of weak correlated features and becomes unstable across repeated training runs. Operational setting: At the model-governance gate before promotion of a new ML artifact.
B:Use a smaller learning rate, optionally freeze early layers, and assess staged unfreezing against a held-out set. This selection concerns fine-tuning pretrained model learning-rate stability rather than the required diagnostic-and-remedy pair for feature selection to reduce overparameterization. Use evidence: Inspect feature importance stability, variance across folds, regularization and model dimensionality. and the matching correction: Use justified feature selection or sparsity constraints fitted on training folds before retraining. Signal: A text classifier includes thousands of weak correlated features and becomes unstable across repeated training runs. Operational setting: At the model-governance gate before promotion of a new ML artifact.
D:Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses. This selection concerns shadow deployment model evaluation without response change rather than the required diagnostic-and-remedy pair for feature selection to reduce overparameterization. Use evidence: Inspect feature importance stability, variance across folds, regularization and model dimensionality. and the matching correction: Use justified feature selection or sparsity constraints fitted on training folds before retraining. Signal: A text classifier includes thousands of weak correlated features and becomes unstable across repeated training runs. Operational setting: At the model-governance gate before promotion of a new ML artifact.