Amazon AWS Certified Machine Learning Engineer - Associate MLA-C01 Practice Test 36
Question 350
During a production MLOps reliability and security review, the following condition is observed: A monitor issues thousands of alerts after a small harmless shift in an optional numeric feature and on-call engineers begin ignoring all violations. The owner needs an auditable resolution without broadening privileges. Which measured outcome would confirm that the correction worked as intended?
- Parity checks match transformations within documented freshness tolerances for test entities.
- Representative benign fluctuations stay quiet while a known severe schema or distribution break pages the owner.
- A query isolates the failing job versions and recurring error class within the incident interval.
- Training jobs reach the intended service privately while unauthorized external egress stays blocked.
Explanation
Correct Answer
B:Representative benign fluctuations stay quiet while a known severe schema or distribution break pages the owner. This is the acceptance evidence for the described model monitor violation alert calibration scenario. Monitoring must balance sensitivity and alert fatigue; treating every small statistical change as an incident is unreliable. The corrective implementation it validates is: Tune justified thresholds and group actionable monitor violations by severity without suppressing critical data-integrity checks. Reported signal: A monitor issues thousands of alerts after a small harmless shift in an optional numeric feature and on-call engineers begin ignoring all violations. Operational setting: During a production MLOps reliability and security review.
Incorrect Answers
A:Parity checks match transformations within documented freshness tolerances for test entities. This could validate training-serving feature skew incident, but it does not establish that model monitor violation alert calibration is resolved for this workload. Confirm instead: Representative benign fluctuations stay quiet while a known severe schema or distribution break pages the owner. Observed problem: A monitor issues thousands of alerts after a small harmless shift in an optional numeric feature and on-call engineers begin ignoring all violations. Operational setting: During a production MLOps reliability and security review.
C:A query isolates the failing job versions and recurring error class within the incident interval. This could validate cloudwatch logs insights error correlation, but it does not establish that model monitor violation alert calibration is resolved for this workload. Confirm instead: Representative benign fluctuations stay quiet while a known severe schema or distribution break pages the owner. Observed problem: A monitor issues thousands of alerts after a small harmless shift in an optional numeric feature and on-call engineers begin ignoring all violations. Operational setting: During a production MLOps reliability and security review.
D:Training jobs reach the intended service privately while unauthorized external egress stays blocked. This could validate private vpc endpoint policy for sagemaker apis, but it does not establish that model monitor violation alert calibration is resolved for this workload. Confirm instead: Representative benign fluctuations stay quiet while a known severe schema or distribution break pages the owner. Observed problem: A monitor issues thousands of alerts after a small harmless shift in an optional numeric feature and on-call engineers begin ignoring all violations. Operational setting: During a production MLOps reliability and security review.
Question 351
During a production MLOps reliability and security review, the following condition is observed: SageMaker Model Monitor model-quality jobs cannot match later labels to predictions because inference requests lack stable inference IDs. The owner needs an auditable resolution without broadening privileges. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect tuning max jobs and parallel jobs, usage forecasts, AWS Budgets thresholds, and quota settings.
- Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed.
- Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis.
- Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it.
- Inspect captured request IDs, ground-truth keys, and the quality monitor dataset schema.
Explanation
Correct Answers
C:Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis. This is the appropriate controlled remedy for model monitor ground truth association. Model quality analysis requires reliable association between predictions and subsequent truth records. Verify the cause with: Inspect captured request IDs, ground-truth keys, and the quality monitor dataset schema. Then confirm: The monitor joins each known labeled request to the corresponding earlier prediction and computes expected metrics. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: SageMaker Model Monitor model-quality jobs cannot match later labels to predictions because inference requests lack stable inference IDs. Operational setting: During a production MLOps reliability and security review.
E:Inspect captured request IDs, ground-truth keys, and the quality monitor dataset schema. This directly tests model monitor ground truth association in the reported AWS ML workflow. Model quality analysis requires reliable association between predictions and subsequent truth records. It distinguishes the problem before choosing a change. The related targeted response is: Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: SageMaker Model Monitor model-quality jobs cannot match later labels to predictions because inference requests lack stable inference IDs. Operational setting: During a production MLOps reliability and security review.
Incorrect Answers
A:Inspect tuning max jobs and parallel jobs, usage forecasts, AWS Budgets thresholds, and quota settings. This selection concerns aws budgets spend threshold for experimentation rather than the required diagnostic-and-remedy pair for model monitor ground truth association. Use evidence: Inspect captured request IDs, ground-truth keys, and the quality monitor dataset schema. and the matching correction: Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis. Signal: SageMaker Model Monitor model-quality jobs cannot match later labels to predictions because inference requests lack stable inference IDs. Operational setting: During a production MLOps reliability and security review.
B:Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. This selection concerns serving output confidence calibration drift rather than the required diagnostic-and-remedy pair for model monitor ground truth association. Use evidence: Inspect captured request IDs, ground-truth keys, and the quality monitor dataset schema. and the matching correction: Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis. Signal: SageMaker Model Monitor model-quality jobs cannot match later labels to predictions because inference requests lack stable inference IDs. Operational setting: During a production MLOps reliability and security review.
D:Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. This selection concerns spot training cost versus interruption recovery rather than the required diagnostic-and-remedy pair for model monitor ground truth association. Use evidence: Inspect captured request IDs, ground-truth keys, and the quality monitor dataset schema. and the matching correction: Capture stable prediction identifiers and ingest verified labels with matching keys for model-quality analysis. Signal: SageMaker Model Monitor model-quality jobs cannot match later labels to predictions because inference requests lack stable inference IDs. Operational setting: During a production MLOps reliability and security review.
Question 352
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A new marketing campaign adds hundreds of thousands of unique user IDs, making naïve one-hot features extremely sparse and too large to maintain. The dataset contract must remain reproducible for a backfill. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect sampler placement, duplicate neighbor IDs, fold membership, and class-conditioned metric changes.
- Use a CSV parser with explicit quoting and schema validation; quarantine malformed records before training.
- Split first; fit the imputer only on the training partition and apply those learned statistics to validation and test data.
- Inspect categorical cardinality, sparse matrix dimensions, feature contribution, and generalization to unseen identifiers.
- Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling.
Explanation
Correct Answers
D:Inspect categorical cardinality, sparse matrix dimensions, feature contribution, and generalization to unseen identifiers. This directly tests high-cardinality categorical encoding in the reported AWS ML workflow. One-hot encoding every nearly unique identifier can waste capacity and overfit identities instead of learning transferable patterns. It distinguishes the problem before choosing a change. The related targeted response is: Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A new marketing campaign adds hundreds of thousands of unique user IDs, making naïve one-hot features extremely sparse and too large to maintain. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
E:Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling. This is the appropriate controlled remedy for high-cardinality categorical encoding. One-hot encoding every nearly unique identifier can waste capacity and overfit identities instead of learning transferable patterns. Verify the cause with: Inspect categorical cardinality, sparse matrix dimensions, feature contribution, and generalization to unseen identifiers. Then confirm: Memory use stays within limits and previously unseen categories are processed without dimension mismatch. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A new marketing campaign adds hundreds of thousands of unique user IDs, making naïve one-hot features extremely sparse and too large to maintain. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
A:Inspect sampler placement, duplicate neighbor IDs, fold membership, and class-conditioned metric changes. This selection concerns class imbalance handled inside cross-validation rather than the required diagnostic-and-remedy pair for high-cardinality categorical encoding. Use evidence: Inspect categorical cardinality, sparse matrix dimensions, feature contribution, and generalization to unseen identifiers. and the matching correction: Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling. Signal: A new marketing campaign adds hundreds of thousands of unique user IDs, making naïve one-hot features extremely sparse and too large to maintain. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
B:Use a CSV parser with explicit quoting and schema validation; quarantine malformed records before training. This selection concerns csv schema ambiguity during ingestion rather than the required diagnostic-and-remedy pair for high-cardinality categorical encoding. Use evidence: Inspect categorical cardinality, sparse matrix dimensions, feature contribution, and generalization to unseen identifiers. and the matching correction: Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling. Signal: A new marketing campaign adds hundreds of thousands of unique user IDs, making naïve one-hot features extremely sparse and too large to maintain. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Split first; fit the imputer only on the training partition and apply those learned statistics to validation and test data. This selection concerns median imputation fitted on training split rather than the required diagnostic-and-remedy pair for high-cardinality categorical encoding. Use evidence: Inspect categorical cardinality, sparse matrix dimensions, feature contribution, and generalization to unseen identifiers. and the matching correction: Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling. Signal: A new marketing campaign adds hundreds of thousands of unique user IDs, making naïve one-hot features extremely sparse and too large to maintain. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 353
As a regulated ML workload prepares for an automated regional rollout, the following condition is observed: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. The team must demonstrate a repeatable monitoring or security control. What targeted change should be implemented after verifying the stated cause?
- Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting.
- Enable appropriate CloudTrail S3 data events for the sensitive dataset with protected storage and retention.
- Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable.
- Identify the component contributing to total request time and optimize network or overhead separately from actual model inference.
Explanation
Correct Answer
A:Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. This is the appropriate controlled remedy for tag-based ml training cost attribution. Cost Explorer grouping requires supported activated resource tags; naming resources does not automatically allocate charges. Verify the cause with: Inspect resource tags, cost allocation tag activation, Cost Explorer grouping, and unallocated resource types. Then confirm: Cost reports attribute the majority of in-scope ML spend to the correct team without altering access controls. Reported signal: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Incorrect Answers
B:Enable appropriate CloudTrail S3 data events for the sensitive dataset with protected storage and retention. This addresses a different issue involving cloudtrail training data access audit coverage. It does not correct the stated tag-based ml training cost attribution mechanism. The targeted response is: Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. Case evidence: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
C:Use stable user-cohort assignment or an appropriate experiment design while keeping exposed groups measurable. This addresses a different issue involving champion challenger a/b assignment consistency. It does not correct the stated tag-based ml training cost attribution mechanism. The targeted response is: Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. Case evidence: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
D:Identify the component contributing to total request time and optimize network or overhead separately from actual model inference. This addresses a different issue involving cloudwatch invocations and modellatency distinction. It does not correct the stated tag-based ml training cost attribution mechanism. The targeted response is: Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. Case evidence: An organization cannot allocate expensive SageMaker training jobs among departments because jobs and dependent storage lack common cost allocation tags. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Question 354
As a regulated ML workload prepares for an automated regional rollout, the following condition is observed: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. The team must demonstrate a repeatable monitoring or security control. Which observation most directly identifies the root cause of the reported behavior?
- Inspect constraint threshold selection, false-positive incidents, feature importance and baseline seasonal variation.
- Compare evaluated artifact hash, S3 object version, package provenance, and deployment manifest.
- Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics.
- Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution.
Explanation
Correct Answer
C:Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. This directly tests target-label delay in production quality monitoring in the reported AWS ML workflow. True supervised production quality generally needs observed outcomes; prediction outputs alone are not ground truth. It distinguishes the problem before choosing a change. The related targeted response is: Separate prompt input-distribution monitoring from delayed model-quality evaluation and join only observed labels to predictions. Reported signal: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Incorrect Answers
A:Inspect constraint threshold selection, false-positive incidents, feature importance and baseline seasonal variation. This is a possible diagnostic for model monitor violation alert calibration, but it does not directly test the target-label delay in production quality monitoring failure reported here. Prioritize: Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. The relevant case is: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
B:Compare evaluated artifact hash, S3 object version, package provenance, and deployment manifest. This is a possible diagnostic for model artifact supply-chain integrity, but it does not directly test the target-label delay in production quality monitoring failure reported here. Prioritize: Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. The relevant case is: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
D:Inspect endpoint 5XXErrors, InvocationsPerInstance, instance availability, and per-minute latency distribution. This is a possible diagnostic for cloudwatch 5xxerrors endpoint saturation, but it does not directly test the target-label delay in production quality monitoring failure reported here. Prioritize: Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. The relevant case is: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Question 355
During a production MLOps reliability and security review, the following condition is observed: A regulated training process requires private network paths, but its isolated subnet cannot call the intended SageMaker API without going through public internet. The owner needs an auditable resolution without broadening privileges. Which corrective action most directly addresses this ML workflow problem?
- Use a consistent versioned feature definition and test offline/online values at the same observation cutoff.
- Use approved SageMaker interface endpoints and relevant private DNS and policy settings for required API access.
- Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting.
- Grant only the needed S3 and KMS actions for the approved data prefix and ensure appropriate role trust.
Explanation
Correct Answer
B:Use approved SageMaker interface endpoints and relevant private DNS and policy settings for required API access. This is the appropriate controlled remedy for private vpc endpoint policy for sagemaker apis. An IAM role permits API actions but network access and endpoint policies must also support the private path. Verify the cause with: Inspect subnet routes, VPC interface endpoints, private DNS settings, endpoint security groups, and endpoint policies. Then confirm: Training jobs reach the intended service privately while unauthorized external egress stays blocked. Reported signal: A regulated training process requires private network paths, but its isolated subnet cannot call the intended SageMaker API without going through public internet. Operational setting: During a production MLOps reliability and security review.
Incorrect Answers
A:Use a consistent versioned feature definition and test offline/online values at the same observation cutoff. This addresses a different issue involving training-serving feature skew incident. It does not correct the stated private vpc endpoint policy for sagemaker apis mechanism. The targeted response is: Use approved SageMaker interface endpoints and relevant private DNS and policy settings for required API access. Case evidence: A regulated training process requires private network paths, but its isolated subnet cannot call the intended SageMaker API without going through public internet. Operational setting: During a production MLOps reliability and security review.
C:Apply consistent approved cost allocation tags to ML jobs and supported linked resources and activate relevant reporting. This addresses a different issue involving tag-based ml training cost attribution. It does not correct the stated private vpc endpoint policy for sagemaker apis mechanism. The targeted response is: Use approved SageMaker interface endpoints and relevant private DNS and policy settings for required API access. Case evidence: A regulated training process requires private network paths, but its isolated subnet cannot call the intended SageMaker API without going through public internet. Operational setting: During a production MLOps reliability and security review.
D:Grant only the needed S3 and KMS actions for the approved data prefix and ensure appropriate role trust. This addresses a different issue involving sagemaker execution role s3 least privilege. It does not correct the stated private vpc endpoint policy for sagemaker apis mechanism. The targeted response is: Use approved SageMaker interface endpoints and relevant private DNS and policy settings for required API access. Case evidence: A regulated training process requires private network paths, but its isolated subnet cannot call the intended SageMaker API without going through public internet. Operational setting: During a production MLOps reliability and security review.
Question 356
At a production AWS ML data lake supporting customer-facing predictions, the following condition is observed: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Engineers must avoid changing unrelated source partitions. Which corrective action most directly addresses this ML workflow problem?
- Define event-time windows with appropriate lateness handling and a documented revision policy for late records.
- Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling.
- Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features.
- Choose partition keys with enough cardinality or reshard the stream for the intended throughput, while preserving required per-key ordering.
Explanation
Correct Answer
C:Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features. This is the appropriate controlled remedy for glue schema evolution with mixed input types. A catalog entry does not automatically reconcile incompatible field types across all source partitions. Verify the cause with: Compare source schemas by partition, Glue Data Catalog types, and failed cast rows for the evolving field. Then confirm: Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path. Reported signal: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Incorrect Answers
A:Define event-time windows with appropriate lateness handling and a documented revision policy for late records. This addresses a different issue involving streaming feature rolling window correctness. It does not correct the stated glue schema evolution with mixed input types mechanism. The targeted response is: Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features. Case evidence: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
B:Drop leakage-prone IDs or adopt controlled hashing or suitable learned encodings with stable unknown-category handling. This addresses a different issue involving high-cardinality categorical encoding. It does not correct the stated glue schema evolution with mixed input types mechanism. The targeted response is: Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features. Case evidence: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
D:Choose partition keys with enough cardinality or reshard the stream for the intended throughput, while preserving required per-key ordering. This addresses a different issue involving kinesis data streams shard hot key. It does not correct the stated glue schema evolution with mixed input types mechanism. The targeted response is: Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features. Case evidence: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Question 357
While reproducing the alert in an isolated AWS ML integration account, the following condition is observed: An unconstrained SageMaker tuning experiment launches hundreds of expensive GPU jobs, but the team notices the bill only at month end. The proposed change must preserve logging and least privilege. What postchange observation provides the strongest acceptance evidence?
- The new container operates with rotated credentials and image scans find no active embedded password.
- Representative benign fluctuations stay quiet while a known severe schema or distribution break pages the owner.
- A test forecast crossing the approved spend threshold alerts before uncontrolled parallel job growth.
- The chosen fix reduces measured end-to-end p95 latency and component metrics explain the change.
Explanation
Correct Answer
C:A test forecast crossing the approved spend threshold alerts before uncontrolled parallel job growth. This is the acceptance evidence for the described aws budgets spend threshold for experimentation scenario. Budgets detect or warn about costs but do not automatically limit every ML training request; job limits are also needed. The corrective implementation it validates is: Set budget alerts and bounded tuning job quotas before launching experiments, and enforce an owner-approved stopping policy. Reported signal: An unconstrained SageMaker tuning experiment launches hundreds of expensive GPU jobs, but the team notices the bill only at month end. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Incorrect Answers
A:The new container operates with rotated credentials and image scans find no active embedded password. This could validate secret rotation for inference container credentials, but it does not establish that aws budgets spend threshold for experimentation is resolved for this workload. Confirm instead: A test forecast crossing the approved spend threshold alerts before uncontrolled parallel job growth. Observed problem: An unconstrained SageMaker tuning experiment launches hundreds of expensive GPU jobs, but the team notices the bill only at month end. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
B:Representative benign fluctuations stay quiet while a known severe schema or distribution break pages the owner. This could validate model monitor violation alert calibration, but it does not establish that aws budgets spend threshold for experimentation is resolved for this workload. Confirm instead: A test forecast crossing the approved spend threshold alerts before uncontrolled parallel job growth. Observed problem: An unconstrained SageMaker tuning experiment launches hundreds of expensive GPU jobs, but the team notices the bill only at month end. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
D:The chosen fix reduces measured end-to-end p95 latency and component metrics explain the change. This could validate cloudwatch invocations and modellatency distinction, but it does not establish that aws budgets spend threshold for experimentation is resolved for this workload. Confirm instead: A test forecast crossing the approved spend threshold alerts before uncontrolled parallel job growth. Observed problem: An unconstrained SageMaker tuning experiment launches hundreds of expensive GPU jobs, but the team notices the bill only at month end. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Question 358
As a regulated ML workload prepares for an automated regional rollout, the following condition is observed: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. The team must demonstrate a repeatable monitoring or security control. Which observation most directly identifies the root cause of the reported behavior?
- Measure memory, throughput, p95 latency and cost across supported candidate endpoint configurations.
- Inspect image layers, environment configuration, secret source, runtime role permissions, and rotation logs.
- Inspect group-level prediction and label distributions, subgroup sample sizes, and configured bias metrics.
- Inspect online versus offline feature computation code, event-time cutoffs, freshness, and sampled entity values.
Explanation
Correct Answer
C:Inspect group-level prediction and label distributions, subgroup sample sizes, and configured bias metrics. This directly tests clarify postdeployment group bias monitoring in the reported AWS ML workflow. Aggregate performance can mask group-specific shifts; fairness monitoring must examine appropriate group metrics. It distinguishes the problem before choosing a change. The related targeted response is: Use appropriate SageMaker Clarify model-bias monitoring and investigate supported group differences before deployment changes. Reported signal: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Incorrect Answers
A:Measure memory, throughput, p95 latency and cost across supported candidate endpoint configurations. This is a possible diagnostic for inference recommender benchmarking instance family, but it does not directly test the clarify postdeployment group bias monitoring failure reported here. Prioritize: Inspect group-level prediction and label distributions, subgroup sample sizes, and configured bias metrics. The relevant case is: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
B:Inspect image layers, environment configuration, secret source, runtime role permissions, and rotation logs. This is a possible diagnostic for secret rotation for inference container credentials, but it does not directly test the clarify postdeployment group bias monitoring failure reported here. Prioritize: Inspect group-level prediction and label distributions, subgroup sample sizes, and configured bias metrics. The relevant case is: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
D:Inspect online versus offline feature computation code, event-time cutoffs, freshness, and sampled entity values. This is a possible diagnostic for training-serving feature skew incident, but it does not directly test the clarify postdeployment group bias monitoring failure reported here. Prioritize: Inspect group-level prediction and label distributions, subgroup sample sizes, and configured bias metrics. The relevant case is: A model's overall approval rate remains stable while approval differences between customer groups increase after a data-source migration. Operational setting: As a regulated ML workload prepares for an automated regional rollout.
Question 359
At a production AWS ML data lake supporting customer-facing predictions, the following condition is observed: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Engineers must avoid changing unrelated source partitions. Which verification result BEST demonstrates that the issue is resolved?
- Agreement on a held-out gold set rises without silently discarding rare legitimate classes.
- Annotation workers can label required content without viewing direct identifiers and authorized audits can trace approved mappings.
- Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path.
- Concurrent workers observe consistent supported shared-file operations and training artifact delivery remains durable.
Explanation
Correct Answer
C:Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path. This is the acceptance evidence for the described glue schema evolution with mixed input types scenario. A catalog entry does not automatically reconcile incompatible field types across all source partitions. The corrective implementation it validates is: Define explicit compatible conversions and schema contracts, isolate incompatible versions, and validate before publishing features. Reported signal: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Incorrect Answers
A:Agreement on a held-out gold set rises without silently discarding rare legitimate classes. This could validate ground truth labeling agreement review, but it does not establish that glue schema evolution with mixed input types is resolved for this workload. Confirm instead: Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path. Observed problem: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
B:Annotation workers can label required content without viewing direct identifiers and authorized audits can trace approved mappings. This could validate pii masking before model labeling, but it does not establish that glue schema evolution with mixed input types is resolved for this workload. Confirm instead: Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path. Observed problem: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
D:Concurrent workers observe consistent supported shared-file operations and training artifact delivery remains durable. This could validate s3 versus efs training storage choice, but it does not establish that glue schema evolution with mixed input types is resolved for this workload. Confirm instead: Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path. Observed problem: New event producers change a numerical source field to strings while legacy objects remain numeric, causing the shared Glue transformation to fail. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Question 360
While reproducing a training incident in an independent SageMaker experiment, the following condition is observed: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. The experiment must be compared fairly against its prior baseline. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints.
- Inspect squared versus absolute residual contributions, target distribution, and large-error business costs.
- Register immutable model package versions with lineage and require explicit approval before promotion to production.
- Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy.
- Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses.
Explanation
Correct Answers
A:Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. This directly tests hyperparameter bayesian tuning strategy in the reported AWS ML workflow. Hyperparameter optimization needs a defined objective and controlled comparable trials, not arbitrary parameter changes. It distinguishes the problem before choosing a change. The related targeted response is: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
D:Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. This is the appropriate controlled remedy for hyperparameter bayesian tuning strategy. Hyperparameter optimization needs a defined objective and controlled comparable trials, not arbitrary parameter changes. Verify the cause with: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. Then confirm: Trials record comparable objective values and a selected candidate meets an independent validation target within budget. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Incorrect Answers
B:Inspect squared versus absolute residual contributions, target distribution, and large-error business costs. This selection concerns rmse versus mae outlier sensitivity rather than the required diagnostic-and-remedy pair for hyperparameter bayesian tuning strategy. Use evidence: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. and the matching correction: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. Signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
C:Register immutable model package versions with lineage and require explicit approval before promotion to production. This selection concerns sagemaker model registry candidate approval rather than the required diagnostic-and-remedy pair for hyperparameter bayesian tuning strategy. Use evidence: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. and the matching correction: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. Signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
E:Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses. This selection concerns shadow deployment model evaluation without response change rather than the required diagnostic-and-remedy pair for hyperparameter bayesian tuning strategy. Use evidence: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. and the matching correction: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. Signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
