Amazon AWS Certified Machine Learning Engineer - Associate MLA-C01 Practice Test 37
Question 360
While reproducing a training incident in an independent SageMaker experiment, the following condition is observed: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. The experiment must be compared fairly against its prior baseline. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints.
- Inspect squared versus absolute residual contributions, target distribution, and large-error business costs.
- Register immutable model package versions with lineage and require explicit approval before promotion to production.
- Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy.
- Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses.
Explanation
Correct Answers
A:Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. This directly tests hyperparameter bayesian tuning strategy in the reported AWS ML workflow. Hyperparameter optimization needs a defined objective and controlled comparable trials, not arbitrary parameter changes. It distinguishes the problem before choosing a change. The related targeted response is: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
D:Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. This is the appropriate controlled remedy for hyperparameter bayesian tuning strategy. Hyperparameter optimization needs a defined objective and controlled comparable trials, not arbitrary parameter changes. Verify the cause with: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. Then confirm: Trials record comparable objective values and a selected candidate meets an independent validation target within budget. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Incorrect Answers
B:Inspect squared versus absolute residual contributions, target distribution, and large-error business costs. This selection concerns rmse versus mae outlier sensitivity rather than the required diagnostic-and-remedy pair for hyperparameter bayesian tuning strategy. Use evidence: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. and the matching correction: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. Signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
C:Register immutable model package versions with lineage and require explicit approval before promotion to production. This selection concerns sagemaker model registry candidate approval rather than the required diagnostic-and-remedy pair for hyperparameter bayesian tuning strategy. Use evidence: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. and the matching correction: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. Signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
E:Deploy a SageMaker shadow variant for observational comparison before promoting the new model to served responses. This selection concerns shadow deployment model evaluation without response change rather than the required diagnostic-and-remedy pair for hyperparameter bayesian tuning strategy. Use evidence: Inspect training metrics per trial, search bounds, tuning-objective direction, and compute constraints. and the matching correction: Configure SageMaker Automatic Model Tuning with a bounded search space, objective metric, budget, and justified Bayesian or random strategy. Signal: An ML engineer manually tests hundreds of random combinations for an expensive model without recording objectives or resource budgets. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Question 361
At a production AWS ML data lake supporting customer-facing predictions, the following condition is observed: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Engineers must avoid changing unrelated source partitions. Which investigation provides the most decisive FIRST evidence for the underlying problem?
- Inspect sample raw records, quoting rules, expected headers, and parsed column counts in the ingestion job.
- Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale.
- Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set.
- Compare the query predicates with S3 object prefixes, partition metadata, and query plans to confirm partitions are not being pruned.
Explanation
Correct Answer
B:Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. This directly tests skewed numerical feature log transformation in the reported AWS ML workflow. A monotonic log transform can reduce heavy-tail influence, but invalid domains and train-serving parity must be handled. It distinguishes the problem before choosing a change. The related targeted response is: Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. Reported signal: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Incorrect Answers
A:Inspect sample raw records, quoting rules, expected headers, and parsed column counts in the ingestion job. This is a possible diagnostic for csv schema ambiguity during ingestion, but it does not directly test the skewed numerical feature log transformation failure reported here. Prioritize: Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. The relevant case is: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
C:Inspect per-class confusion, worker agreement, label instructions, and disagreements on an adjudicated reference set. This is a possible diagnostic for ground truth labeling agreement review, but it does not directly test the skewed numerical feature log transformation failure reported here. Prioritize: Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. The relevant case is: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
D:Compare the query predicates with S3 object prefixes, partition metadata, and query plans to confirm partitions are not being pruned. This is a possible diagnostic for partition pruning in s3 parquet training input, but it does not directly test the skewed numerical feature log transformation failure reported here. Prioritize: Inspect quantiles, heavy-tail plots, outliers, and the target model's response to feature scale. The relevant case is: A training job is dominated by extreme transaction-amount magnitudes, although most records have modest nonnegative amounts. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Question 362
At a production AWS ML data lake supporting customer-facing predictions, the following condition is observed: A daily AWS Glue ingestion job reprocesses a month of files after a deployment because its job bookmark state was reset or the job identity changed. Engineers must avoid changing unrelated source partitions. Which corrective action most directly addresses this ML workflow problem?
- Preserve the intended bookmark state across deployments and run an explicit controlled backfill only when required.
- Encode hour with sine and cosine cyclic features and validate with a time-aware split.
- Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory.
- Use controlled incremental extraction or a suitable read replica and throttle export jobs within the approved data freshness target.
Explanation
Correct Answer
A:Preserve the intended bookmark state across deployments and run an explicit controlled backfill only when required. This is the appropriate controlled remedy for glue job bookmark state error. Glue job bookmarks track supported processed state and must be managed deliberately during job changes. Verify the cause with: Check job names, bookmark settings, run history, source object timestamps, and the previously committed checkpoint. Then confirm: Two ordinary runs ingest only newly eligible objects while a separately invoked backfill handles prior intervals. Reported signal: A daily AWS Glue ingestion job reprocesses a month of files after a deployment because its job bookmark state was reset or the job identity changed. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Incorrect Answers
B:Encode hour with sine and cosine cyclic features and validate with a time-aware split. This addresses a different issue involving time-series cyclical calendar feature. It does not correct the stated glue job bookmark state error mechanism. The targeted response is: Preserve the intended bookmark state across deployments and run an explicit controlled backfill only when required. Case evidence: A daily AWS Glue ingestion job reprocesses a month of files after a deployment because its job bookmark state was reset or the job identity changed. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
C:Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory. This addresses a different issue involving glue spark join skew on dominant identifier. It does not correct the stated glue job bookmark state error mechanism. The targeted response is: Preserve the intended bookmark state across deployments and run an explicit controlled backfill only when required. Case evidence: A daily AWS Glue ingestion job reprocesses a month of files after a deployment because its job bookmark state was reset or the job identity changed. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
D:Use controlled incremental extraction or a suitable read replica and throttle export jobs within the approved data freshness target. This addresses a different issue involving rds extraction overload into ml. It does not correct the stated glue job bookmark state error mechanism. The targeted response is: Preserve the intended bookmark state across deployments and run an explicit controlled backfill only when required. Case evidence: A daily AWS Glue ingestion job reprocesses a month of files after a deployment because its job bookmark state was reset or the job identity changed. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Question 363
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. The dataset contract must remain reproducible for a backfill. Which observation most directly identifies the root cause of the reported behavior?
- Compare vocabulary hashes, normalization steps, token IDs for test strings, and serialized preprocessing versions.
- Measure file-size distribution, object request count, Spark task scheduling overhead, and read throughput.
- Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics.
- Inspect feature availability timestamps and whether each label time follows the last feature observation.
Explanation
Correct Answer
A:Compare vocabulary hashes, normalization steps, token IDs for test strings, and serialized preprocessing versions. This directly tests text tokenization mismatch in serving in the reported AWS ML workflow. Model embeddings depend on stable token-ID semantics; swapping tokenizers changes the feature representation. It distinguishes the problem before choosing a change. The related targeted response is: Package the training tokenizer and normalization configuration with the model artifact and require serving compatibility checks. Reported signal: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
B:Measure file-size distribution, object request count, Spark task scheduling overhead, and read throughput. This is a possible diagnostic for columnar file compaction for small objects, but it does not directly test the text tokenization mismatch in serving failure reported here. Prioritize: Compare vocabulary hashes, normalization steps, token IDs for test strings, and serialized preprocessing versions. The relevant case is: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics. This is a possible diagnostic for glue spark join skew on dominant identifier, but it does not directly test the text tokenization mismatch in serving failure reported here. Prioritize: Compare vocabulary hashes, normalization steps, token IDs for test strings, and serialized preprocessing versions. The relevant case is: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
D:Inspect feature availability timestamps and whether each label time follows the last feature observation. This is a possible diagnostic for chronological split for forecast evaluation, but it does not directly test the text tokenization mismatch in serving failure reported here. Prioritize: Compare vocabulary hashes, normalization steps, token IDs for test strings, and serialized preprocessing versions. The relevant case is: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 364
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A medical-image classifier uses random row splitting so multiple scans from the same patient occur in training and validation. The dataset contract must remain reproducible for a backfill. What should the ML engineer inspect FIRST to distinguish this failure from unrelated issues?
- Check job names, bookmark settings, run history, source object timestamps, and the previously committed checkpoint.
- Inspect shared entity identifiers across partitions and compare patient-level versus image-level evaluation results.
- Inspect event identifiers, object versions or ETags, processing ledger, and downstream record keys for duplicate deliveries.
- Inspect feature event timestamps, target prediction cutoffs, and join predicates in the historical training query.
Explanation
Correct Answer
B:Inspect shared entity identifiers across partitions and compare patient-level versus image-level evaluation results. This directly tests group-leakage train and validation split in the reported AWS ML workflow. Correlated records from one entity across splits can inflate performance without improving true generalization. It distinguishes the problem before choosing a change. The related targeted response is: Split by stable patient identity before training and reserve a disjoint held-out group for evaluation. Reported signal: A medical-image classifier uses random row splitting so multiple scans from the same patient occur in training and validation. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
A:Check job names, bookmark settings, run history, source object timestamps, and the previously committed checkpoint. This is a possible diagnostic for glue job bookmark state error, but it does not directly test the group-leakage train and validation split failure reported here. Prioritize: Inspect shared entity identifiers across partitions and compare patient-level versus image-level evaluation results. The relevant case is: A medical-image classifier uses random row splitting so multiple scans from the same patient occur in training and validation. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Inspect event identifiers, object versions or ETags, processing ledger, and downstream record keys for duplicate deliveries. This is a possible diagnostic for s3 object event at-least-once processing, but it does not directly test the group-leakage train and validation split failure reported here. Prioritize: Inspect shared entity identifiers across partitions and compare patient-level versus image-level evaluation results. The relevant case is: A medical-image classifier uses random row splitting so multiple scans from the same patient occur in training and validation. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
D:Inspect feature event timestamps, target prediction cutoffs, and join predicates in the historical training query. This is a possible diagnostic for point-in-time feature data leakage, but it does not directly test the group-leakage train and validation split failure reported here. Prioritize: Inspect shared entity identifiers across partitions and compare patient-level versus image-level evaluation results. The relevant case is: A medical-image classifier uses random row splitting so multiple scans from the same patient occur in training and validation. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 365
At a production AWS ML data lake supporting customer-facing predictions, the following condition is observed: A Spark transformation joins billions of records on a region key, and one region contains most observations, stalling one shuffle task. Engineers must avoid changing unrelated source partitions. Select TWO: the appropriate diagnostic check and the targeted implementation change.
- Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics.
- Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving.
- Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory.
- Inspect shared entity identifiers across partitions and compare patient-level versus image-level evaluation results.
- Use controlled incremental extraction or a suitable read replica and throttle export jobs within the approved data freshness target.
Explanation
Correct Answers
A:Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics. This directly tests glue spark join skew on dominant identifier in the reported AWS ML workflow. A low-cardinality or skewed key can concentrate a distributed join into one expensive partition. It distinguishes the problem before choosing a change. The related targeted response is: Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A Spark transformation joins billions of records on a region key, and one region contains most observations, stalling one shuffle task. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
C:Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory. This is the appropriate controlled remedy for glue spark join skew on dominant identifier. A low-cardinality or skewed key can concentrate a distributed join into one expensive partition. Verify the cause with: Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics. Then confirm: Long-tail task duration and shuffle spill decline without changing the joined record counts. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A Spark transformation joins billions of records on a region key, and one region contains most observations, stalling one shuffle task. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Incorrect Answers
B:Apply a justified log1p transformation with documented treatment of invalid negatives and package it for serving. This selection concerns skewed numerical feature log transformation rather than the required diagnostic-and-remedy pair for glue spark join skew on dominant identifier. Use evidence: Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics. and the matching correction: Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory. Signal: A Spark transformation joins billions of records on a region key, and one region contains most observations, stalling one shuffle task. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
D:Inspect shared entity identifiers across partitions and compare patient-level versus image-level evaluation results. This selection concerns group-leakage train and validation split rather than the required diagnostic-and-remedy pair for glue spark join skew on dominant identifier. Use evidence: Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics. and the matching correction: Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory. Signal: A Spark transformation joins billions of records on a region key, and one region contains most observations, stalling one shuffle task. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
E:Use controlled incremental extraction or a suitable read replica and throttle export jobs within the approved data freshness target. This selection concerns rds extraction overload into ml rather than the required diagnostic-and-remedy pair for glue spark join skew on dominant identifier. Use evidence: Inspect Spark stage durations, partition size distribution, join key frequencies, and executor spill metrics. and the matching correction: Repartition or salt the skewed join appropriately, or use a justified broadcast join when the small side fits memory. Signal: A Spark transformation joins billions of records on a region key, and one region contains most observations, stalling one shuffle task. Operational setting: At a production AWS ML data lake supporting customer-facing predictions.
Question 366
While reproducing a training incident in an independent SageMaker experiment, the following condition is observed: A team declares a new model better by comparing metrics from runs that used different target definitions and data partitions. The experiment must be compared fairly against its prior baseline. Which engineering response resolves the actual issue without masking the symptoms?
- Use tracked SageMaker Experiments runs with fixed comparable evaluation splits and a documented metric contract.
- Use appropriate SageMaker Debugger or framework instrumentation; adjust learning rate or gradient clipping after confirming instability.
- Select a regression approach with suitable loss and evaluation metrics instead of forcing continuous amounts into unrelated classes.
- Evaluate Amazon Textract form and table extraction first, adding custom processing only for unsupported fields.
Explanation
Correct Answer
A:Use tracked SageMaker Experiments runs with fixed comparable evaluation splits and a documented metric contract. This is the appropriate controlled remedy for reproducible sagemaker experiments comparisons. Performance claims are meaningful only when experiments control relevant changes and expose comparable evidence. Verify the cause with: Inspect experiment run lineage, dataset/version hashes, code revisions, random seeds, and metric definitions. Then confirm: Candidate comparisons reference equivalent datasets and metrics and can be rerun from stored lineage. Reported signal: A team declares a new model better by comparing metrics from runs that used different target definitions and data partitions. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Incorrect Answers
B:Use appropriate SageMaker Debugger or framework instrumentation; adjust learning rate or gradient clipping after confirming instability. This addresses a different issue involving model debugger exploding gradient diagnosis. It does not correct the stated reproducible sagemaker experiments comparisons mechanism. The targeted response is: Use tracked SageMaker Experiments runs with fixed comparable evaluation splits and a documented metric contract. Case evidence: A team declares a new model better by comparing metrics from runs that used different target definitions and data partitions. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
C:Select a regression approach with suitable loss and evaluation metrics instead of forcing continuous amounts into unrelated classes. This addresses a different issue involving regression versus classification problem formulation. It does not correct the stated reproducible sagemaker experiments comparisons mechanism. The targeted response is: Use tracked SageMaker Experiments runs with fixed comparable evaluation splits and a documented metric contract. Case evidence: A team declares a new model better by comparing metrics from runs that used different target definitions and data partitions. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
D:Evaluate Amazon Textract form and table extraction first, adding custom processing only for unsupported fields. This addresses a different issue involving managed ai service versus custom ocr model. It does not correct the stated reproducible sagemaker experiments comparisons mechanism. The targeted response is: Use tracked SageMaker Experiments runs with fixed comparable evaluation splits and a documented metric contract. Case evidence: A team declares a new model better by comparing metrics from runs that used different target definitions and data partitions. Operational setting: While reproducing a training incident in an independent SageMaker experiment.
Question 367
While reproducing the alert in an isolated AWS ML integration account, the following condition is observed: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. The proposed change must preserve logging and least privilege. Which TWO steps best establish the evidence and correct the identified root cause? Select TWO.
- Separate prompt input-distribution monitoring from delayed model-quality evaluation and join only observed labels to predictions.
- Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics.
- Inspect S3 object encryption, key policy, grant permissions, IAM role, and denied KMS decrypt events.
- Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed.
- Pin an immutable approved artifact version or content digest and verify integrity before deployment.
Explanation
Correct Answers
A:Separate prompt input-distribution monitoring from delayed model-quality evaluation and join only observed labels to predictions. This is the appropriate controlled remedy for target-label delay in production quality monitoring. True supervised production quality generally needs observed outcomes; prediction outputs alone are not ground truth. Verify the cause with: Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. Then confirm: Model quality updates only for labeled cohorts while latency and feature drift are monitored promptly. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
B:Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. This directly tests target-label delay in production quality monitoring in the reported AWS ML workflow. True supervised production quality generally needs observed outcomes; prediction outputs alone are not ground truth. It distinguishes the problem before choosing a change. The related targeted response is: Separate prompt input-distribution monitoring from delayed model-quality evaluation and join only observed labels to predictions. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Incorrect Answers
C:Inspect S3 object encryption, key policy, grant permissions, IAM role, and denied KMS decrypt events. This selection concerns s3 sse-kms cross-account model artifact access rather than the required diagnostic-and-remedy pair for target-label delay in production quality monitoring. Use evidence: Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. and the matching correction: Separate prompt input-distribution monitoring from delayed model-quality evaluation and join only observed labels to predictions. Signal: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
D:Monitor calibration against delayed truth and retrain or recalibrate on representative permitted data when drift is confirmed. This selection concerns serving output confidence calibration drift rather than the required diagnostic-and-remedy pair for target-label delay in production quality monitoring. Use evidence: Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. and the matching correction: Separate prompt input-distribution monitoring from delayed model-quality evaluation and join only observed labels to predictions. Signal: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
E:Pin an immutable approved artifact version or content digest and verify integrity before deployment. This selection concerns model artifact supply-chain integrity rather than the required diagnostic-and-remedy pair for target-label delay in production quality monitoring. Use evidence: Inspect inference timestamps, eventual ground-truth arrival, join keys, and the provenance of displayed quality metrics. and the matching correction: Separate prompt input-distribution monitoring from delayed model-quality evaluation and join only observed labels to predictions. Signal: A loan default label becomes available months after inference, but a dashboard falsely reports current prediction accuracy using unlabeled requests. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Question 368
During an operational load and release rehearsal for SageMaker-hosted inference, the following condition is observed: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Change control requires measurable endpoint or pipeline acceptance evidence. Which TWO complementary choices provide the decisive diagnosis AND corrective response? Select TWO.
- Inspect service quotas, minimum and maximum endpoint capacity, scaling cooldowns, startup time, and burst size.
- Grant the build role only required object and key operations for its specific artifact prefix and encryption key.
- Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage.
- Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts.
- Use a compatible SageMaker inference pipeline with ordered containers or a single packaged equivalent for shared preprocessing.
Explanation
Correct Answers
C:Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage. This directly tests sagemaker pipelines parameterized retraining in the reported AWS ML workflow. Pipelines make dependencies explicit; manually ordered notebook operations are difficult to repeat or audit. It distinguishes the problem before choosing a change. The related targeted response is: Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
D:Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts. This is the appropriate controlled remedy for sagemaker pipelines parameterized retraining. Pipelines make dependencies explicit; manually ordered notebook operations are difficult to repeat or audit. Verify the cause with: Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage. Then confirm: Repeated runs with the same inputs reproduce the ordered processing and generate auditable versioned outputs. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Incorrect Answers
A:Inspect service quotas, minimum and maximum endpoint capacity, scaling cooldowns, startup time, and burst size. This selection concerns endpoint scale-to-zero cold capacity planning rather than the required diagnostic-and-remedy pair for sagemaker pipelines parameterized retraining. Use evidence: Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage. and the matching correction: Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts. Signal: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
B:Grant the build role only required object and key operations for its specific artifact prefix and encryption key. This selection concerns codebuild iam least-privilege artifact upload rather than the required diagnostic-and-remedy pair for sagemaker pipelines parameterized retraining. Use evidence: Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage. and the matching correction: Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts. Signal: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
E:Use a compatible SageMaker inference pipeline with ordered containers or a single packaged equivalent for shared preprocessing. This selection concerns multi-container inference pipeline responsibilities rather than the required diagnostic-and-remedy pair for sagemaker pipelines parameterized retraining. Use evidence: Inspect the DAG of processing, training, evaluation and registration steps, along with parameter and artifact lineage. and the matching correction: Define a parameterized SageMaker Pipeline that orders steps and records source data, code and model artifacts. Signal: A retraining workflow copies notebook cells into a scheduled script without tracking data snapshots or processing dependencies, leading to inconsistent model artifacts. Operational setting: During an operational load and release rehearsal for SageMaker-hosted inference.
Question 369
During a controlled replay of the data-preparation workflow in a separate ML staging account, the following condition is observed: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. The dataset contract must remain reproducible for a backfill. Which measured outcome would confirm that the correction worked as intended?
- A held-out split uses the training-fitted median and independent evaluation no longer incorporates held-out distribution values.
- An equivalent recent-date query reads only its selected partitions while returning the same labeled rows.
- Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path.
- Identical inputs create identical token IDs in train and serving paths, including punctuation and unknown words.
Explanation
Correct Answer
D:Identical inputs create identical token IDs in train and serving paths, including punctuation and unknown words. This is the acceptance evidence for the described text tokenization mismatch in serving scenario. Model embeddings depend on stable token-ID semantics; swapping tokenizers changes the feature representation. The corrective implementation it validates is: Package the training tokenizer and normalization configuration with the model artifact and require serving compatibility checks. Reported signal: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Incorrect Answers
A:A held-out split uses the training-fitted median and independent evaluation no longer incorporates held-out distribution values. This could validate median imputation fitted on training split, but it does not establish that text tokenization mismatch in serving is resolved for this workload. Confirm instead: Identical inputs create identical token IDs in train and serving paths, including punctuation and unknown words. Observed problem: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
B:An equivalent recent-date query reads only its selected partitions while returning the same labeled rows. This could validate partition pruning in s3 parquet training input, but it does not establish that text tokenization mismatch in serving is resolved for this workload. Confirm instead: Identical inputs create identical token IDs in train and serving paths, including punctuation and unknown words. Observed problem: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
C:Both supported source versions produce the agreed typed feature, and invalid values enter an auditable reject path. This could validate glue schema evolution with mixed input types, but it does not establish that text tokenization mismatch in serving is resolved for this workload. Confirm instead: Identical inputs create identical token IDs in train and serving paths, including punctuation and unknown words. Observed problem: A sentiment model trained with one tokenizer receives text encoded by a different tokenizer in production and generates incorrect input token IDs. Operational setting: During a controlled replay of the data-preparation workflow in a separate ML staging account.
Question 370
While reproducing the alert in an isolated AWS ML integration account, the following condition is observed: A long training workflow uses only On-Demand capacity even though checkpoints and flexible completion time permit interruptions, and its compute cost breaches target. The proposed change must preserve logging and least privilege. Which TWO complementary choices provide the decisive diagnosis AND corrective response? Select TWO.
- Inspect routing hashes, experiment assignments, variant weights, and request identity consistency.
- Permit iam:PassRole only for the named execution role, constrained to the intended SageMaker service as appropriate.
- Inspect training interruption tolerance, persistent checkpoints, allowed instance families, and deadline requirements.
- Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it.
- Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family.
Explanation
Correct Answers
C:Inspect training interruption tolerance, persistent checkpoints, allowed instance families, and deadline requirements. This directly tests spot training cost versus interruption recovery in the reported AWS ML workflow. Spot savings depend on interruption handling; a lower hourly rate is not a guarantee of lower total job cost. It distinguishes the problem before choosing a change. The related targeted response is: Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A long training workflow uses only On-Demand capacity even though checkpoints and flexible completion time permit interruptions, and its compute cost breaches target. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
D:Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. This is the appropriate controlled remedy for spot training cost versus interruption recovery. Spot savings depend on interruption handling; a lower hourly rate is not a guarantee of lower total job cost. Verify the cause with: Inspect training interruption tolerance, persistent checkpoints, allowed instance families, and deadline requirements. Then confirm: Measured training cost drops while interrupted test jobs resume from checkpoints within the delivery window. This is one part of a select-two decision; the matching evidence and corrective action are both required. Reported signal: A long training workflow uses only On-Demand capacity even though checkpoints and flexible completion time permit interruptions, and its compute cost breaches target. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
Incorrect Answers
A:Inspect routing hashes, experiment assignments, variant weights, and request identity consistency. This selection concerns champion challenger a/b assignment consistency rather than the required diagnostic-and-remedy pair for spot training cost versus interruption recovery. Use evidence: Inspect training interruption tolerance, persistent checkpoints, allowed instance families, and deadline requirements. and the matching correction: Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. Signal: A long training workflow uses only On-Demand capacity even though checkpoints and flexible completion time permit interruptions, and its compute cost breaches target. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
B:Permit iam:PassRole only for the named execution role, constrained to the intended SageMaker service as appropriate. This selection concerns iam passrole constraint for model deployment rather than the required diagnostic-and-remedy pair for spot training cost versus interruption recovery. Use evidence: Inspect training interruption tolerance, persistent checkpoints, allowed instance families, and deadline requirements. and the matching correction: Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. Signal: A long training workflow uses only On-Demand capacity even though checkpoints and flexible completion time permit interruptions, and its compute cost breaches target. Operational setting: While reproducing the alert in an isolated AWS ML integration account.
E:Use SageMaker Inference Recommender or equivalent controlled load tests to rightsize the serving instance family. This selection concerns inference recommender benchmarking instance family rather than the required diagnostic-and-remedy pair for spot training cost versus interruption recovery. Use evidence: Inspect training interruption tolerance, persistent checkpoints, allowed instance families, and deadline requirements. and the matching correction: Use supported managed Spot Training with durable checkpoints for interruptible phases and retain On-Demand capacity when deadlines demand it. Signal: A long training workflow uses only On-Demand capacity even though checkpoints and flexible completion time permit interruptions, and its compute cost breaches target. Operational setting: While reproducing the alert in an isolated AWS ML integration account.