Microsoft AI-300 Practice Test 47
Question 460
While optimizing a production RAG or fine-tuned model service, the following condition occurs: A product support team plans expensive fine-tuning to memorize weekly price lists, even though the answers require current frequently edited catalog values. The resulting model must meet held-out quality and security criteria. Which targeted Azure AI operations change best addresses the documented problem?
- Generate sanitized synthetic patterns from approved nonidentifying templates and screen outputs for memorized or copied private content.
- Tune chunk size and overlap with structure-aware splitting so qualifying rules and exceptions remain retrievable together.
- Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation.
- Retain governed immutable fine-tune artifacts and record deployment lineage with tested restore/rollback procedures.
Explanation
Correct Answer
C:Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation. This targets fine-tuning task selected when retrieval would better handle changing facts. Fine-tuning changes model behavior and knowledge imperfectly; retrieval is generally more suitable for fast-changing external facts. Verify the failure through: Inspect information update frequency, retrieval freshness requirement, model task errors, training cost and data-change workflow. Validate success with: A changed product price appears in answers after authorized index refresh without retraining the entire model.
Incorrect Answers
A:Generate sanitized synthetic patterns from approved nonidentifying templates and screen outputs for memorized or copied private content. This could address synthetic training examples copy private customer identifiers, but it does not solve the specified fine-tuning task selected when retrieval would better handle changing facts failure. The matching remediation is: Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation.
B:Tune chunk size and overlap with structure-aware splitting so qualifying rules and exceptions remain retrievable together. This could address rag retrieval chunks truncate critical policy exception, but it does not solve the specified fine-tuning task selected when retrieval would better handle changing facts failure. The matching remediation is: Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation.
D:Retain governed immutable fine-tune artifacts and record deployment lineage with tested restore/rollback procedures. This could address fine-tune production endpoint references deleted training artifact, but it does not solve the specified fine-tuning task selected when retrieval would better handle changing facts failure. The matching remediation is: Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation.
Question 461
As an MLOps engineer reviews a model training or inference pipeline, the following condition occurs: A deployed churn model receives increasing nulls in a new CRM field but its data drift monitor tracks only average values and ignores missingness and schema validity. The resolution must preserve quality gates, documented artifacts and rollback options. Which investigation best verifies the cause before changing production configuration?
- Inspect model input data quality signals, null fractions, expected feature schema, training baseline and monitoring sampling.
- Review job output mappings, script save path, managed output directories, job artifacts and downstream register-component input.
- Inspect model retraining pipeline graph, evaluation outputs, baseline comparison, deployment step conditions and release records.
- Inspect reference data time window, drift feature metrics, alert threshold, seasonal segmentation and actual model outcomes.
Explanation
Correct Answer
A:Inspect model input data quality signals, null fractions, expected feature schema, training baseline and monitoring sampling. This is the most relevant evidence for missing feature values escape input drift checks. Data quality and data distribution drift are related but distinct signals; averages alone can conceal feature loss. The matching controlled remediation is: Add suitable data quality and schema drift signals with thresholds for missingness alongside distribution drift. This matters here because as an MLOps engineer reviews a model training or inference pipeline, the following condition occurs: A deployed….
Incorrect Answers
B:Review job output mappings, script save path, managed output directories, job artifacts and downstream register-component input. This would investigate training job does not persist output artifacts rather than the documented missing feature values escape input drift checks issue. The relevant investigation is: Inspect model input data quality signals, null fractions, expected feature schema, training baseline and monitoring sampling.
C:Inspect model retraining pipeline graph, evaluation outputs, baseline comparison, deployment step conditions and release records. This would investigate retraining pipeline deploys without model quality gate rather than the documented missing feature values escape input drift checks issue. The relevant investigation is: Inspect model input data quality signals, null fractions, expected feature schema, training baseline and monitoring sampling.
D:Inspect reference data time window, drift feature metrics, alert threshold, seasonal segmentation and actual model outcomes. This would investigate data drift monitor uses wrong reference season rather than the documented missing feature values escape input drift checks issue. The relevant investigation is: Inspect model input data quality signals, null fractions, expected feature schema, training baseline and monitoring sampling.
Question 462
As an MLOps engineer reviews a model training or inference pipeline, the following condition occurs: An interactive fraud-screening API must respond in under 200 milliseconds, but engineers deploy an Azure ML batch endpoint requiring asynchronous data job submission. The resolution must preserve quality gates, documented artifacts and rollback options. Which TWO complementary choices identify the issue and correct it? Select TWO.
- Inspect request payload schema, scoring script init/run implementation, model input signature, container logs and test invocation.
- Allow the narrowly scoped supported compute-to-compute communication paths and correct the distributed training rendezvous configuration.
- Compare application latency SLA, current endpoint type, request/response behavior, compute resources and intended inference pattern.
- Deploy the approved model through a suitable managed online endpoint and provision capacity for synchronous requests.
- Apply a controlled small traffic allocation to the candidate, monitor quality and latency, then gradually increase only after passing gates.
Explanation
Correct Answers
C:Compare application latency SLA, current endpoint type, request/response behavior, compute resources and intended inference pattern. This is the most relevant evidence for online scoring requirement deployed as batch endpoint. Azure ML online endpoints support synchronous prediction while batch endpoints are designed for asynchronous high-volume jobs. The matching controlled remediation is: Deploy the approved model through a suitable managed online endpoint and provision capacity for synchronous requests.
D:Deploy the approved model through a suitable managed online endpoint and provision capacity for synchronous requests. This targets online scoring requirement deployed as batch endpoint. Azure ML online endpoints support synchronous prediction while batch endpoints are designed for asynchronous high-volume jobs. Verify the failure through: Compare application latency SLA, current endpoint type, request/response behavior, compute resources and intended inference pattern. Validate success with: A controlled API load test meets latency and correctness targets without requiring clients to poll batch jobs.
Incorrect Answers
A:Inspect request payload schema, scoring script init/run implementation, model input signature, container logs and test invocation. This is associated with endpoint scoring script expects wrong json schema, not the essential evidence-and-fix pair for online scoring requirement deployed as batch endpoint. First examine: Compare application latency SLA, current endpoint type, request/response behavior, compute resources and intended inference pattern. Then correct: Deploy the approved model through a suitable managed online endpoint and provision capacity for synchronous requests.
B:Allow the narrowly scoped supported compute-to-compute communication paths and correct the distributed training rendezvous configuration. This is associated with distributed job workers cannot communicate over required network path, not the essential evidence-and-fix pair for online scoring requirement deployed as batch endpoint. First examine: Compare application latency SLA, current endpoint type, request/response behavior, compute resources and intended inference pattern. Then correct: Deploy the approved model through a suitable managed online endpoint and provision capacity for synchronous requests.
E:Apply a controlled small traffic allocation to the candidate, monitor quality and latency, then gradually increase only after passing gates. This is associated with canary rollout sends all traffic to unvalidated candidate, not the essential evidence-and-fix pair for online scoring requirement deployed as batch endpoint. First examine: Compare application latency SLA, current endpoint type, request/response behavior, compute resources and intended inference pattern. Then correct: Deploy the approved model through a suitable managed online endpoint and provision capacity for synchronous requests.
Question 463
As an MLOps engineer reviews a model training or inference pipeline, the following condition occurs: A churn model is registered and deployed, but online inference uses different feature lookup keys from training because the model artifact lacks the feature retrieval specification. The resolution must preserve quality gates, documented artifacts and rollback options. Select TWO: the strongest initial evidence and the aligned controlled correction.
- Archive or retire the version through a governed registry lifecycle and remove it from production deployment aliases after dependents migrate.
- Inspect registered model metadata, online feature lookup mapping, feature timestamps, key names and schema contract from training.
- Inspect model version IDs, labels/aliases, approval evidence, endpoint deployment configuration and registry audit events.
- Remove target-leakage columns and rerun the AutoML search with an appropriate time-aware validation split and realistic available features.
- Package and version the approved feature retrieval specification with the model and enforce the same mapping at serving time.
Explanation
Correct Answers
B:Inspect registered model metadata, online feature lookup mapping, feature timestamps, key names and schema contract from training. This is the most relevant evidence for feature retrieval specification missing from registered model. Model weights alone do not specify how feature values were retrieved; production consistency requires a versioned feature contract. The matching controlled remediation is: Package and version the approved feature retrieval specification with the model and enforce the same mapping at serving time.
E:Package and version the approved feature retrieval specification with the model and enforce the same mapping at serving time. This targets feature retrieval specification missing from registered model. Model weights alone do not specify how feature values were retrieved; production consistency requires a versioned feature contract. Verify the failure through: Inspect registered model metadata, online feature lookup mapping, feature timestamps, key names and schema contract from training. Validate success with: Offline validation and online scoring use equivalent feature definitions and identical customer keys without silent missing-value substitution.
Incorrect Answers
A:Archive or retire the version through a governed registry lifecycle and remove it from production deployment aliases after dependents migrate. This is associated with retired model kept active as discoverable deployment candidate, not the essential evidence-and-fix pair for feature retrieval specification missing from registered model. First examine: Inspect registered model metadata, online feature lookup mapping, feature timestamps, key names and schema contract from training. Then correct: Package and version the approved feature retrieval specification with the model and enforce the same mapping at serving time.
C:Inspect model version IDs, labels/aliases, approval evidence, endpoint deployment configuration and registry audit events. This is associated with model registry latest alias points to unapproved experiment, not the essential evidence-and-fix pair for feature retrieval specification missing from registered model. First examine: Inspect registered model metadata, online feature lookup mapping, feature timestamps, key names and schema contract from training. Then correct: Package and version the approved feature retrieval specification with the model and enforce the same mapping at serving time.
D:Remove target-leakage columns and rerun the AutoML search with an appropriate time-aware validation split and realistic available features. This is associated with automated ml experiment uses leakage feature, not the essential evidence-and-fix pair for feature retrieval specification missing from registered model. First examine: Inspect registered model metadata, online feature lookup mapping, feature timestamps, key names and schema contract from training. Then correct: Package and version the approved feature retrieval specification with the model and enforce the same mapping at serving time.
Question 464
As an AI operations team checks production agent evaluation evidence, the following condition occurs: A user-facing agent becomes slow after an LLM deployment change, but the team only records total response time and cannot tell retrieval delay from model generation delay. The workflow must produce measurable quality, safety and performance evidence. What approved correction resolves the failure without modifying unrelated healthy services?
- Mask sensitive tokens and personal data, apply least-privilege log access and retain only necessary structured debugging evidence.
- Apply privacy-conscious representative sampling or stratified monitoring that includes error and underrepresented user cohorts.
- Include appropriate complete conversation histories and tool events in the supported agent evaluation workflow while preserving privacy protections.
- Instrument stage-level latency and compare generation and retrieval spans before applying a targeted capacity or retrieval change.
Explanation
Correct Answer
D:Instrument stage-level latency and compare generation and retrieval spans before applying a targeted capacity or retrieval change. This targets generation latency spike blamed on retrieval without breakdown. Latency budgets should separate agent orchestration, retrieval and generation; total duration alone cannot locate the bottleneck. Verify the failure through: Examine end-to-end request traces, retrieval timing, model time to first token, generation duration, request concurrency and percentiles. Validate success with: The responsible stage is identified and P95 user response time improves without gratuitous changes to healthy components.
Incorrect Answers
A:Mask sensitive tokens and personal data, apply least-privilege log access and retain only necessary structured debugging evidence. This could address debug traces leak secrets and retrieved personal records, but it does not solve the specified generation latency spike blamed on retrieval without breakdown failure. The matching remediation is: Instrument stage-level latency and compare generation and retrieval spans before applying a targeted capacity or retrieval change.
B:Apply privacy-conscious representative sampling or stratified monitoring that includes error and underrepresented user cohorts. This could address continuous monitoring has no representative sampling coverage, but it does not solve the specified generation latency spike blamed on retrieval without breakdown failure. The matching remediation is: Instrument stage-level latency and compare generation and retrieval spans before applying a targeted capacity or retrieval change.
C:Include appropriate complete conversation histories and tool events in the supported agent evaluation workflow while preserving privacy protections. This could address multi-turn agent evaluation uses isolated last message only, but it does not solve the specified generation latency spike blamed on retrieval without breakdown failure. The matching remediation is: Instrument stage-level latency and compare generation and retrieval spans before applying a targeted capacity or retrieval change.
Question 465
During a regulated Azure ML workspace infrastructure review, the following condition occurs: A distributed deep-learning job remains queued because the Azure ML compute cluster requests more regional GPU-family vCPU quota than the subscription currently has. The team must preserve approved access, model provenance and private network constraints. Which observable postchange result proves the intended remediation succeeded?
- The job reads the required training blobs and cannot write or list unrelated protected containers.
- The training job acquires nodes inside documented quota and starts without creating unapproved capacity or dropping the distributed requirement.
- A scoped job retrieves only the approved secret and unrelated project secrets remain inaccessible.
- An incident analyst can reproduce the model version's code and dependencies from its recorded source commit.
Explanation
Correct Answer
B:The training job acquires nodes inside documented quota and starts without creating unapproved capacity or dropping the distributed requirement. Azure ML computes draw from Azure regional VM family quota; declaring a cluster does not reserve unlimited GPU instances. a direct corrective action is to request approved additional quota or rescope the job to a permitted compute size and region while retaining necessary GPU capability.
Incorrect Answers
A:The job reads the required training blobs and cannot write or list unrelated protected containers. That addresses a different configuration issue rather than the main requirement. Verify instead: The training job acquires nodes inside documented quota and starts without creating unapproved capacity or dropping the distributed requirement.
C:A scoped job retrieves only the approved secret and unrelated project secrets remain inaccessible. That addresses a different configuration issue rather than the main requirement. Verify instead: The training job acquires nodes inside documented quota and starts without creating unapproved capacity or dropping the distributed requirement.
D:An incident analyst can reproduce the model version's code and dependencies from its recorded source commit. That addresses a different configuration issue rather than the main requirement. Verify instead: The training job acquires nodes inside documented quota and starts without creating unapproved capacity or dropping the distributed requirement.
Question 466
While preparing a secure Foundry model or agent release, the following condition occurs: A production foundation model deployment is reachable from all development subnets after network security rules are copied from a permissive sandbox environment. The remediation must meet enterprise access and reliability requirements without a broad bypass. Which diagnostic evidence most directly distinguishes the described AI operations failure?
- Audit Foundry network ACLs, private endpoint connections, VNet peerings, firewall rules and effective caller identities.
- Examine model package runtime requirements, container startup traceback, tokenizer/config files and managed compute environment image digest.
- Inspect workflow identity claims, login failure logs, secret references, federated identity options and target resource permissions.
- Inspect Bicep template resource references, model deployment name outputs, CI step dependencies and Azure deployment operations.
Explanation
Correct Answer
A:Audit Foundry network ACLs, private endpoint connections, VNet peerings, firewall rules and effective caller identities. This is the most relevant evidence for network allowlist exposes production model endpoint to test subnet. Network segmentation and API authorization are complementary; an allowed network path should still follow environment separation policies. The matching controlled remediation is: Apply least-privilege subnet and identity restrictions to production model access and remove inherited broad sandbox exemptions.
Incorrect Answers
B:Examine model package runtime requirements, container startup traceback, tokenizer/config files and managed compute environment image digest. This would investigate managed foundation model deployment lacks compatible runtime image rather than the documented network allowlist exposes production model endpoint to test subnet issue. The relevant investigation is: Audit Foundry network ACLs, private endpoint connections, VNet peerings, firewall rules and effective caller identities.
C:Inspect workflow identity claims, login failure logs, secret references, federated identity options and target resource permissions. This would investigate foundry service principal credential rotated without updating pipeline rather than the documented network allowlist exposes production model endpoint to test subnet issue. The relevant investigation is: Audit Foundry network ACLs, private endpoint connections, VNet peerings, firewall rules and effective caller identities.
D:Inspect Bicep template resource references, model deployment name outputs, CI step dependencies and Azure deployment operations. This would investigate foundry bicep rollout omits model deployment dependencies rather than the documented network allowlist exposes production model endpoint to test subnet issue. The relevant investigation is: Audit Foundry network ACLs, private endpoint connections, VNet peerings, firewall rules and effective caller identities.
Question 467
While deploying reproducible Azure Machine Learning infrastructure, the following condition occurs: A GitHub federated identity credential accepts tokens from any branch, allowing an unreviewed pull request to reach Azure Machine Learning production resources. The change must remain limited to authorized workspaces and avoid unreviewed production impact. Select TWO actions: one decisive diagnostic and its corresponding targeted remediation.
- Request approved additional quota or rescope the job to a permitted compute size and region while retaining necessary GPU capability.
- Bind every training job and model registration to a reviewed Git commit SHA and immutable environment/data version in the workflow.
- Restrict the federated identity subject to the protected deployment environment or branch and enforce repository approval rules.
- Review federated credential issuer, audience and subject, GitHub environment protection rules, branch patterns and run provenance.
- Inspect workflow trigger, environment configuration, protected branch policies, required reviewers and current deployment audit records.
Explanation
Correct Answers
C:Restrict the federated identity subject to the protected deployment environment or branch and enforce repository approval rules. This targets oidc subject filter trusts unreviewed pull-request branches. Federated authentication remains vulnerable to overbroad trust claims if token subject matching does not reflect deployment governance. Verify the failure through: Review federated credential issuer, audience and subject, GitHub environment protection rules, branch patterns and run provenance. Validate success with: Only authorized protected workflows obtain production Azure tokens while unreviewed pull requests fail the access test. This matters here because while deploying reproducible Azure Machine Learning infrastructure, the following condition occurs: A GitHub….
D:Review federated credential issuer, audience and subject, GitHub environment protection rules, branch patterns and run provenance. This is the most relevant evidence for oidc subject filter trusts unreviewed pull-request branches. Federated authentication remains vulnerable to overbroad trust claims if token subject matching does not reflect deployment governance. The matching controlled remediation is: Restrict the federated identity subject to the protected deployment environment or branch and enforce repository approval rules. This matters here because while deploying reproducible Azure Machine Learning infrastructure, the following condition occurs: A GitHub….
Incorrect Answers
A:Request approved additional quota or rescope the job to a permitted compute size and region while retaining necessary GPU capability. This is associated with compute cluster cannot allocate requested gpu quota, not the essential evidence-and-fix pair for oidc subject filter trusts unreviewed pull-request branches. First examine: Review federated credential issuer, audience and subject, GitHub environment protection rules, branch patterns and run provenance. Then correct: Restrict the federated identity subject to the protected deployment environment or branch and enforce repository approval rules.
B:Bind every training job and model registration to a reviewed Git commit SHA and immutable environment/data version in the workflow. This is associated with production training script commit cannot be traced, not the essential evidence-and-fix pair for oidc subject filter trusts unreviewed pull-request branches. First examine: Review federated credential issuer, audience and subject, GitHub environment protection rules, branch patterns and run provenance. Then correct: Restrict the federated identity subject to the protected deployment environment or branch and enforce repository approval rules. This matters here because while deploying reproducible Azure Machine Learning infrastructure, the following condition occurs: A GitHub….
E:Inspect workflow trigger, environment configuration, protected branch policies, required reviewers and current deployment audit records. This is associated with github deployment pipeline omits secure environment approvals, not the essential evidence-and-fix pair for oidc subject filter trusts unreviewed pull-request branches. First examine: Review federated credential issuer, audience and subject, GitHub environment protection rules, branch patterns and run provenance. Then correct: Restrict the federated identity subject to the protected deployment environment or branch and enforce repository approval rules.
Question 468
During a Microsoft Foundry generative AI deployment review, the following condition occurs: A team claims prompt B is better than prompt A, but B runs on a newer foundation model so the measured improvement cannot be attributed to the prompt change. The deployment must retain governed identities, endpoint contracts and auditable release history. Select TWO: the strongest initial evidence and the aligned controlled correction.
- Separate trusted instructions from quoted retrieved context and enforce restricted tool permissions and injection-resilient prompt handling.
- Compare experiment assignments, model deployment versions, test inputs, sample size, evaluator configuration and prompt hashes.
- Separate governed development and production environments with distinct project configuration, least-privilege credentials and deployment gates.
- Evaluate both prompt variants against the same pinned model and representative dataset with controlled assignment and consistent metrics.
- Inspect the prompt template, target JSON schema, model output examples, parser exceptions and current evaluation dataset.
Explanation
Correct Answers
B:Compare experiment assignments, model deployment versions, test inputs, sample size, evaluator configuration and prompt hashes. This is the most relevant evidence for prompt a/b test uses different model versions in each arm. Experimental prompt comparisons require controlling confounders; a simultaneous model upgrade prevents a valid prompt-only attribution. The matching controlled remediation is: Evaluate both prompt variants against the same pinned model and representative dataset with controlled assignment and consistent metrics.
D:Evaluate both prompt variants against the same pinned model and representative dataset with controlled assignment and consistent metrics. This targets prompt a/b test uses different model versions in each arm. Experimental prompt comparisons require controlling confounders; a simultaneous model upgrade prevents a valid prompt-only attribution. Verify the failure through: Compare experiment assignments, model deployment versions, test inputs, sample size, evaluator configuration and prompt hashes. Validate success with: The comparison isolates prompt differences and produces reproducible evidence of quality and latency trade-offs.
Incorrect Answers
A:Separate trusted instructions from quoted retrieved context and enforce restricted tool permissions and injection-resilient prompt handling. This is associated with prompt includes untrusted retrieved instructions as system policy, not the essential evidence-and-fix pair for prompt a/b test uses different model versions in each arm. First examine: Compare experiment assignments, model deployment versions, test inputs, sample size, evaluator configuration and prompt hashes. Then correct: Evaluate both prompt variants against the same pinned model and representative dataset with controlled assignment and consistent metrics.
C:Separate governed development and production environments with distinct project configuration, least-privilege credentials and deployment gates. This is associated with foundry environment mixes development and production agent resources, not the essential evidence-and-fix pair for prompt a/b test uses different model versions in each arm. First examine: Compare experiment assignments, model deployment versions, test inputs, sample size, evaluator configuration and prompt hashes. Then correct: Evaluate both prompt variants against the same pinned model and representative dataset with controlled assignment and consistent metrics.
E:Inspect the prompt template, target JSON schema, model output examples, parser exceptions and current evaluation dataset. This is associated with prompt omits explicit structured answer schema, not the essential evidence-and-fix pair for prompt a/b test uses different model versions in each arm. First examine: Compare experiment assignments, model deployment versions, test inputs, sample size, evaluator configuration and prompt hashes. Then correct: Evaluate both prompt variants against the same pinned model and representative dataset with controlled assignment and consistent metrics.
Question 469
During a production Azure Machine Learning model lifecycle incident, the following condition occurs: A fraud-screening AutoML run optimizes overall accuracy on a highly imbalanced dataset and selects a model that misses most fraud cases. The operator must safeguard training reproducibility and serving continuity. What approved correction resolves the failure without modifying unrelated healthy services?
- Implement feasible deterministic seeding and record input data split and environment versions for each run with acknowledged nondeterministic GPU constraints.
- Configure a relevant supported primary metric and validation scheme for rare positive events and compare operating thresholds after training.
- Configure a suitable seasonal baseline or context-aware drift monitor while retaining meaningful anomaly thresholds.
- Evaluate candidate and baseline on the same governed holdout dataset using the same business-aligned metrics and thresholds.
Explanation
Correct Answer
B:Configure a relevant supported primary metric and validation scheme for rare positive events and compare operating thresholds after training. This targets automl primary metric does not match rare-event goal. Accuracy can look high when predicting the majority class; model selection metrics should reflect the operational target and class imbalance. Verify the failure through: Inspect class prevalence, AutoML primary metric, confusion matrix, recall and precision of candidate runs and business error costs. Validate success with: Selected candidates improve the approved fraud-detection target while documented false-positive trade-offs remain acceptable.
Incorrect Answers
A:Implement feasible deterministic seeding and record input data split and environment versions for each run with acknowledged nondeterministic GPU constraints. This could address training script ignores reproducible seed and input contract, but it does not solve the specified automl primary metric does not match rare-event goal failure. The matching remediation is: Configure a relevant supported primary metric and validation scheme for rare positive events and compare operating thresholds after training.
C:Configure a suitable seasonal baseline or context-aware drift monitor while retaining meaningful anomaly thresholds. This could address data drift monitor uses wrong reference season, but it does not solve the specified automl primary metric does not match rare-event goal failure. The matching remediation is: Configure a relevant supported primary metric and validation scheme for rare positive events and compare operating thresholds after training.
D:Evaluate candidate and baseline on the same governed holdout dataset using the same business-aligned metrics and thresholds. This could address run comparison uses incompatible validation populations, but it does not solve the specified automl primary metric does not match rare-event goal failure. The matching remediation is: Configure a relevant supported primary metric and validation scheme for rare positive events and compare operating thresholds after training.
Question 470
During a retrieval or model-customization performance review, the following condition occurs: A product support team plans expensive fine-tuning to memorize weekly price lists, even though the answers require current frequently edited catalog values. The change must keep citations, privacy and previously approved behavior intact. Which TWO complementary choices identify the issue and correct it? Select TWO.
- Halt promotion, review training data and fine-tuning choices, and require safety regression gates on the customized candidate before release.
- Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation.
- Evaluate both retrievers with relevance, factual support and business outcomes under stable randomized cohorts and explicit guardrails.
- Review live prompt distribution, new product identifier coverage, task quality metrics, fine-tuned model version and evaluation refresh cadence.
- Inspect information update frequency, retrieval freshness requirement, model task errors, training cost and data-change workflow.
Explanation
Correct Answers
B:Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation. This targets fine-tuning task selected when retrieval would better handle changing facts. Fine-tuning changes model behavior and knowledge imperfectly; retrieval is generally more suitable for fast-changing external facts. Verify the failure through: Inspect information update frequency, retrieval freshness requirement, model task errors, training cost and data-change workflow. Validate success with: A changed product price appears in answers after authorized index refresh without retraining the entire model. This matters here because a product support team plans expensive fine-tuning to memorize weekly price lists, even though the answers require….
E:Inspect information update frequency, retrieval freshness requirement, model task errors, training cost and data-change workflow. This is the most relevant evidence for fine-tuning task selected when retrieval would better handle changing facts. Fine-tuning changes model behavior and knowledge imperfectly; retrieval is generally more suitable for fast-changing external facts. The matching controlled remediation is: Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation. This matters here because a product support team plans expensive fine-tuning to memorize weekly price lists, even though the answers require….
Incorrect Answers
A:Halt promotion, review training data and fine-tuning choices, and require safety regression gates on the customized candidate before release. This is associated with fine-tuned model loses safety boundary after customization, not the essential evidence-and-fix pair for fine-tuning task selected when retrieval would better handle changing facts. First examine: Inspect information update frequency, retrieval freshness requirement, model task errors, training cost and data-change workflow. Then correct: Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation. This matters here because a product support team plans expensive fine-tuning to memorize weekly price lists, even though the answers require….
C:Evaluate both retrievers with relevance, factual support and business outcomes under stable randomized cohorts and explicit guardrails. This is associated with rag a/b experiment judges only click-through not factuality, not the essential evidence-and-fix pair for fine-tuning task selected when retrieval would better handle changing facts. First examine: Inspect information update frequency, retrieval freshness requirement, model task errors, training cost and data-change workflow. Then correct: Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation.
D:Review live prompt distribution, new product identifier coverage, task quality metrics, fine-tuned model version and evaluation refresh cadence. This is associated with fine-tune evaluation lacks drift checks for new customer terminology, not the essential evidence-and-fix pair for fine-tuning task selected when retrieval would better handle changing facts. First examine: Inspect information update frequency, retrieval freshness requirement, model task errors, training cost and data-change workflow. Then correct: Use a governed retrieval-augmented grounding path for volatile factual content and reserve fine-tuning for stable behavior or output-format adaptation.