Question 130
A customer service model gives reliable advice in one language but fails with less common local terms. What evidence should the team examine FIRST before making a decision?
Explanation
Correct Answer
Inspect dataset representation, disaggregated performance, adverse outcomes, reviewer criteria and monitored drift. These observations test the stated cause before the team changes a system or policy. Bias can originate in training data or evaluation design; a high overall accuracy score can conceal subgroup failures. The most suitable approach is: Evaluate outputs across representative user groups and rare edge cases, and introduce mitigation and human oversight where unequal error rates matter. Scenario: A customer service model gives reliable advice in one language but fails with less common local terms.
Incorrect Answers
Inspect supported agent tools, delegated identities, workflow boundaries, rollback and deployment test results. These observations are relevant to Agent Platform builder and custom agent orchestration, but do not isolate the main question. The decisive evidence is: Inspect dataset representation, disaggregated performance, adverse outcomes, reviewer criteria and monitored drift. Scenario: A customer service model gives reliable advice in one language but fails with less common local terms.
Inspect required task order, stage input-output contracts, evidence lineage, exception handling and retry budgets. These observations are relevant to Prompt chaining and staged transformations, but do not isolate the main question. The decisive evidence is: Inspect dataset representation, disaggregated performance, adverse outcomes, reviewer criteria and monitored drift. Scenario: A customer service model gives reliable advice in one language but fails with less common local terms.
Inspect evaluation baselines, model/prompt versions, sampling, error rates, cost metrics, alerts and release changes. These observations are relevant to Model monitoring versioning and quality KPIs, but do not isolate the main question. The decisive evidence is: Inspect dataset representation, disaggregated performance, adverse outcomes, reviewer criteria and monitored drift. Scenario: A customer service model gives reliable advice in one language but fails with less common local terms.


