Practice Exams:

Microsoft AI-103: Evaluation Datasets That Reveal Failure

AI & Machine Learning

An AI evaluation report can show a reassuring score even when the system repeatedly fails the situations that matter to users. An agent that handles straightforward questions may still expose a confidential document, execute the wrong tool, or miss an exception buried in a policy. The problem is often the dataset rather than the evaluator: convenient examples reward predictable demonstrations and ignore real operational variation. Microsoft Foundry supports structured evaluation datasets, but engineers must decide which behaviors constitute success, curate cases independently from development, and preserve enough source evidence to explain disagreements. A reliable dataset is a living test contract, not a collection of favorite prompts.

On this page
  1. Define the task and its acceptable outcomes
  2. Build cases from real task variation
  3. Separate development, regression, and release evidence
  4. Represent conversations and tools faithfully
  5. Match evaluators to failure modes
  6. Include rights, safety, and provenance in the labels
  7. Use errors to direct engineering, not just rank models

Define the task and its acceptable outcomes

An evaluation dataset for a Microsoft AI-103 project should begin with the decision an application makes and the observable evidence a reviewer can test. A grounded assistant may need to identify an applicable policy, cite its effective version, and abstain when evidence is missing. A tool-using agent may need to choose a permitted action, pass arguments that satisfy a schema, request confirmation when authority is unclear, and leave state unchanged on refusal. Those criteria differ from subjective fluency. Tie each case to a failure consequence so evaluation focuses on user value instead of whichever score is easiest to calculate.

Define more than one valid answer where appropriate. Two summaries may convey the same correct limitation with different wording, while a structured transaction can have only one allowed recipient or account. A rigid single reference string creates false failures for good responses and false passes for memorized text. Record the expected facts, relevant source versions, prohibited actions, and required evidence; let the evaluator distinguish essential correctness from acceptable natural variation.

Build cases from real task variation

Sample real workflows across user roles, document types, input sizes, languages, and common error modes after applying privacy restrictions. Include ordinary requests, ambiguous instructions, contradictory source documents, unsupported questions, outdated versions, and deliberate malicious or misleading inputs. A balanced dataset need not contain equal numbers of every category, but it should show enough rare failures to support decisions about the risk they pose. Keep metadata for the category and severity of each example so a single average cannot conceal a weak subgroup.

Imagine evaluating a support agent that answers contract questions and can open a ticket. A strong test record should include the authorized user’s role, the exact contract revision, the question, passages the caller may inspect, the action requested and the reason a particular outcome is acceptable. A case where the source document is missing is not a failed retrieval that should be rescued by guesswork: the expected action is to report the evidence gap or request a permitted source. A case where the right source is present but the answer ignores a termination exception is a different failure, even when the final response sounds plausible. Keep those error classes separate so engineering knows whether to adjust index freshness, retrieval ranking, prompt instructions or the tool workflow.

For a ticket-creation task, annotate not only the conversational answer but the resulting record. The agent may say that it created a ticket without the API actually succeeding, or it may retry after a timeout and create duplicates. Use test doubles or an isolated test environment for actions, assign a stable operation identifier and verify persisted state before marking the scenario successful. Include disallowed requests to prove the agent respects tenant and user boundaries. Record how often a reviewer must dispute an automatic grading result; disagreements are evidence that the evaluator rubric itself needs revision. A dataset is useful when it exposes both the system’s defect and the limits of the measurement.

Document who selected each case. Engineers often curate examples from known successes because they are easy to review; that can bias the set before any model runs. Seek support escalations, incident reports, failed searches, repeated user clarifications, and authorized expert corrections. Turn those into representative and reviewable cases without copying private chat logs indiscriminately. Include input conditions needed to reproduce a failure, such as the source corpus version and the user authorization scope at the time.

Separate development, regression, and release evidence

Maintain distinct working and held-out sets. The development set helps teams improve prompts and retrieval, but repeated tuning against it makes scores optimistic. A stable regression set catches reintroduced defects, while an independently maintained release set tests whether a candidate generalizes beyond the examples engineers repeatedly inspected. If production behavior changes, add a new versioned test partition rather than silently rewriting old labels and making historical comparisons uninterpretable.

Avoid leakage between cases and reference sources. If a model’s training or retrieval content contains the exact evaluation answer, the test may measure memorization or easy lookup instead of task reasoning. For retrieval applications, that may be intentional in a grounding test, but the evaluator must then check whether the correct source was retrieved under the proper authorization and whether the answer represents it accurately. Distinguish expected retrieval from improper answer contamination explicitly.

Represent conversations and tools faithfully

A single-turn query is not enough for many agents. Multi-turn cases should preserve clarifications, state changes, previous tool responses, and user corrections. For an action agent, include tool definitions, allowed permission context, expected arguments, and any simulated external state. Evaluate what the agent attempted as well as what it said. A polite final response cannot excuse a prohibited API call, and a harmless-looking tool call may succeed only because a test environment was overly permissive.

Microsoft Foundry evaluation schemas support message-oriented records or separate query and response fields, with evaluator-specific columns such as context, ground truth, or tool definitions. Choose a representation consistent with the evaluation scenario rather than inventing column names from memory. For stored traces, capture references that can be resolved during a governed review without placing secrets inside a broadly shared dataset. A test case should be executable by a reviewer who did not write the agent.

Match evaluators to failure modes

Use deterministic checks for schema shape, required fields, allowed action names, citation presence, and forbidden data exposure. Use human judgment or carefully calibrated model-based evaluation for nuanced quality, such as whether an answer correctly synthesizes contradictory documents. Record the evaluator’s version, rubrics, thresholds, sampling settings, and limitations. Some metrics look numerical while depending heavily on a subjective rubric or another model’s behavior; report those as estimates rather than facts about universal correctness.

Spot-check evaluator decisions against independently labeled examples. When a judge disagrees with reviewers, investigate whether the case is ambiguous, the instructions are weak, or the judge prefers a style over factual substance. Maintain examples of false positives and false negatives for each evaluator. An evaluation program should be able to explain why an excellent system was blocked or a weak candidate passed without blaming unexplained score noise.

Include rights, safety, and provenance in the labels

For RAG, annotate which sources a user may see, what is current, and which contradictory material should be excluded. A result that accurately quotes another tenant’s confidential document is still a severe failure. For agents, label allowed and forbidden side effects and situations that require an explicit human approval. The dataset should reward correct refusal, abstention, or escalation where appropriate; otherwise the model may be incentivized to sound certain and helpful when it should stop.

Treat labels as auditable decisions. A review rubric may change after a new policy is approved, so store its effective date and the reviewer’s rationale. Avoid embedding access tokens, personal identifiers, or full sensitive records where derived test cases would suffice. Separate the right to run the tests from the right to inspect restricted ground truth. Security and responsible AI controls must be present in the dataset process, not added as a final scoring column after content is collected.

Use errors to direct engineering, not just rank models

Break down failures by stage: retrieval missed evidence, the model ignored a retrieved passage, a tool argument violated policy, a downstream service timed out, or the final answer exceeded an output contract. Each suggests a different fix. Better chunking or access filters may beat retraining, and a deterministic validation step may beat another prompt revision. Include a small set of labeled traces in the release report so a team can see the cause and estimate the cost of repair.

Continue measuring after release using sampled and privacy-governed production traces, especially where users adopt features in new ways. Periodically rotate held-out material as the task changes while retaining a stable historical benchmark for longitudinal tracking. A successful evaluation dataset helps engineers predict real outcomes and make defensible release decisions. Its value lies in revealing what an AI system must improve, not in producing the highest possible score.

Continue learning

Related guides

AI App ObservabilityTrace prompts, retrieved evidence, tool actions, and answers across AI incidents.Evaluating Agents for Accuracy and SafetyTest agent decisions, tool outcomes, safety, and task completion.Online Evaluation for AI SystemsMonitor real-user AI outcomes beyond a static offline benchmark.

Related Posts

• The Ultimate Overview of Microsoft AI-900: What You Should Know

• How to Use Microsoft Copilot: Your Guide to Boosting Productivity with AI

• Microsoft AI-300: Reproducibility Is the First Test of Production ML

• Microsoft AI-300: From Experiment Tracking to an Auditable ML Lifecycle

• Microsoft AI-103: Safe Tool-Using Agents

• Microsoft AI-103: Hybrid Search and Reranking

• Microsoft AB-900: The Microsoft 365 Admin Role Now Includes AI

• Microsoft AI-901: Computer Vision, Language, and Generative AI

• Microsoft AI-103: Choosing Embeddings for Azure Search

• Microsoft AI-103: Chunking Documents for Reliable RAG