Practice Exams:

Microsoft AI-103: Designing Agent Memory with Boundaries

AI & Machine Learning

An assistant that forgets every conversation can frustrate users, but one that remembers too much can leak private information or act on outdated assumptions. The word memory often hides several different systems: a short conversation window, retrieved enterprise facts, durable user preferences, and state left by an unfinished tool workflow. Microsoft Foundry Agent Service includes managed memory capabilities, yet teams still need to decide what may persist, who can retrieve it, and how deletion or correction works. Reliable agent memory is therefore a data-governance and product-design task as much as a modeling feature. Continuity should never become uncontrolled collection of personal context.

On this page
  1. Separate short-term context from persistent memory
  2. Define when information is allowed to be remembered
  3. Design retrieval and isolation as authorization controls
  4. Manage consolidation without rewriting history
  5. Implement forgetting as a real data operation
  6. Evaluate usefulness and privacy together
  7. Keep memory subordinate to current facts

Separate short-term context from persistent memory

Short-term conversational state helps an agent interpret what “that document” or “the second option” refers to during the current task. It may be held in a session, a compact summary, or a durable conversation record with a defined expiry. Persistent memory is different: it retains selected information across sessions, such as an explicitly expressed preference or a stable project setting. Enterprise retrieval is different again, because it answers from governed source documents rather than a user-specific memory store. Avoid using one storage bucket for all three. Agent memory in Microsoft AI-103 needs explicit retention and access boundaries before a temporary conversation detail becomes durable context.

Document each memory type’s purpose, sensitivity, owner, retention and expected update behavior. A user’s temporary travel dates should not automatically become a permanent preference. A recalled project decision should identify its source and validity window, while an external tool’s state belongs in the authoritative system rather than an unverified conversational note. This classification guides which facts deserve remembering and which should simply be retrieved again when needed.

Define when information is allowed to be remembered

A memory ingestion rule should select only information that will help future authorized tasks. Explicit user preferences may be candidates, but sensitive medical, identity, financial or security details need strict necessity and consent controls. Summarization alone does not make sensitive content safe to retain. Limit memory extraction to approved sources and purposes, and prevent an agent from converting arbitrary instructions inside a retrieved web page into a new durable user preference.

Consider a helpdesk assistant handling three conversations with the same employee. In the first conversation, the employee says they prefer concise steps; this might be a useful optional preference if the product gives clear consent controls. In the second, they upload a confidential incident transcript to diagnose a problem; the assistant must not automatically turn that sensitive document into durable personal memory. In the third, they ask what changed in the approved troubleshooting procedure; the answer should come from the current controlled knowledge base, not a remembered statement from an earlier conversation. These categories require different rules for collection, expiry, retrieval, and correction. A summary is not harmless merely because it is shorter than the original text.

Test two accounts with similar names but different tenants, one employee who deletes an optional preference and one whose group membership is revoked. Verify that saved memories do not cross identities or reappear from a cached index after deletion. Retain an audit receipt showing that the deletion process completed, without retaining the deleted text as an uncontrolled log field. If an agent uses memory to prepare an external action, show the relevant remembered fact and require a fresh authorized confirmation before the action executes. A useful memory design improves continuity while ensuring that current evidence, consent, and user control outrank convenience.

Distinguish direct user statements from model inferences. An assistant might infer that someone prefers a product because it appeared in one conversation; treating that as a stable personal fact creates a misleading profile. Store provenance, confidence, date and scope where appropriate. If the user corrects or revokes a preference, the system must be able to update or invalidate the old representation rather than keep both contradictory items indefinitely.

Design retrieval and isolation as authorization controls

Memory retrieval must be scoped to the correct user, organization, project and purpose. A shared agent serving several tenants should not accidentally retrieve another tenant’s conversational details because embeddings look semantically similar. Enforce identity and access checks before returning memory to the model; a prompt that says “use only my memories” is not an access-control mechanism. Test group changes, cross-tenant queries and session switching explicitly.

Control what the agent can do with recalled information. Remembering that a user usually works in a particular Azure subscription does not authorize deploying resources there today. For tool execution, require current user authorization and a fresh state check. Treat memory as possibly stale contextual input, not a source of permission. Sensitive operations should rely on authenticated identities, live policy and explicit approvals independently of conversational continuity.

Manage consolidation without rewriting history

Long-running systems may merge repeated facts into compact memories, but that process can erase qualifications. “Prefers short reports on weekdays” is not equivalent to “always prefers short reports.” Preserve boundaries such as date ranges, exceptions and the context in which a preference was expressed. When two statements conflict, prefer asking for clarification or preserving an uncertainty marker over silently making one authoritative.

Version consolidation rules and test their effect on held-out conversations. A model update may change what it extracts, what it ignores and how it resolves contradictions. Review cases where summarization turns a tentative question into a durable claim. Maintain a source link or restricted trace reference so an authorized reviewer can understand and correct errors without exposing the entire conversation to ordinary application users.

Implement forgetting as a real data operation

Users and organizations need a way to inspect, correct and remove persistent memory under applicable retention rules. Deleting one visible summary does not prove all derived embeddings, caches and replicas have been reconciled. Document deletion propagation and its latency, preserve only required audit evidence, and test that a forgotten item no longer influences future retrieval. If legal or contractual retention applies, explain the distinct status of retained audit records versus active personalization memory.

A system may also need time-to-live policies for short-lived facts, automatic expiry of old project settings, and explicit invalidation when a source system changes. Test the negative path: can an agent still recall a deleted preference through a stale cache or previous session summary? Memory privacy is not complete until the retrieval path respects the user’s latest correction and the system can account for remaining retained copies.

Consider a travel-booking agent that records a user’s preferred city, then later stores a passport number pasted into a chat even though the application only needs it for one confirmed transaction. The passport field should never enter general reusable memory merely because it may be relevant someday. Classify values at ingestion: short-lived transactional identifiers stay in an authorized system of record, while low-risk preferences can be retained only under a defined purpose. When a user requests deletion of a remembered preference, the process must find its durable memory entry, vector representations, cached summaries and queued background consolidation tasks, not just hide the value from the conversation screen.

Write a deletion test with two sessions and two accounts. Save an innocuous preference for Account A, check that it is never retrieved for B, then delete it from A’s approved memory store. Reissue a query that would previously have retrieved the old value. The expected result is an absence of the deleted preference in memory search, summaries and agent context, with a narrowly retained audit event if policy requires one. Next, restart a queued summary worker to test that an old job cannot recreate the deleted item from stale input. The application should record the deletion identifier, scopes traversed, number of affected derivatives and remaining exception or retention obligations without reproducing the private value in common logs. These results distinguish genuine erasure from UI-only forgetting.

Evaluate usefulness and privacy together

A memory benchmark should include recurring tasks where remembering helps, tasks where irrelevant memory should not be used, conflicting preferences, stale facts, malicious attempts to plant memory, and user deletion requests. Measure whether the system retrieves useful context, respects scope, attributes source correctly, and abstains from using uncertain information. A higher recall rate can be a negative outcome if it retrieves private or outdated material.

Use controlled examples across multiple users and tenants to test isolation. Do not log unrestricted personal memories into model-quality telemetry by default. For security review, inspect both the memory store and the way application prompts inject its contents into model context. Prompt injection remains possible if stored memory is treated as privileged instructions rather than lower-trust data. Authorization and instruction hierarchy must stay distinct.

Keep memory subordinate to current facts

An assistant’s memory may improve convenience, but current authoritative systems should win when task facts change. A remembered delivery address is not proof that a customer still lives there; a stored project permission does not override a new access policy. Before taking actions, reconcile relevant memory with the user’s present request and live records. If the system has insufficient evidence, ask for confirmation rather than assuming older context remains true.

A trustworthy memory design makes the scope of continuity visible and controllable. Users should benefit from not repeating stable preferences without losing the ability to correct or forget them. For Azure AI agents, the operating standard is not how much information the system can remember, but how selectively, securely and accurately it uses retained context to support work the user has actually authorized.

Continue learning

Related guides

Prompt Injection Defenses on AzureSeparate untrusted retrieved instructions from application authority and tools.Agent Identity in Azure AI FoundryAssign agent identities and restrict access as workloads are deployed.Building Multi-Agent Workflows on AzureCoordinate agent handoffs, shared state, permissions, and failure recovery.

Related Posts

• Think Smart, Build Smarter: Mastering AI-900 with Microsoft Azure

• Microsoft AI-300: Serving GenAI: Balancing Latency, Throughput, and Cost

• Microsoft AB-620: Give an AI Agent a Job Before More Tools

• Microsoft AB-100: Integration Design for AI-Powered Business Processes

• Microsoft AI-103: Building Reliable RAG Retrieval

• Microsoft AB-900: Conditional Access in an AI-Enabled Workplace

• Microsoft AI-900: Machine Learning, Generative AI, and Agents

• Microsoft AI-103: Capacity Planning for Azure AI Services

• Microsoft AI-103: Operating GenAIOps on Azure

• Microsoft AI-103: Grounding Azure AI in Enterprise Data