Microsoft AI-103: Managing Agent Memory on Azure
Agent memory is useful when an AI system needs continuity beyond one request or one session. It can remember a user’s stable preferences, summarize prior interactions, or retain reusable procedures learned across conversations. That same persistence creates governance questions: what deserves to be remembered, how long should it live, who can read it, and how can it be corrected or deleted?
Microsoft Foundry Agent Service currently provides managed long-term memory in preview. The service distinguishes short-term conversational context from persistent memory and supports user profile memory, chat summary memory, and procedural memory. It also includes memory item operations and store-level retention controls such as default TTL.
Because the capability is preview, teams should separate experiments from production commitments and evaluate whether the current support, regions, limits, and data-handling terms match the workload.
Short-term context and long-term memory are different
Conversation history helps an agent maintain continuity inside a session. Long-term memory persists distilled information across sessions and can be retrieved later.
Keeping those layers separate prevents a common design mistake: storing an entire raw conversation forever simply because some detail may matter later. A long-term memory system should extract durable facts or summaries rather than become an uncontrolled transcript archive.
The broader agent lifecycle should define when information moves from ephemeral context into persistent memory.
User profile memory should hold stable preferences
User profile memory is designed for durable context such as language, recurring defaults, accessibility needs, or other stable preferences. It is most useful when retrieved early enough to personalize the interaction.
Do not store every user statement as profile data. A one-time choice is not automatically a preference, and a speculative inference should not silently become a durable fact.
Memory creation should therefore have confidence and policy rules. High-impact profile data may require explicit confirmation before persistence.
Chat summaries preserve continuity without full transcripts
Chat summary memory distills previous conversation topics and threads so a later session can recover useful continuity without replaying every message.
Summaries can still become stale or misleading. A user’s project can change, a preference can be revoked, or an old decision can no longer apply. The memory layer needs update and deletion paths, not just append-only storage.
For recurring work, retrieve the relevant summary from the current conversation context rather than flooding every new interaction with every historical memory.
Procedural memory captures reusable routines
Procedural memory can represent recurring how-to patterns inferred from prior interactions. This is useful when an agent repeatedly helps with the same workflow and the learned routine remains valid.
Procedure memory should not bypass current permissions or business rules. A remembered sequence can say how a task is normally done, but each execution still needs current authorization and fresh system state.
This aligns with agent permissions: memory can influence behavior, but it should not silently expand authority.
Memory is not a substitute for enterprise knowledge
Foundry documentation distinguishes managed memory from organizational grounding. Stable company policies, manuals, product data, and governed content belong in knowledge sources such as Foundry IQ or Azure AI Search, not in a user’s personal memory store.
Knowledge grounding should answer what the organization says; memory should answer what this agent learned about the interaction history.
Mixing the two layers makes provenance difficult. A user preference and an official policy should not have the same authority.
Retention should be intentional
Memory stores can apply default retention behavior such as TTL, and memory items can be managed individually. Use those controls to match the business need instead of retaining every memory indefinitely.
Different memory types may need different lifetimes. A temporary project preference may expire quickly. An accessibility preference may remain until the user changes it. A procedural memory may require review when the underlying system changes.
Retention rules should be visible to the product and governance teams, not buried only in agent configuration.
Memory creates security and privacy risk
Persistent memory can contain sensitive personal or business context. Access should follow identity boundaries, and the application should avoid storing secrets or information that the agent does not need to retain.
Memory poisoning is also possible: a malicious user or untrusted content may try to make the agent remember false or harmful instructions. Treat memory writes as a controlled operation, especially when the remembered item can influence future tool use.
Prompt injection defenses should therefore include the memory layer, not only the current user prompt.
Memory quality needs evaluation
An agent can fail by forgetting important context, remembering the wrong thing, retrieving an irrelevant memory, or applying a correct memory in the wrong situation. Those are distinct evaluation cases.
Agent testing should include multi-session scenarios where the expected memory behavior is known. Verify what is saved, what is recalled, and whether deletion or correction actually affects future responses.
Production telemetry should distinguish a retrieved memory from a model-generated assumption so operators can diagnose where personalization came from.
Use preview memory selectively
Managed Foundry memory can remove significant custom infrastructure for workloads that need persistent user or agent context. The preview status still matters. Teams should validate supported regions, quotas, pricing, security requirements, and lifecycle guarantees before making it a critical dependency.
For current AI-103 work, the durable architecture lesson is broader than one feature: separate session context, durable user memory, enterprise knowledge, and authoritative business state. Each has a different owner, retention rule, and trust level. Good agent memory preserves useful continuity without turning the model’s history into an uncontrolled source of truth.
Memory retrieval should also be selective. Returning every stored memory on every turn can increase prompt size, expose irrelevant personal context, and create confusing behavior when older memories conflict with the current request. Retrieve by relevance and memory type, then cap the amount of remembered context admitted to the prompt. Stable profile facts may be loaded early, while procedural or chat-summary memories can be fetched only when the current task needs them.
Conflict resolution needs a product rule. If the user says today that a preference has changed, that current statement should normally outrank an older memory. If two memories disagree, the agent may need to ask rather than choose silently. Memory systems should preserve enough metadata—such as timestamps and memory type—to support that decision.
Memory should also be separated by identity and environment. Development agents should not share persistent stores with production unless that is an explicit test requirement. Multi-tenant products need strong partitioning so one user’s memories cannot influence another user’s session.
For regulated or sensitive products, give users or administrators a way to inspect, correct, and delete remembered information where the product requirements call for it. A memory system that personalizes without a correction path can turn a small inference error into a repeated experience problem.
Operationally, monitor memory writes, retrieval frequency, failures, and unusually large growth. Sudden spikes can indicate a prompt bug, a poisoned interaction pattern, or a change that is remembering far more information than intended. Memory should be observable as its own subsystem instead of being hidden inside agent behavior.
Memory extraction itself should be observable. If the system creates a durable memory, log the memory type, source interaction, timestamp, and policy decision without exposing unnecessary sensitive content. That gives operators a way to explain why the agent later behaved as if a fact were true.
Use memory sparingly for high-impact attributes. Identity, legal status, medical facts, financial permissions, and other sensitive or consequential information may belong in authoritative systems rather than inferred agent memory. The agent can query those systems when needed instead of persisting its own interpretation.
Because Foundry memory is preview, production teams should keep a portability plan. The application should understand which business behavior depends on managed memory and what minimum data would need to be migrated if the storage model, API, or preview limitations change.
Memory should not become the only place important user state lives. If a preference drives billing, access, legal consent, or another business-critical decision, persist it in the system of record and let the agent read that state. Agent memory is best for conversational continuity and helpful personalization, not as a replacement for authoritative transactional data.
Test deletion as carefully as creation. A user or administrator may remove a memory, but the agent can still appear to “remember” it if the same fact remains in conversation history, a knowledge source, or another store. A deletion workflow should clarify which layer is being changed and what residual context can remain.