Microsoft AI-103: Blue-Green Releases for Azure AI
An AI deployment can be healthy by ordinary infrastructure measures and still be unsuitable for users. A changed model may interpret the same prompt differently, an embedding update may move relevant documents down the ranking, and an agent tool schema can introduce unexpected actions. Blue-green delivery gives teams a way to build and evaluate a replacement environment before routing customers to it. The technique is useful only when the two environments are genuinely separable and rollback covers the behavior-changing assets, not just an endpoint. For Azure AI applications, release boundaries must include models, prompts, search indexes, tools, identity, and operational evidence.
On this page
- Establish what blue and green actually contain
- Bring green up without exposing real users
- Compare behavior before flipping traffic
- Mirror traffic without duplicating consequences
- Use a controlled switch and a clear rollback rule
- Beware shared indexes and schema migrations
- Treat prompt and model changes as versioned releases
- Validate after switching, not only before
- Blue-green delivery is a behavior-control practice
Establish what blue and green actually contain
Define the deployment unit before arranging traffic. The blue environment represents the currently approved application behavior. Green contains a proposed revision with its own runnable configuration, model deployment references, prompt versions, retrieval settings, and tool definitions. If both versions point to one mutable prompt store or index without versioning, they are not isolated behavioral environments even if they use two different web URLs. Document which components are shared and which are deliberately independent. Shared systems can be appropriate, but they must be governed so a green experiment cannot silently change blue. During a Microsoft AI-103 deployment, blue-green release safety depends on traffic switching, rollback evidence, and compatibility with existing consumers.
Use immutable release manifests to make each environment reproducible. A manifest should reference the application build, infrastructure version, model/deployment configuration, prompt and agent versions, search index schema, and required identities. Keep secrets outside manifests while recording the approved connection names or permission scopes. The aim is to answer a simple question after an incident: which exact assets produced this answer? If engineers cannot reconstruct green after a rollback, they also cannot reliably explain why it behaved differently from blue.
Bring green up without exposing real users
Provision the new environment and test its basic operation before sending customer traffic. Verify authentication, network routes, model inference, search access, downstream tools, and tracing. Run synthetic health checks that exercise realistic critical paths; a basic HTTP check may succeed while search retrieval is broken. Ensure service identities have the intended permissions, including distinct published agent identities where Foundry publication changes the principal used by tools. A working development playground call is not sufficient evidence that production green can access its authorized resources.
Use non-production test data or safely sanitized fixtures during this phase. A green agent with a payment tool should not execute real payments while an automated smoke test checks its workflow. Configure dry-run or sandbox endpoints and assert that prohibited actions are rejected. Test empty results, expired tokens, and timeouts as well as happy paths. Record a baseline from blue using the same fixtures so the team can detect real changes rather than guessing whether green is “better.” Only then should the release progress to exposure testing.
Compare behavior before flipping traffic
Run both versions against a stable evaluation dataset. Compare groundedness, task completion, tool-call arguments, sensitive-data handling, cost, and latency. Some tests should be deterministic: correct record IDs, schema validation, and unauthorized-action denials. Others may use rubric evaluation with sampled expert review. A model upgrade can legitimately change wording, so a simple exact-string comparison is a poor quality gate for natural-language answers. Focus on business outcomes and safety invariants. Keep separate scores for high-impact workflows so average improvements cannot conceal dangerous regressions.
Investigate differences at the request-trace level. If green performs worse, determine whether the candidate evidence changed, a prompt directed the model differently, or the tool returned a different result. Do not treat every response difference as a defect; a corrected citation or clearer refusal may be an improvement. But unsupported factual claims, expanded permissions, and unexpected side effects require explicit resolution. A successful green evaluation is a documented risk decision, not a claim that two models always produce identical text.
Mirror traffic without duplicating consequences
Traffic mirroring can reveal production input diversity before users see green’s answers. The safe pattern is to provide green with representative requests while preventing its tool calls from mutating live business state. A mirrored agent must not send another customer email, create a duplicate ticket, charge a card, or update an access policy. Implement a read-only execution mode or route side-effecting tools to controlled simulations. Redact or restrict sensitive payloads according to the green environment’s authorized processing boundary. Mirroring does not automatically grant permission to copy every production conversation into another system.
Compare green and blue responses asynchronously and collect differences without increasing interactive user latency. Log the route, deployment version, prompt version, and any simulated tool actions. Make the sampling policy explicit: an ordinary product FAQ may be mirrored broadly, while regulated workflows may require narrower sampling or may not be eligible at all. If green’s retrieval index is older than blue’s, treat the comparison as a known limitation rather than a model performance result. Traffic mirroring is helpful only when the data and environments support a fair comparison.
Use a controlled switch and a clear rollback rule
Blue-green release is often described as switching all traffic from one environment to another, but that switch should follow a documented gate. Confirm error rates, latency, task quality, and authorization behavior immediately before routing. Where infrastructure supports gradual weights or cohorts, a limited exposure period can catch issues that offline tests miss. Keep the old blue environment available and healthy until the new version has met acceptance criteria. A rollback switch is fast only if the old environment still has access to its required dependencies and compatible data.
Define automatic and human-reviewed rollback triggers. A spike in unauthorized tool attempts or sensitive-data incidents may justify immediate rollback even if latency is excellent. A small increase in harmless response variation may need investigation rather than a full reversal. Track tail latency, cost per successful task, retrieval failures, and task completion by user flow. Assign an operator who has permission to act. A release plan without a realistic rollback owner or route control is merely a hopeful sequence of deploy commands.
Beware shared indexes and schema migrations
Search indexes are common hidden sources of coupling. If green requires a new embedding model or field schema, modifying the shared blue index in place can change the old system during a supposedly isolated test. Consider versioned index names, aliases or routing configuration appropriate to the service, and controlled reindexing from original sources. Capture document update positions so green can catch up with live changes. Test authorization metadata and deletion propagation on the replacement index before directing customer queries to it.
Database and tool API changes create similar risks. A green agent might emit a new argument that an older tool service rejects, or an updated schema might no longer support blue rollback. Prefer backward-compatible changes while both versions coexist. For unavoidable breaks, stage migrations and define which release can read or write which shape. If a shared mutation makes old blue unusable, the deployment is not safely reversible. Release readiness should include an explicit check of all shared dependencies, not only the model endpoint.
Treat prompt and model changes as versioned releases
A prompt edit can change business behavior as significantly as a code deployment. Store the complete instruction set and relevant tool schemas under version control, and record their exact versions with evaluation results. A model replacement may alter the interpretation of a tool description even when application code is unchanged. For agents, trace tool selection and action authorization during green testing so the team sees behavioral differences that do not appear in the final answer. Avoid updating a live shared agent definition after green has been declared frozen for release.
Model and prompt version changes can also affect throughput and token cost. Benchmark under realistic concurrency and long-context cases, not only short static prompts. If a new model is slower or uses more output tokens, the switch may require different deployment quota or queue settings. Tie that capacity decision to the release manifest. A blue-green process is valuable because the new behavior can be observed before the final switch; that advantage disappears if critical assets change during observation.
Validate after switching, not only before
Once green serves real users, verify that its actual responses, traces, and tool effects match the approved release. Check canonical deployment routing rather than assuming an API gateway configuration applied everywhere. Monitor cache behavior and sticky sessions; some users may continue talking to blue after a switch, especially in conversational systems with session state. Plan how those sessions are handled so they do not lose context or accidentally mix incompatible memory formats. An agent with persistent conversations may need migration rules that a stateless text endpoint does not.
Keep blue intact through a defined observation period. After green is accepted, record which environment became the new baseline and archive release evidence. Then decommission unused deployments and indexes deliberately, verifying that no active sessions depend on them. A safe process does not leave expensive orphaned models or broadly authorized service principals indefinitely. Blue-green architecture is not an excuse for duplicated long-term infrastructure; it is a temporary safety mechanism for a specific release transition.
Blue-green delivery is a behavior-control practice
The engineering value of blue-green releases lies in making change observable and reversible. For Azure AI, that means treating retrieval, models, prompts, agent tools, and access boundaries as part of one controlled deployment. Keep the old version stable, prove the new version with realistic evidence, constrain mirrored or test traffic, and verify the live route after switching. No release strategy eliminates unexpected user behavior, but this approach gives teams room to detect and recover from defects before they become irreversible production incidents.