Multi-Agent Systems Need Orchestration Before More Agents
Splitting a problem across several agents can make a system more modular, but adding agents also adds handoffs, context transfer, trust relationships, failure modes, latency, and cost. A multi-agent design should therefore begin with orchestration. The architect needs to know who receives the user’s request, how work is decomposed, which agent owns each responsibility, what information crosses boundaries, and who decides when the overall task is complete.
Microsoft’s current AB-100 scope explicitly includes designing multi-agent orchestrated solutions. That wording is important: orchestration is the architecture, while the individual agents are components. The goal is not to maximize the number of specialized agents. It is to create clear responsibility boundaries that make a complex business workflow easier to reason about, secure, test, and operate.
For the Agentic AI Business Solutions Architect, the useful question is “why is this a separate agent?” If the answer is only that the task has several steps, a workflow or one agent with focused tools may be simpler. Separate agents earn their place when they need distinct goals, knowledge, permissions, lifecycle, ownership, or operational scaling.
Choose decomposition boundaries that remain understandable
A multi-agent system can be decomposed by business domain, capability, security boundary, data ownership, or independent lifecycle. A sales specialist, finance specialist, and policy specialist may each make sense because they use different knowledge and permissions. Splitting “write first paragraph” and “write second paragraph” into separate agents usually creates coordination overhead without a meaningful boundary.
The same reasoning appears in classic software architecture: services are useful when the boundary carries ownership and change isolation, not when every function becomes a service. Agents should have a coherent responsibility that can be described, tested, and monitored. If two agents always change together and share the same data, permissions, and logic, the separation may be architectural theater.
Decide who owns orchestration
Some systems use a front-door agent that recognizes intent and delegates to specialists. Others use a deterministic workflow that invokes agents at predefined stages. A planner may dynamically choose tools and subagents. Event-driven processes may let agents respond to state changes rather than one central conversation. Each pattern changes predictability and the amount of control the architect retains.
Microsoft Copilot Studio supports connected-agent patterns, while Microsoft Foundry supports agent-to-agent and code-oriented orchestration options. The platform choice matters less than making the routing rule explicit. The orchestrator should know which agent is eligible, what input it receives, what output contract it must return, and what happens when confidence is low or several agents could handle the same request.
Keep specialist agents narrow enough to test
Specialization is valuable because it reduces the amount of context and authority any one component needs. A procurement agent can focus on supplier and purchasing policy while a support agent handles cases and entitlements. Narrow responsibility makes it easier to create targeted evaluations, least-privilege permissions, and domain-specific fallback behavior.
That design resembles the ideas behind agent performance, environment, actions, and observations: each agent should have a defined objective, the inputs it can observe, and the actions it can take. A vague “enterprise assistant” boundary makes those elements difficult to constrain or measure.
Treat context transfer as an interface contract
When one agent calls another, context should not be copied blindly. The receiving agent needs enough information to perform its task, but unnecessary conversation history, confidential fields, or irrelevant documents increase cost and risk. Define a message contract that carries the business facts, identity context, and evidence required for the next responsibility.
The contract also needs semantics. If one agent returns “approved,” does that mean a recommendation, a policy decision, or a completed transaction? Typed outputs, status fields, source references, confidence signals, and clear error states reduce ambiguity. Natural language can remain part of the interaction, but machine-readable structure makes orchestration more reliable.
Design for disagreement and uncertainty
Specialized agents can produce conflicting recommendations. A risk agent may reject a transaction that a customer-service agent wants to expedite; a finance agent may calculate a threshold differently from a sales policy agent. The orchestrator needs a rule for resolving conflict instead of assuming that more agents automatically produce consensus.
Resolution may be deterministic precedence, policy-based arbitration, a dedicated reviewer agent, or human escalation. High-impact disagreements should not be hidden by asking another model to “pick the best answer” without a policy. The system should preserve the evidence behind competing recommendations so a person can understand why the conflict occurred.
Limit loops, retries, and delegation depth
Autonomous delegation can create pathological behavior: Agent A asks Agent B, which asks Agent C, which returns to Agent A. Even without an infinite loop, repeated planning and tool calls can create high latency and cost while adding little value. Orchestration needs explicit budgets for steps, retries, recursion depth, time, and spend.
Stopping rules should be visible in telemetry. When the system reaches a limit, it should fail predictably—perhaps by escalating, returning a partial result, or asking for clarification. Quietly truncating the process can be worse than a clear failure because downstream users may assume the task completed successfully.
Make identity and authorization follow the agent boundary
A multi-agent system should not use one overprivileged identity simply because it simplifies integration. If specialists have different responsibilities, their identities and permissions should reflect those differences where the platform allows. The procurement component should not inherit customer-service permissions, and a reasoning-only agent should not receive action privileges it never needs.
Microsoft’s agent platforms increasingly expose dedicated agent identities and scoped access patterns. That makes multi-agent trust boundaries concrete. It also means the architect must decide whether calls preserve end-user context, use an agent’s own identity, or use service-to-service credentials. Each choice changes auditability and the meaning of downstream authorization.
Test the choreography, not just each agent
An agent can pass every unit-style evaluation and still fail inside the larger system. The orchestrator may route the wrong intent, strip necessary context, mis-handle a timeout, or accept malformed output. Multi-agent testing therefore needs scenarios that cross boundaries and verify the complete journey from user request through delegation, action, evidence, and final response.
Teams with implementation responsibility—such as developers working in areas covered by AI-103—often focus on individual agent behavior and tools. The architecture layer adds end-to-end questions: can the workflow recover when a specialist is unavailable, can a result be traced to the agent that produced it, and can permissions be proven at each handoff?
Production traces should show the orchestrator’s decision, the specialist selected, the context sent, tool calls made, latency, errors, and returned result. Without that chain, a user-visible mistake may be impossible to localize. The team needs to know whether the problem came from routing, specialist reasoning, source data, a connector, or the final synthesis step.
Business metrics should also follow the full journey. A system that routes perfectly but takes forty seconds to respond may be unusable. A fast system that escalates half its work may not deliver the intended automation benefit. Orchestration metrics connect technical behavior to outcomes such as completion, deflection, accuracy, cost, and human review volume.
Orchestration should also make concurrency deliberate. Running specialists in parallel can reduce latency when their work is independent, but it can create race conditions when two agents update the same record or depend on one another’s output. The workflow should identify which tasks are safe to parallelize, which require ordering, and which need a final reconciliation step before any external action occurs.
Cost attribution becomes more important as the number of agents grows. One user request may trigger several model calls, retrieval operations, tool executions, and retries across specialists. The architecture should expose the cost of the whole transaction and the contribution of each component. Otherwise a system can appear efficient at the front door while hidden delegation makes each completed task unexpectedly expensive.
Version compatibility is another orchestration concern. If one specialist changes its output fields or policy interpretation, the orchestrator and downstream agents may continue running while producing degraded results. Treat agent interfaces like service contracts: version them, test them together, and avoid relying on undocumented natural-language conventions between independently managed components.
A useful architecture review asks whether the system could be explained as a sequence of responsibilities without mentioning model brands. If the answer is clear—intake, classify, retrieve, decide within policy, request specialist judgment, execute, and record—the orchestration can usually be mapped to agents or workflows sensibly. If the explanation is only “Agent A talks to Agent B, then Agent C,” the design has probably named components before defining the business choreography. Responsibility should come first; implementation topology should follow.
Ownership should follow the same decomposition. If three specialist agents have different business owners, the orchestrator needs a clear owner for the combined outcome and a process for negotiating changes that affect more than one domain. Otherwise each team can optimize its own agent while the end-to-end user experience degrades. Multi-agent governance needs a product-level owner who can resolve cross-boundary trade-offs.
Add an agent only when it creates a stronger boundary
The most maintainable multi-agent systems often contain fewer agents than the first whiteboard sketch. Start with the smallest architecture that expresses the necessary boundaries, then split responsibilities when data, permissions, ownership, scale, or lifecycle truly diverge. This keeps orchestration understandable and prevents a network of agents from becoming harder to operate than the business process it replaced.
A useful rational-agent model still applies: every component should have a goal, observations, and allowed actions. Multi-agent architecture adds a fourth requirement—coordination. If the team cannot explain how responsibilities compose into one reliable outcome, adding another specialist is unlikely to solve the problem.