Coordinating agents

Multi-agent orchestration: coordinating AI agents

What orchestration means for agents

Orchestration is the control problem that sits on top of any agent system. A single agent already orchestrates a loop — call a tool, read the result, decide the next step, repeat until done. Multi-agent orchestration extends that to several agents: routing a task to the right one, deciding when to run agents in parallel versus in sequence, threading context between them, merging their outputs, and bounding the whole thing so it terminates.

Three nearby topics it's easy to conflate, and where each is covered:

• Orchestration as a layer — where the control of the loop sits in a system. That's the architecture view; see agentic AI architecture.
• Workflow patterns — the named control-flow shapes for composing LLM calls (prompt chaining, routing, orchestrator-workers, evaluator-optimizer). Covered in agentic AI design patterns.
• Coordination as a topic — this page: the topologies for wiring multiple agents together, when multi-agent is worth it, and the coordination challenges that decide whether it stays reliable.

Single-agent vs multi-agent: when to go multi

A single agent is one model in one loop with one set of tools and one context window. It is the right default, and it stays the right default for longer than most teams expect. You should be able to name a concrete reason a single agent fails before you add a second one. The reasons that actually justify multi-agent:

• Context won't fit. The work needs more grounding than one context window holds, so you split it across agents that each carry an isolated slice — the central argument in Anthropic's multi-agent research system, where a lead agent spawns subagents that each search with their own context.
• Genuine parallelism. Subtasks are independent and breadth-first (research, multi-source gathering), so fanning out is faster than one agent working serially.
• Specialisation and isolation. Distinct skill sets or trust boundaries are better handled by separate agents with scoped tools than by one over-equipped agent.

The cost is real and worth stating plainly. Anthropic reported their multi-agent research system used roughly 15x the tokens of a plain chat interaction, which is why they reserve it for high-value, parallelisable tasks. And Cognition's Don't Build Multi-Agents argues that naive parallel subagents are fragile precisely because context is split — agents make conflicting assumptions when they can't see each other's work, so writes (decisions that change shared state) should stay effectively single-threaded. Both are true at once: multi-agent can beat single-agent on the right task and quietly break on the wrong one.

Orchestration topologies

How you wire the agents together is the topology — the single most consequential structural decision. Five recur across every serious framework. They differ in who holds control and how work moves between agents.

These aren't competing brands — most real systems combine them (a supervisor whose workers run in parallel, say). The supervisor and parallel topologies are the workhorses; network/handoff buys flexibility at the price of predictability, so reach for it last. Note that supervisor-with-workers is the same shape as the orchestrator-workers design pattern seen from the topology side — the pattern is the control flow, the topology is the wiring.

The hard parts: state, errors, cost, observability

The topology diagram is the easy part. What decides whether a multi-agent system survives contact with production is four coordination problems that a single agent mostly doesn't have:

• State. Which agents share context and which stay isolated, and who is allowed to write. Isolation prevents context bloat but causes the conflicting-assumptions failure; the durable fix is to keep state-changing decisions single-threaded and pass agents enough trace to stay consistent.
• Errors. A worker that times out, loops, or returns garbage can poison the synthesis or cascade to its siblings. Each agent boundary needs an explicit timeout, retry, and fallback, plus a supervisor that can drop or re-dispatch a failed worker rather than hang.
• Cost. Tokens multiply with every agent and every coordination round-trip — the 15x figure above is the warning. Route cheap models to routing and gathering, reserve strong models for hard reasoning, and bound the number of agents and turns.
• Observability. With several non-deterministic agents in flight, a failure is undiagnosable unless you trace every agent, hand-off, tool call, token, and latency across the whole run. Tracing is not optional at this scale; it is how you debug at all.

Notice that three of these four are guardrail and operational-plane concerns — bounding loops, scoping tools, tracing, cost control — which is why orchestration and the operational plane are designed together, not bolted on after.

Tools and frameworks for orchestration

Frameworks exist mostly to make the topologies above expressible without hand-rolling the coordination glue. The current landscape, briefly:

• LangGraph models agents as nodes and control flow as edges over a shared state object, and names the supervisor, hierarchical, and network topologies directly with handoffs implemented as tools.
• CrewAI models a crew of role-based agents with sequential and hierarchical (manager-delegates) process modes.
• OpenAI Agents SDK uses handoffs — each handoff is a tool the model calls to transfer control to another agent — making the network topology first-class.
• Claude Agent SDK supports subagents by default for parallelism and context isolation, the orchestrator-worker shape behind Anthropic's research system.
• Amazon Bedrock offers multi-agent collaboration (a supervisor coordinating collaborator agents) and AgentCore for running agents at production scale; Microsoft's actively developed path is now the Agent Framework, with the original AutoGen in maintenance.

Don't pick a framework before you've chosen a topology and confirmed you need multi-agent at all — the framework is the last decision, not the first. For a deeper comparison of these tools, see AI agent frameworks; for the components each one orchestrates, see agentic AI architecture.

Frequently asked questions

What is agent orchestration?

Agent orchestration is the coordination of an AI agent's work toward a goal: controlling the loop of deciding, acting, and observing, choosing the next step, and bounding the whole thing so it terminates. For a single agent it's the act-observe-decide loop; for several agents it also covers routing, hand-offs, parallelism, and merging results.

What is multi-agent orchestration?

Multi-agent orchestration is coordinating several AI agents that each have their own context and tools so they work toward one goal. It adds the questions a single agent doesn't face — how agents hand off work, share or isolate state, run in parallel or sequence, and recover from each other's failures — and the topology you choose (supervisor, hierarchical, sequential, parallel, or network) answers them.

When should you use multiple agents?

Use multiple agents only when a single agent provably can't do the job: the work needs more grounding than one context window holds, the subtasks are independent and benefit from genuine parallelism, or distinct skills and trust boundaries are better isolated. Multi-agent can use far more tokens than a single agent — Anthropic reported about 15x versus plain chat — so reserve it for high-value, parallelisable tasks and start single-agent otherwise.

What are the multi-agent orchestration patterns?

The recurring topologies are supervisor (an orchestrator routing to worker agents), hierarchical (supervisors of supervisors), sequential pipeline (a fixed chain of agents), parallel fan-out (independent agents run concurrently and aggregated), and network/handoff (decentralised, any agent can pass control to any other). Most real systems combine them. These are topologies — the workflow-level control-flow patterns are covered in the agentic AI design patterns guide.

What are the challenges of multi-agent systems?

Four dominate: state (deciding which agents share context and who may write, to avoid conflicting assumptions), errors (a failed agent cascading to siblings or poisoning the final synthesis, so each boundary needs timeouts, retries, and fallbacks), cost (tokens multiply with every agent and round-trip), and observability (tracing several non-deterministic agents so failures are diagnosable). They're why multi-agent is harder to ship than it looks.

What tools help with agent orchestration?

Common frameworks include LangGraph (agents as nodes over shared state, with supervisor/hierarchical/network topologies), CrewAI (role-based crews, sequential and hierarchical), the OpenAI Agents SDK (handoffs), and the Claude Agent SDK (subagents). On cloud platforms, Amazon Bedrock offers multi-agent collaboration plus AgentCore for production scale. Choose a topology and confirm you need multi-agent before choosing a framework.