Chapter 11: Orchestration Patterns
Orchestration Patterns in Building Agentic AI Systems.
Learning Objectives
By the end of this chapter, you will be able to:
- Explain the agentic AI concept behind Orchestration Patterns.
- Apply Orchestration Patterns to design reliable, production-grade agent systems.
- Recognize operational trade-offs in tool use, orchestration, safety, and cost.
Chapter 11: Orchestration Patterns
Supervisor, Swarm, Pipeline, Fan-out, and Critic loop — with a decision matrix
How Multi-Agent Systems Are Structured
There is no single "correct" multi-agent architecture. The right pattern depends on the task's degree of parallelism, interdependency, and quality requirements. This chapter gives you five production-proven patterns, each with a clear use case and trade-off profile.
1. Supervisor (Hierarchical)
Central controller delegates to specialized workers
2. Swarm (Peer-to-Peer)
Dynamic handoffs between agents, no central controller
3. Sequential Pipeline
Output of one agent becomes input of the next
4. Fan-out + Aggregator
Multiple agents work in parallel; results are merged
5. Critic Loop
Generator + evaluator iterate until quality threshold
Five Patterns in Depth
Pattern 1: Supervisor / Hierarchical
Supervisor Pattern
The supervisor knows the goal and the available workers. It decomposes the task, delegates subtasks to appropriate workers, and assembles results. Workers report back to the supervisor — they do not communicate with each other.
Pattern 2: Swarm
Swarm Pattern — peer handoffs
Each agent decides whether to handle the request or transfer it to a more appropriate peer. No central controller — routing emerges from each agent's own decision. This is the architecture behind many customer support systems.
Pattern 3: Sequential Pipeline
Gather sources
Draft content
Review & revise
Output
Simple, predictable, easy to debug. Weakness: the pipeline is as slow as the sum of each stage's latency. Adding parallelism requires switching to Fan-out or Supervisor patterns.
Pattern 4: Fan-out + Aggregator
Fan-out: parallel agents, then aggregation
source A
source B
source C
Fan-out is the pattern for parallelism. The dispatcher assigns independent subtasks to multiple agents running concurrently. The aggregator merges the results. Total latency = slowest agent, not sum of all. Best for multi-source research or parallel data processing.
Pattern 5: Critic Loop
Score + feedback
Yes
- Output approved
- Return to user
No
- Generator revises based on critic feedback
- Loop again (max N iterations)
Decision Matrix
| Pattern | Parallelism | Control Complexity | Debugging | Latency |
|---|---|---|---|---|
| Supervisor | Medium | High | Moderate | Medium |
| Swarm | Low | Low | Hard (emergent) | Low |
| Pipeline | None | Lowest | Easiest | Highest (sum) |
| Fan-out | Highest | Medium | Easy per agent | Lowest (max) |
| Critic Loop | None | Low | Easy | Variable (iterations) |
State Management Across Agents
The hardest part of multi-agent systems is not the agents themselves — it's the shared state that flows between them.
Shared State Store
- Central dict or database all agents read/write
- Simple; requires locking for concurrent writes
- Good for: supervisor-controlled pipelines
Scoped State (per-agent)
- Each agent has its own state; passes output to next agent
- No shared mutable state; cleaner isolation
- Good for: pipelines with clear handoffs
Handoff-Only State
- All state travels with the handoff message
- Most portable; no external dependency
- Good for: swarm patterns
Checkpointing for fault tolerance
For long multi-agent workflows, save a checkpoint after each major step completes. If the workflow fails mid-way (network error, rate limit, agent crash), you can resume from the last checkpoint rather than restarting from the beginning. LangGraph's built-in checkpointing (covered in Chapter 12) makes this straightforward.
Chapter 11 Quiz
1. In the Fan-out + Aggregator pattern, what determines the total task latency?
2. Which pattern is hardest to debug and why?
3. What is the primary advantage of "handoff-only state" in the Swarm pattern?