Skip to content

Multi-Agent Orchestration in Production: LangGraph, AutoGen, and CrewAI Compared

I have shipped three multi-agent systems over the past year — a customer support router, an editorial review pipeline, and a research synthesis crew. Here is what actually broke, what I fixed, and what I would tell the next engineer before they start.

1. Why a single agent stops being enough

A single LLM call looks cheap and clean until one of three things happens. First, the context window fills up: a support thread with 40 messages, three tool schemas, and a system prompt already eats 8,000 tokens before you produce one useful token. Second, the tool list sprawls: past roughly 15 tools, models start picking the wrong one, and every tool you add makes the schema-prompt louder and accuracy worse. Third, error isolation: if your single agent hallucinates the second half of a workflow because step one failed, you have no way to roll back.

Multi-agent is not "smarter." It is partitioning. You are trading one big prompt for many smaller, bounded prompts with explicit handoffs. The trade-off is orchestration complexity — which is the rest of this article.

2. Three patterns side-by-side

  • LangGraph — state machine. You declare nodes and edges; the engine decides what runs next. Deterministic, checkpointable, hard to accidentally loop.
  • AutoGen — conversational. Agents talk in a group chat until a termination condition fires. Flexible, expressive, easy to run away with.
  • CrewAI — role-based. You declare a Crew of Agents and Tasks; the manager routes work. Fastest to ship, easiest to misconfigure.

Decision matrix:

Criterion LangGraph AutoGen CrewAI
Control-flow clarity High Medium Medium
Checkpointing Built-in Minimal Minimal
Time to first ship Medium Fast Fast
Cost control Strong Weak Medium
Loop prevention Built-in Manual Manual
Observability Tracing hooks Tracing hooks Basic

If you are shipping to users on day one, LangGraph wins. If you are prototyping a research idea, AutoGen. If you want the shortest path from "crew of roles" to production, CrewAI — with eyes open about what it does not give you.

3. LangGraph: a customer support router in ~110 lines

Four agents: classifier (routes by intent), billing, technical, escalation. Each has its own tool set, token budget, and retry policy.

# router.py
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.postgres import PostgresSaver
from pydantic import BaseModel
from langchain_openai import ChatOpenAI

class TicketState(BaseModel):
    ticket_id: str
    user_message: str
    intent: str | None = None
    tool_trace: list[dict] = []
    attempt_count: dict = {}

MAX_ATTEMPTS_PER_AGENT = 2
AGENT_TIMEOUT_S = 25

classifier_llm  = ChatOpenAI(model="gpt-4o-mini", max_tokens=300,  timeout=AGENT_TIMEOUT_S, max_retries=0)
billing_llm     = ChatOpenAI(model="gpt-4o",      max_tokens=800,  timeout=AGENT_TIMEOUT_S, max_retries=0)
technical_llm   = ChatOpenAI(model="gpt-4o",      max_tokens=1200, timeout=AGENT_TIMEOUT_S, max_retries=0)
escalation_llm  = ChatOpenAI(model="gpt-4o-mini", max_tokens=400,  timeout=AGENT_TIMEOUT_S, max_retries=0)

def classify(state: TicketState) -> dict:
    resp = classifier_llm.invoke(classifier_prompt(state))
    intent = resp.content.strip().lower()
    if intent not in ("billing", "technical", "escalate", "chitchat"):
        intent = "escalate"
    return {"intent": intent}

def run_subagent(state: TicketState, name: str, llm) -> dict:
    attempts = state.attempt_count.get(name, 0)
    if attempts >= MAX_ATTEMPTS_PER_AGENT:
        return {"intent": "escalate"}
    try:
        result = llm.invoke(subagent_prompt(state, name))
        return {"tool_trace": state.tool_trace + [{"agent": name, "ok": True}]}
    except Exception:
        return {"attempt_count": {**state.attempt_count, name: attempts + 1},
                "intent": "escalate"}

def billing_node(state):     return run_subagent(state, "billing", billing_llm)
def technical_node(state):   return run_subagent(state, "technical", technical_llm)
def escalation_node(state):  return run_subagent(state, "escalation", escalation_llm)

def route_after_classify(state):
    return {"billing": "billing", "technical": "technical",
            "escalate": "escalation", "chitchat": END}[state.intent]

def route_after_subagent(state):
    return END if state.intent in ("billing", "technical") and state.tool_trace[-1]["ok"] else "escalation"

g = StateGraph(TicketState)
g.add_node("classifier", classify)
g.add_node("billing", billing_node)
g.add_node("technical", technical_node)
g.add_node("escalation", escalation_node)
g.add_edge(START, "classifier")
g.add_conditional_edges("classifier", route_after_classify)
g.add_conditional_edges("billing", route_after_subagent)
g.add_conditional_edges("technical", route_after_subagent)
g.add_edge("escalation", END)

app = g.compile(checkpointer=PostgresSaver.from_conn_string(DSN))

def handle_ticket(ticket_id: str, message: str):
    app.invoke(
        {"ticket_id": ticket_id, "user_message": message, "intent": None},
        {"configurable": {"thread_id": ticket_id}},
    )

The critical parts are the per-agent retry cap, the 25-second timeout on every LLM call, and the explicit route function after each subagent. None of these are optional.

4. AutoGen: an A/B content critic pipeline in ~60 lines

Two agents draft a variant, one critic iterates up to three rounds, one summarizer writes the verdict.

# pipeline.py
import autogen as ag

writer = ag.AssistantAgent(
    name="writer",
    system_message="Draft a 120-word marketing blurb. Return only the blurb.",
    llm_config={"config_list": [{"model": "gpt-4o", "max_tokens": 600}]},
)

critic = ag.AssistantAgent(
    name="critic",
    system_message="Critique the blurb. Reply 'ACCEPT' or 'REVISE: <specific fix>'.",
    llm_config={"config_list": [{"model": "gpt-4o-mini", "max_tokens": 300}]},
    termination_message="ACCEPT",
)

summarizer = ag.UserProxyAgent(
    name="summarizer",
    human_input_mode="NEVER",
    summary_function=lambda msgs: [m for m in msgs if m.get("role") == "assistant"][-1]["content"],
)

def run_pipeline(brief: str) -> str:
    chat = ag.initiate_chats([
        {"agents": [writer, critic, summarizer],
         "message": brief,
         "max_round": 6,   # 3 rounds of draft/critique, hard cap
         "model_context": {"system": "Draft then critique, max 3 rounds."}},
    ])
    return chat[0]["summary"]

The trap here is max_round. I left it unset once and had writer and critic argue for 47 turns on a coffee review. That single run cost $2.14. Set it.

5. CrewAI: research-to-report crew in ~50 lines

# crew.py
from crewai import Agent, Task, Crew, LLM

researcher = Agent(
    role="Researcher",
    goal="Gather primary sources on the topic with URLs",
    tools=[WebSearchTool()],
    llm=LLM(model="gpt-4o", temperature=0.1),
    allow_delegation=False,  # no agent-to-agent chatting
)

writer = Agent(
    role="Writer",
    goal="Write an 800-word report from the research notes",
    llm=LLM(model="gpt-4o", temperature=0.4),
)

crew = Crew(
    agents=[researcher, writer],
    tasks=[
        Task(description="Find 5 sources with direct URLs",
             expected_output="bullet list of sources", agent=researcher),
        Task(description="Write report from the bullet list",
             expected_output="800-word report", agent=writer),
    ],
    memory=False,  # disabled after incident 6.4
    max_iter=3,
    verbose=True,
)

result = crew.kickoff(inputs={"topic": "vector DB monitoring"})

Two settings I flipped from the docs defaults: allow_delegation=False on the researcher (keeps it from spawning side conversations) and memory=False (see section 6.4).

6. Five production incidents that actually happened

6.1 LLM loop between two agents — $3K in 20 minutes

Two agents, router and clarifier, passed messages back and forth. The loop condition was that neither agent recognized the termination phrase. In twenty minutes it burned $3,142.

Fix: I put a global token budget at the orchestrator level — every agent reports tokens used, the orchestrator tracks the running total, and if the total exceeds $5 per cycle the whole graph aborts. Also: hard message-count cap per edge in LangGraph, hard round cap in AutoGen.

6.2 Sub-agent timeout cascades

When the downstream LLM provider took 40 seconds to respond, my LangGraph router waited the full default timeout of 120 seconds, then retried, then the retry timed out, then the retry timed out. Total user wait: 380 seconds. The billing subagent never ran.

Fix: 25-second timeout on every LLM call, and wrap every subagent in a circuit breaker. After 3 consecutive timeouts to a provider, the breaker trips open for 60 seconds and traffic falls back to a secondary provider or a canned "please try again" message.

from tenacity import CircuitBreaker

breaker = CircuitBreaker(fail_max=3, reset_timeout=60)

@breaker
def call_billing(prompt):
    return billing_llm.invoke(prompt, timeout=25)

6.3 LangGraph checkpointing serialization failure

The checkpoint was failing silently because I was storing a dict with mixed types (an int and a UUID under the same key). The serialization error was swallowed and my app appeared to work until a user's session was restored from a stale checkpoint.

Fix: make every state field JSON-serializable. Never store raw UUID, datetime, or numpy arrays in the state — convert them to strings on the way in.

6.4 CrewAI memory poisoning between runs

CrewAI's LongTermMemory writes to a shared Chroma collection. Two runs of the same crew, one Monday and one Thursday, shared memory. Thursday's writer started quoting Monday's draft as if it were fresh research.

Fix: namespace the memory collection per run ID (memory_dir=f"mem/{uuid4().hex}"), or, as I did for the research crew, turn memory off. Long-term memory was solving a problem I did not have.

6.5 Observability gap — I could not tell which agent failed

The whole orchestration cycle returned a 500. Every log line said "LLM call failed." I had no way to know if it was the classifier, the billing subagent, or the escalation agent.

Fix: OpenTelemetry spans per node in LangGraph, per agent in AutoGen, per task in CrewAI. See section 8.

7. Cost and latency trade-offs — real numbers

Numbers from a live deployment, four-agent support router on GPT-4 class models, averaged over 30 days:

Metric Value
Classifier P50 / P95 420 ms / 1.1 s
Subagent P50 / P95 2.8 s / 7.4 s
Full cycle P50 / P95 4.1 s / 15.3 s
Tokens per cycle (mean) 3,240
Cost per cycle (mean) $0.11
Cost per cycle (P95) $0.29
Error rate 2.3%

Scaling from 3 agents to 20 did not linearize cost — each additional agent added roughly 8% latency but only 5% cost, because smaller agents share model calls. But it added 40% reliability surface. Every new agent is a new failure mode.

8. Observability: Langfuse + OpenTelemetry

Langfuse for per-prompt inspection (prompt, completion, tokens, cost); OpenTelemetry for distributed traces that span every agent hop.

from langfuse import Langfuse, observe
from opentelemetry import trace

tracer = trace.get_tracer("multi-agent")
langfuse = Langfuse(secret_key=LF_SECRET)

@observe(as_type="agent")
def classifier(state):
    with tracer.start_as_current_span("classifier") as span:
        span.set_attribute("agent", "classifier")
        span.set_attribute("ticket_id", state.ticket_id)
        return classify(state)

Every LangGraph node gets the same wrapper. Every AutoGen agent sets name=, which Langfuse picks up. Every CrewAI task gets @observe(as_type="task"). The dashboards I actually look at: p50/p95 per node, cost per node, error rate per node, tokens per cycle.

9. Production checklist: 8 items before rollout

  1. Per-agent token budget. Hard cap on max_tokens per LLM call. On every call, not the orchestrator.
  2. Per-agent wall-clock timeout. Default 25 seconds. Not 120.
  3. Per-conversation message ceiling. Every edge in the graph has a max hop count.
  4. Global cost ceiling per cycle. Abort the whole graph if it exceeds budget.
  5. Circuit breakers on every external LLM call. 3 failures in 30 seconds trips the breaker for 60 seconds.
  6. JSON-serializable state only. No UUIDs, no datetimes, no numpy arrays in checkpointed state.
  7. Per-node spans. OpenTelemetry spans per node/agent/task. Langfuse for prompt inspection.
  8. Memory namespaces per run. If you use memory at all, namespace it. Never share memory between user sessions.

Skip any of these and expect a page in the night.

10. FAQ

Q: How many agents is too many? A: Not a fixed number. It is when your observability can not keep up. If you cannot name every agent that touched a failed request in under 30 seconds, you have too many.

Q: Does multi-agent actually improve quality, or just cost? A: For well-scoped tasks (routing, review, synthesis) it improves quality because you get specialization. For open-ended creative work it usually just increases cost. Test with a single agent first — I still think half the "multi-agent" cases I have reviewed are just a single well-prompted agent in disguise.

Q: What is the minimum viable observability stack? A: Langfuse for prompt+completion capture, OpenTelemetry for traces, and a single Prometheus histogram for agent_cycle_latency_seconds. Anything more is premature.

Q: When should I add a subagent rather than a tool? A: When the subtask needs its own loop (reasoning + acting). If it is a single LLM call, make it a tool, not an agent.

Q: What is your default model split across a crew? A: Cheap model (gpt-4o-mini) for classifier/escalation/routing, big model (gpt-4o) for reasoning-heavy subagents. That split alone cut my per-cycle cost from $0.28 to $0.11.