Skip to main content

Multi-Agent Reliability: Fix the Handoffs Before the Model

· 6 min read
Victor Jimenez
Victor Jimenez
Senior Product Engineer

Most multi-agent failures are orchestration failures, and a smarter model does not repair a handoff that never said what done means. Before I tune any model, I would implement four controls: typed handoffs, explicit state contracts, task-level evals and transactional rollback. GitHub's engineering deep dive, published on February 24, shows the same core pattern.

Where multi-agent workflows break​

The deep dive covers what happens when a single coding assistant becomes several specialized agents. The pain points are practical, and none of them is about model intelligence:

  1. Handoffs are ambiguous, so downstream agents infer missing context.
  2. Shared state mutates without schema discipline, causing drift and duplication.
  3. Success checks run at the end of the run, so bad branches accumulate cost.
  4. Failed steps are hard to isolate, so recovery means "start over" instead of rollback.

The last one is the most expensive. "Start over" has the precision of turning the laptop off and on again, and it pays the full token bill every time. The replacement is a checkpoint after every passed gate, so a failure rolls back one step instead of the whole run.

One handoff, many retries

One weak handoff can trigger a cascade of retries across the planner, implementer and verifier roles. That failure costs both tokens and time.

Controls mapped to failure patterns​

Reliability comes from workflow design before model tuning. Each failure pattern gets one control and one rollback trigger.

Failure pattern from GitHub deep diveReliability controlImplementation detailRollback trigger
Missing context between agentsTyped handoff envelopeEvery agent emits goal, constraints, artifacts, done_criteriaEnvelope missing required keys
Shared memory driftState contract with versionsKeep state_version and an immutable event log per stepState schema validation fails
Late quality detectionStep-level eval gatesRun checks after each agent output, not only at the endEval score below threshold
Retry stormsBounded retries + policy routingMax retries per class (format, tool, logic)Retry budget exhausted
Full restart recoveryTransactional checkpointsSnapshot repo + plan after each passed gateGate fails after side effects

The handoff envelope​

Every inter-agent transfer uses a strict JSON envelope. An agent that infers missing context is guessing on the token budget, which is the most expensive way to read a ticket. The fix is a set of required keys, goal, constraints, artifacts and done_criteria, with the envelope rejected at the boundary when any of them is missing.

handoff-envelope.json
{
  "handoff_id": "uuid",
  "from_agent": "planner",
  "to_agent": "implementer",
  "goal": "Apply fix for flaky checkout test",
  "constraints": ["no schema changes", "keep API stable"],
  "artifacts": ["failing_test_trace.md", "target_file_list.json"],
  "done_criteria": ["tests pass", "diff limited to 2 files"],
  "state_version": 12
}

This mirrors GitHub's emphasis on explicit structure in tool inputs and outputs, and it keeps downstream behavior deterministic.

The state and eval loop​

Checks run after every agent output, and retries have a budget. A retry loop without a budget is while (true) with an invoice attached. A max retry count per failure class (format, tool, logic) turns it into a decision: when the budget runs out, the workflow rolls back to the last checkpoint and escalates with a failure report.

Evals that run per step​

Each step gets checked before the next agent sees its output. The format, tool and task evals look like this:

eval-format.json
{
  "eval_type": "format",
  "check": "Output matches required schema",
  "why": "Prevents parser/runtime failures in next agent",
  "pass_criteria": "JSON schema validates without errors"
}

The policy eval adds constraint checks such as no-depr-api and no-secret on top of the others.

Eval typeExample checkWhy it matters
Format evalOutput matches required schemaPrevents parser/runtime failures in the next agent
Tool evalTool call used allowed inputs onlyPrevents silent side effects and permission drift
Task evalUnit target passed for scoped filesCatches regressions before the next handoff
Policy evalConstraints respected (no-depr-api, no-secret)Keeps compliance and security intact

A deprecation warning lets a bad step pass with a note attached. My call is to make it a hard stop.

Deprecation is a failure

Treat deprecated APIs and deprecated workflow patterns as an immediate eval failure, not a warning. When an agent proposes a deprecated hook, function or integration path, fail fast and route it back with a replacement hint in the envelope.

Agent lifecycle states​

Every run ends either checkpointed and back to idle, or rolled back and escalated with a report. No state leads to "start again from scratch".

Migration checklist​

  • Define typed handoff envelope schema
  • Implement state versioning and immutable event log
  • Add step-level eval gates after each agent output
  • Configure bounded retry budgets per failure class
  • Implement transactional checkpoints with rollback
  • Add deprecation check to policy eval
  • Wire eval results into monitoring/alerting

What I learned​

  • Multi-agent reliability is mostly an interface design problem: handoff contracts beat prompt tweaks.
  • State versioning plus event logs makes incident replay and root cause analysis much faster.
  • Step-level evals reduce blast radius and token waste because bad branches are cut early.
  • Rollback has to be first-class. Without it, every failure becomes a full restart.
  • A deprecation gate is cheap insurance against subtle breakage during upgrades.
Related posts

References​