AgentStatus
We tested every public MCP server we could find. Calling is solved. Delivery is not.
Read the reportThe era of single agents is ending.
Modern applications increasingly rely on multiple agents collaborating: one for research, one for reasoning, one for action. Orchestrators coordinate specialists. Tool-using agents call other agents as tools.
Research shows multi-agent systems have the highest failure rates-the HB-Eval benchmark found that complexity directly correlates with reliability gaps, with basic architectures dropping from 86.9% to 44.0% success under perturbations.
This creates a new challenge: how do you trust a system where no single component is fully in control?
User → Agent → ResponseTrust model: Does this agent work?
Monitoring: Validate the agent.
User → Orchestrator → Research Agent → [External APIs]
→ Reasoning Agent → [Knowledge bases]
→ Action Agent → [Tools, other agents]
← Synthesized Response ←Trust model: Does each component work? Do they work together? Does the composition work?
Monitoring: Much harder.
One broken agent breaks the chain.
Example:
User sees: Generic error or timeout.
Debugging challenge: Where did it actually fail?
Components degrade without obvious failure.
Example:
User sees: Confident wrong answer.
Monitoring challenge: Each component reports "success."
Multi-agent systems can exhibit behaviors that no single agent would.
Example:
Trust challenge: Testing agents individually doesn't catch composition issues.
When something goes wrong, who's responsible?
Example:
Accountability: Distributed across components.
Each agent individually validated.
Requirements:
What this catches: Individual agent failures.
Connections between agents validated.
Requirements:
What this catches: Incompatibilities, format mismatches, coordination failures.
End-to-end system validated as a whole.
Requirements:
What this catches: Emergent issues, composition problems.
Track requests across agent boundaries.
Trace ID: abc123
├── Orchestrator (start)
│ ├── Research Agent (200ms)
│ │ └── External API (150ms)
│ ├── Reasoning Agent (800ms)
│ │ └── Knowledge Base (300ms)
│ └── Action Agent (400ms)
└── Orchestrator (end) [total: 1500ms]Benefits:
Verify that outputs match expected inputs.
# Research Agent output schema
{
"sources": [...],
"summary": "...",
"confidence": 0.85
}
# Reasoning Agent expected input
{
"context": {...}, # Must include sources
"query": "..."
}If Research output doesn't match Reasoning input schema, fail fast.
End-to-end validation with known answers.
Input: "Research quantum computing and explain it simply"
Expected flow:
1. Research → retrieves quantum computing info
2. Reasoning → formulates simple explanation
3. Action → formats response
Expected output: Contains "superposition" or "qubits" in accessible languageIf the chain can't handle this, something is broken.
Allocate trust across components.
System health = 95%
If Research = 98%, Reasoning = 97%, Action = 99%
Combined = 98% × 97% × 99% = 94.1%
Requirement: Each component > 98% to achieve system > 95%One weak link breaks the chain.
Current:
Building toward:
1. Wrap multi-agent calls with monitoring
async def orchestrate(query):
with trace.start("orchestration"):
research = await monitor(research_agent, query)
reasoning = await monitor(reasoning_agent, research)
action = await monitor(action_agent, reasoning)
return action2. Add chain-level evaluation prompts
Don't just test agents. Test the chain:
CHAIN_GOLD_PROMPTS = [
{
"input": "What is 2+2? Research and explain.",
"expected_in_output": "4"
}
]3. Set latency budgets
LATENCY_BUDGET = {
"research": 500, # ms
"reasoning": 800,
"action": 200,
"total": 2000
}Alert if any component exceeds budget.
4. Implement circuit breakers
async def call_with_breaker(agent, input, fallback):
if circuit_breaker.is_open(agent):
return fallback()
try:
result = await agent(input)
circuit_breaker.record_success(agent)
return result
except Exception:
circuit_breaker.record_failure(agent)
raiseTechnical monitoring is necessary but not sufficient.
Who's responsible when the multi-agent system fails?
Anti-pattern: Each team owns their agent. Nobody owns the composition.
Better: Dedicated team owns system-level reliability.
Incident response for distributed systems.
Needed:
Define them at the system level, decompose to components.
System SLA:
Component budgets:
| Component | Availability | Correctness | Latency |
|---|---|---|---|
| Research | 99.9% | 99% | 500ms |
| Reasoning | 99.9% | 98% | 1000ms |
| Action | 99.9% | 99% | 500ms |
Agent-to-agent contracts:
Composition testing:
Trust markets:
Multi-agent systems that are:
We're not there yet. But every step toward component reliability is a step toward system trust.
Don't boil the ocean.
Week 1: Monitor each agent independently
Week 2: Add chain-level evaluation prompts
Week 3: Implement distributed tracing
Week 4: Set latency budgets
Build trust incrementally. The foundation is individual agent reliability.