AgentStatus
We tested every public MCP server we could find. Calling is solved. Delivery is not.
Read the reportLatency is the single variable that separates an AI agent users tolerate from one they abandon. The numbers below are drawn from representative user-side measurements against the public endpoints of eight major model providers from three geographic regions. They are illustrative - provider performance changes week to week, and the only numbers you should make product decisions on are the ones from validations running against your own traffic. What holds across runs is the shape of the problem: p50 vs p99 gaps, geographic variance, and why TTFB, not total latency, is the metric to watch.
Methodology is reproducible; the sample numbers in the table below are a representative snapshot rather than a live feed. Run the same validations against your own endpoints to get numbers that reflect your traffic.
We want other teams to be able to reproduce or replace this benchmark. The setup:
Every provider received the same 120-token prompt: a deterministic question about a JSON schema with a 40-token expected response. This keeps output length comparable and avoids measuring how long each provider takes to generate a 2,000-token essay.
Representative TTFB p50 / p99 by provider, averaged across three regions (milliseconds). Use these as a shape, not a source of truth - the absolute numbers are snapshot-dependent.
| Provider | Model | TTFB p50 | TTFB p99 | Total p50 | Err rate |
|---|---|---|---|---|---|
| Groq | llama-3.3-70b | 180 | 2,200 | 780 | 0.12% |
| OpenAI | GPT-4o | 420 | 3,800 | 1,450 | 0.08% |
| Anthropic | Sonnet 4.6 | 510 | 2,900 | 1,720 | 0.04% |
| Fireworks | qwen2.5-72b | 540 | 4,200 | 1,910 | 0.21% |
| Together | llama-3.3-70b | 620 | 5,400 | 2,100 | 0.34% |
| Mistral | mistral-large | 690 | 4,700 | 2,240 | 0.18% |
| Cohere | command-r-plus | 740 | 3,900 | 2,310 | 0.11% |
| Gemini 2.5 Pro | 820 | 6,100 | 2,580 | 0.27% |
If a provider has a great p50 and a terrible p99, one in every hundred user interactions with your agent is going to feel broken. For a conversational agent making 20 calls per session, that's 20% of sessions with at least one bad call. Always tune on p99.
Some providers have strong presence in every major cloud region; others are effectively single-region behind a CDN. If your users are global, the provider's p50 from the other side of the planet is what they experience, not the provider's marketing benchmark.
Total latency depends on output length, which varies by prompt. TTFB doesn't. If you're trying to catch provider degradation in real time, watch TTFB - it's the first thing to move when something is wrong on their side. For more on the distinction, see our TTFB vs latency explainer.
Most providers stay under 0.5% over a month. What you actually care about is whether those errors cluster - if you get 200 errors at the same minute, you have an incident, not an error rate. Always alert on change, not on absolute thresholds.
If you're starting fresh and can pick any provider, the heuristic we'd use - from this data alone - is:
And in all cases: monitor user-side. These benchmark numbers are a snapshot. Provider performance changes week to week, especially when new models launch. If you don't have an user-side validate running, you won't know until your users do.
Every tool we used is public. The validate scripts, assertion configs, and regional vantage points are what Agent Status runs for every customer by default. If you want to run this against your own endpoints - including ones behind auth - you can set up the same validations on your own agent in about 10 minutes. Framework-specific starting points: LangChain, Anthropic SDK, OpenAI Assistants.
We plan to publish updated numbers quarterly as we expand our public dataset. If there's a specific provider or model you want us to add, let us know. In the meantime, if you want live per-provider dashboards, see OpenAI status monitoring and Anthropic status monitoring.