How we rank public MCP servers from the outside.
Residential dual-era probes, a published scoring formula, Spec readiness for the modern MCP path, and explicit limits on what the score claims. See the live scores for current rankings.
What the Score measures
User-side MCP reliability, not a catalog score.
The Public MCP Score measures public MCP servers the way a client would: negotiate the protocol dialect, discover tools, invoke them, and record whether the path held up from a residential network — not a datacenter ping.
This is not a popularity list and not a lab capability benchmark. It is continuous validation of production MCP surfaces, scored with public math.
See also MCP validation for the product surface, and the live scores for current rankings.
Reach the MCP endpoint
Confirm the server is reachable from a residential origin.
Negotiate dual-era
Modern 2026-07-28 path first (server/discover + per-request meta); legacy initialize/session if required.
Discover tools
tools/list (+ resources & prompts when exposed) on the negotiated path.
Invoke tools + score results
Call safe read tools (and schema-linked workflows when the catalog supports them), then score invocation vs functional outcomes separately.
Aggregate into 30d Score inputs
Roll per-run evidence into the public composite score, Spec readiness, and protocol-era badges.
Why user-side
What datacenter checks miss.
MCP servers often look healthy from a datacenter laptop and fail for the clients that actually matter: agents and apps on residential or carrier networks, behind real DNS and TLS paths, with real latency tails.
Geo / IP filtering
Transport fragility
Tool timeouts
Partial discovery
Protocol-era skew
Score probes run on the same distributed residential network used for AgentStatus agent validation. The backend orchestrates; nodes execute the workloads.
Protocol eras
Modern, legacy, dual — and Spec readiness.
MCP clients are mid-migration. A server that only answers the legacy initialize + Mcp-Session-Id path will fail modern clients; a server that only answers the stateless 2026-07-28 path will fail older SDKs. We negotiate both and publish which dialect worked.
Modern (2026-07-28)
Legacy (session)
Dual-era
Broken / split
Spec readiness is a separate 0–100 migration score from the dual-era probe (modern path, server/discover, header routing, list-cache hints). Filter the Index by protocol era to cut the leaderboard to modern-only, legacy-only, dual, or broken servers.
Paid AgentStatus monitors can probe both eras every cycle (Growth+) to catch asymmetric deploys. The public Index always negotiates modern-first with legacy fallback so every listed server is measured on a comparable path.
Signals
What each column answers.
Each public column answers one plain question — and has an explicit non-claim:
Uptime (30d)
What share of MCP validation runs reached the server?
Latency P95
How slow were the slowest successful probes from residential origins?
Tool success
Did tool calls invoke cleanly and look functionally correct?
Protocol conformance
After dual-era negotiate, did transport, discovery, and invocation behave like MCP?
Score
Score calculation.
The Score is a composite metric from 0 to 100:
reachable_runs / total_mcp_runs over the last 30 days. Run-based reachability — not a continuous synthetic heartbeat.
P95 response time mapped to 0–100. Perfect score at ≤200 ms; linear decay to 0 at ≥2000 ms.
invocation_rate × 0.6 + functional_rate × 0.4. Falls back to whichever rate is available if only one exists.
Average per-run score after dual-era negotiate: reachable 30% + transport 25% + discovery 25% + invocation 20%. Missing invocation evidence renormalizes the rest.
Verdict bonus (ranking math)
Operational
Impaired
Down
If a component is missing, remaining weights renormalize. The verdict bonus is still applied. Final score is capped at 100. Spec readiness is displayed alongside the Score and is not folded into these weights.
Displayed vs Rank
Why the drawer and the table can disagree.
The Index shows two scores for the same server. That is not a bug — thin evidence is not allowed to dominate the leaderboard.
Displayed Score
Rank Score
Confidence shrinkage (rate components only)
For uptime, tool success, and protocol conformance (each as a 0–1 rate p with n runs), Rank Score first replaces p with a blend toward a neutral prior. Latency P95 is not shrunk — the measured curve stays.
n₀ = 7 · prior p₀ = 0.5 (50%) · Low sample when n < 7
Then Rank Score = weighted(p_rank rates, unchanged latency score) + same verdict bonus. As n grows, p_rank → p and the two scores converge.
Worked example (n = 2 runs)
Suppose Reach, Tool, and Spec are all 100%, P95 latency is 1575 ms (latency score ≈ 23.6), and status is Operational (+5).
| Step | Math | Result |
|---|---|---|
| Displayed base | 0.35×100 + 0.25×23.6 + 0.25×100 + 0.15×100 | 80.9 |
| Shrink each 100% rate | (2/9)×1.0 + (7/9)×0.5 | 61.1% |
| Rank base | 0.35×61.1 + 0.25×23.6 + 0.25×61.1 + 0.15×61.1 | 51.7 |
| Rank Score | 51.7 + 5 (Operational) | 56.7 |
Drawer can show ~80.9 while the table sorts on 56.7. The subtitle “Rank uses confidence-adjusted …” is that Rank Score — not a second mystery metric.
Ranking rules
Eligibility, ranking & sample size.
Indexed
Hidden
Ranked
Displayed Score
Rank Score
Low sample
Unrated
Identity
Status chip
Protocol era filter
Freshness
Cadence & data freshness.
| Surface | Freshness |
|---|---|
| Validation runs | Per monitor schedule (public MCP fleet commonly ~6h; configurable down to 5 min) |
| Live leaderboard | Computed from the latest 30-day window of MCP validation results |
| Trend delta | Vs yesterday’s daily Score snapshot (00:15 UTC). Null until day 2 of snapshots |
| Validation runs counter | Real 30-day run count for indexed servers — not a marketing session total |
The split
What we are, and are not.
We are not
- • A paid placement directory
- • A full MCP spec certification body
- • An inside-out APM / tracing product
We are
- • An independent user-side validator
- • A public reliability ranking with published math
- • A dual-era client: modern MCP first, legacy when required
- • The client’s perspective at residential network scale
FAQ
Common questions.
| Question | Answer |
|---|---|
| Is this a lab benchmark of MCP servers? | No. We probe production MCP endpoints from residential nodes on a schedule, the same network class a real client would use. It scores reliability in the wild, not capability in a controlled room. |
| What goes into the score? | Four weighted signals plus a small verdict bonus: uptime (35%), latency (25%), tool success (25%), and protocol conformance (15%). Missing components renormalize the remaining weights. The formula is public. |
| What is dual-era negotiation? | MCP clients are mid-migration. We try the modern 2026-07-28 path first (stateless server/discover + per-request meta, no sticky Mcp-Session-Id), then fall back to legacy initialize/session when required. Spec readiness and protocol-era badges report which dialect actually worked. |
| What is Spec readiness? | A 0–100 migration score from the dual-era probe: modern path, server/discover, header routing, and list-cache hints. Separate from the composite Score — it answers “how ready is this server for modern MCP clients?” |
| Why does status say “Modern only — legacy fails” instead of Degraded? | When we know the cause, the public Status chip names it: modern-only, legacy-only, multi-edge protocol split, auth required, unreachable, and similar. Internal severity can still be degraded for ranking math; the label is never opaque when the probe classified the failure. |
| How is uptime measured? | Run-based reachability over the last 30 days: reachable MCP validation runs divided by total MCP validation runs for that server. This matches the monitor’s schedule (public MCP monitors are commonly ~every 6 hours; paid agents can schedule down to 5 minutes), not a fake continuous heartbeat. |
| What does tool success mean? | A blend of invocation success (did the tools/call complete with a well-formed MCP result?) at 60% and functional success (did the result look correct for the tool’s purpose?) at 40%. If only one signal exists, we use that one. The default public index path exercises safe read tools with synthesized arguments — not a full docs/contract assertion suite unless an agent is configured for deeper probes. |
| What is protocol conformance? | Per-run adherence to the MCP surface we exercise after dual-era negotiation: reachability, transport support, tool discovery, and invocation. Those pieces are weighted 30% / 25% / 25% / 20% and averaged across runs. It is not a full optional-feature certification of every MCP extension. |
| Why does the drawer show one Score and the table another? | They are two intentional numbers. Displayed Score (drawer hero) is the raw composite from measured rates. The table Score — and sort order — use Rank Score: the same formula after confidence shrinkage on rate components (uptime, tool success, protocol conformance) toward a 50% prior with strength 7. Latency is never shrunk. Under 7 runs the gap is large on purpose; at high sample the two converge. |
| Why are some servers marked Low sample? | Fewer than 7 validation runs in the scoring window. They still get a Displayed Score. Leaderboard position uses Rank Score so a lucky 2-run streak cannot outrank a proven fleet. Worked math is in Ranking rules below. Unrated means zero scored runs. |
| Can a server pay for a better rank? | No. Rankings come from measured results only. Paused monitors are hidden from the public index entirely. |
| Do you publish agent UUIDs? | No. The public index uses display names and stable slugs only. |
Independence
Trust is measured, not purchased.
The Public MCP Score is operated by Carmel Labs / AgentStatus as an independent validator. Rankings are determined by measured validation results. Server operators cannot pay for a better score.
The only way to move up is to improve reachability, latency, tool behavior, protocol conformance, and dual-era readiness under user-side probes.
