AgentStatus
We tested every public MCP server we could find. Calling is solved. Delivery is not.
Read the report"What's your agent's uptime?"
It's the first question everyone asks. And it's the wrong question.
Uptime measures whether your agent responds. It doesn't measure whether your agent works.
For AI agents, that distinction is everything.
Traditional uptime monitoring answers: "Did the endpoint return a response?"
For most software, this is sufficient. A payments API either processes the payment or returns an error. A database query either returns data or fails.
AI agents don't work that way.
An agent can return a perfect 200 response that is:
All of these are failures. None trigger uptime alerts.
"Can we reach the agent at all?"
This is traditional uptime. It's necessary but not sufficient.
Catches: Complete outages, infrastructure failures
"Does the agent respond in acceptable time?"
For streaming agents, TTFB is critical-it's when the user sees "typing..."
Catches: Performance degradation, cold starts, upstream slowdowns
"Is the response correctly formatted?"
Catches: Broken response formatting, API changes
"Does the agent actually work?"
This is where AI-specific monitoring begins.
Evaluation prompts are queries with known-correct answers:
If the agent fails these, it's broken-regardless of HTTP status.
Catches: "Up but broken" states, model degradation, auth failures
"Does the agent work everywhere?"
Test from multiple locations:
Catches: Regional outages, CDN issues, geographic rate limiting
Evaluation prompts are the core of semantic validation. Here's how to design them:
Purpose: Verify the agent can think and respond
Characteristics:
Examples:
Pass criteria: Answer must be present somewhere in response
Purpose: Verify the agent follows expected format
Characteristics:
Examples:
Pass criteria: Response must match expected format precisely
Purpose: Verify domain-specific functionality
Characteristics:
Examples (for a support bot):
Pass criteria: Application-specific correctness
| Metric | What It Measures | Target |
|---|---|---|
| Reachability | Can we connect? | 99.9%+ |
| Eval Pass Rate | Does it work correctly? | 95%+ |
| TTFB P50 | Perceived speed | <3s |
| TTFB P95 | Worst-case experience | <10s |
| Metric | What It Measures | Why It Matters |
|---|---|---|
| Regional variance | Geographic consistency | Catches local outages |
| Latency trend | Performance over time | Catches degradation |
| Health vs Contract pass | Reasoning vs format | Isolates failure types |
| Metric | Formula | Meaning |
|---|---|---|
| Effective uptime | Reachability × Eval Pass Rate | True availability |
| SLA compliance | % of checks meeting all thresholds | Overall health |
| Error budget remaining | (Target - Actual) × Period | Deployment capacity |
Raw metrics need interpretation. Here's our framework:
All systems normal.
Something's wrong, but it's working.
Not working.
We use thresholds, not worst-case, for a reason.
A single failed test shouldn't tank your status. Networks are flaky. Devices have issues. Transient errors happen.
What matters is: "Is the agent generally working?"
Threshold-based verdicts answer that question without creating alert fatigue.
| Event | Urgency | Action |
|---|---|---|
| DOWN | High | Wake someone up |
| DEGRADED | Medium | Investigate soon |
| Recovery | Low | Close incident |
| Trend degradation | Low | Investigate proactively |
| Urgency | Channel |
|---|---|
| High | PagerDuty/phone |
| Medium | Slack/email |
| Low | Dashboard only |
Match alert urgency to channel urgency.
No monitoring. Learn about issues from customers.
Ping checks. Know when it's completely down.
Latency tracking. Know when it's slow.
Evaluation prompts. Know when it's broken but responding.
Multi-region tests. Know when it's broken somewhere.
Trend analysis, proactive alerts. Know before it breaks.
Most teams are at Level 1 or 2. The opportunity is Levels 3-5.
You don't need perfect monitoring on day one.
Start small and iterate: one evaluation prompt and TTFB in week one, geographic coverage in week two, pass-rate trends in week three. Each week makes you more reliable than you were before.