Continuous monitoring + validation, user-side.
Reachability from real devices. Outcome verification once reached — scenario, compositional, safety, and stability checks. Tool validating. MCP discovery. Multi-turn testing.
Architecture
Built from the inside out
At the core, 2,500+ devices across 70 countries. Agent Status on top. Your agent on the outside.
AGENTS
Any HTTP endpoint
OpenAI, Anthropic, LangChain, custom APIs
ALERTS
Instant notifications
Slack, PagerDuty, email, webhooks
PLATFORM
Gold prompt eval
Correctness checks, schema validation, attribution
PLATFORM
Verdicts & dashboards
UP / DEGRADED / DOWN with root cause
FABRIC
Residential tests
Real consumer devices, 43 countries, 892 ISPs
FABRIC
Crypto-verified
Ed25519 signed, hash chain, no proxy
Your dashboard can be completely fine.Your users can still get wrong answer.
The most expensive failures in production AI today are the ones every existing monitoring tool would mark green. The server responded, the latency was good, the error rate was zero - and the user was lied to.
- HTTP status
- 200 OK
- p95 response latency
- 284 ms
- Error rate (5xx)
- 0.02 %
- Token throughput
- 432 / s
- Model availability
- 99.99 %
- Eval set pass@1
- 94.2 %
Hey, I bought the wrong size - can I return it for store credit?
The observability era sees a successful request. The customer sees a fabricated refund. Both are correct.
There are six ways an AI agent can be broken. Most tools check one.
The system responds at all.
The output is logically valid given the inputs.
The output matches a ground-truth reference.
Performance holds under adversarial or out-of-distribution inputs.
The same input produces the same (or equivalent) output.
The joint property - the system does the right thing repeatedly under conditions it has not seen before.
Note. Each of the first five rows is independently measurable and can be maximized in isolation. Only the last row composes all five, which is why benchmark "passes" so often coexist with shipped-product failure.
Validation Depth
Eight ways to validate.
Zero code changes.
Every capability is live in production today. Configure per-agent via API or dashboard.
Adapts validation per agent type: code executors, orchestrators, artifact creators.
Tests multi-turn conversations, validating quality across turns.
Triggers tool calls and verifies the agent used the right tools.
Queries tools/list, resources/list, then executes tools/call.
GPT-4o-mini evaluates response quality with custom criteria.
Prepends user location to prompts. Catches geo-divergent behavior.
Compares claimed SLAs to actual performance. Breach alerts.
Measures TTFB, chunk timing, and stream completion.
Adapts validation per agent type: code executors, orchestrators, artifact creators.
Tests multi-turn conversations, validating quality across turns.
Triggers tool calls and verifies the agent used the right tools.
Queries tools/list, resources/list, then executes tools/call.
GPT-4o-mini evaluates response quality with custom criteria.
Prepends user location to prompts. Catches geo-divergent behavior.
Compares claimed SLAs to actual performance. Breach alerts.
Measures TTFB, chunk timing, and stream completion.
One right answer doesn't make the whole chat reliable.
A pass rate measured one turn at a time tells you nothing about what a real user experiences over a real conversation. The benchmark you ship is the wrong number to ship.
Session reliability
One customer · 10 messages · live
The score
messages went right.
54 out of every 100 customers get a conversation this clean.
How often the agent gets a single message right
94%A per-turn pass rate is a benchmark. Reliability is what the user feels by message ten.
Note. Math assumes independent turn failures. In production, failures correlate within a session: a hallucinating agent tends to keep hallucinating, a degraded retrieval tends to keep degrading. The real session pass rates are lower than the curve above.
By agent type
Chat, voice, and MCP. One platform.
Different surfaces your agent exposes. Same simple dashboard.
Integrations
Works with everything you already use.
FAQ
Common questions
| Question | Answer |
|---|---|
| What's the difference between chat and voice validation? | Chat tests text and API agents. Voice places real test calls from home networks. Same dashboard, same plain verdicts. |
| What are consistency checks? | We ask the same thing in different ways and flag answers that don't hold together. You don't need a perfect script for every possibility. |
| Do I need to install anything? | No. Point us at your agent or tell us how to reach it. Tests run from the outside. |
| How is this different from Datadog or LangSmith? | They watch from inside your stack. We test from where your users are - and tell you if answers are actually good. |