AgentStatus × Pydantic
Outside-in monitoring for Logfire, from real residential devices in 70 countries.
You mentioned Pydantic uses Checkly today and finds it a bit minimal, that you'd want an agent that says "what's the state of my system" with click-click-go simplicity. AgentStatus is that, built for AI agents specifically. Logfire instruments what your agents do inside. AgentStatus observes what they do outside.
What we understand about Logfire
OpenTelemetry-native observability is the right inside-out story.
Logfire's internal-trace coverage is excellent: prompt lifecycle, tool calls, distributed tracing. The adjacent question is what those agents look like from outside the application, on real consumer networks, across regions, in multi-turn conversations under semantic pressure.
Internal tracing sees what the agent did from inside its own process. It doesn't see what a user in Mumbai actually got back, or whether voice paths drift differently than chat paths.
What AgentStatus is
What AgentStatus does that Checkly architecturally doesn't.
Checkly is excellent at HTTP and browser synthetic monitoring. We're not pitching replacement. We're pitching the four things they don't cover, that matter for AI agents.
| Capability | AgentStatus | Checkly |
|---|---|---|
| Multi-turn conversational validations | Stateful conversations across turns, including under semantic pressure. | Multistep API checks, independent HTTP requests, no conversation state. |
| Adversarial conformance validations | Gold prompts, semantic-superposition attacks, boundary pressure, scored at per-validate-slot resolution. | JSON path equality, status codes, latency thresholds. |
| Voice channel | Voice agents (ElevenLabs, Retell, Vapi-shaped) on the same pipeline. | HTTP and browser only. |
| Network origin | 2,500+ real residential devices on consumer ISPs, 70 countries. | ~20 cloud data center regions. |
First three are categorical, not implementation, differences.
Outside-in validation is two separate jobs
Reachability
Outcome
Residential path
Your monitor hits the VIP lane. Users hit the WAF. Datacenter checks get blocked, throttled, or allowlisted. Residential observers take the inbound path customers take — so “up” means reachable from home networks, not from AWS.
Reliability
Outcome
Answer quality
Reachable and self-contradicting is still broken. Rephrase flips, drift, and policy breaks need no ground truth. Gold and dual judges cover the rest when truth exists. Uptime grades none of that.
Where we fit
We sit beside the platform. We do not replace it.
What outside-in surfaces
Recent representative findings on production fleets we've operated:
- High-single-digit to ~20% verdict-tier degradation on otherwise-healthy fleets, mostly latency SLA from real consumer networks, invisible from cloud-region validations.
- Semantic-tail concentration: 246 gold-fail rows resolved to 227 failures on one boundary-awareness validate slot. Targeted, fixable. Only legible because evidence preserves per-slot resolution.
- Region-specific failures invisible to centralized telemetry: agents that pass internal evals and cloud-region checks but stay unreachable from specific residential ISPs — an access gap, not a different answer after HTTP 200.
Whether these exist on Logfire is what two weeks would answer.
The two-week parallel run
Point us at Logfire's public endpoints. We run multi-turn validations covering common Logfire flows, gold-prompt and conformance checks, from 5 to 10 residential locations, alongside your existing Checkly setup. Weekly report. No Checkly replacement. No call required to start.
What we need: Logfire endpoint(s), auth method if any, green light for conservative-rate residential validations.
Honest finding at the end: "this is useful, keep going" or "Checkly is sufficient." Either is fine.
The split
How the work divides
How the work divides
Their platform
- • Internal traces of agent execution
- • Span-level visibility into LLM, tool, DB calls
- • OpenTelemetry instrumentation in your code
- • Inside-the-VPC observability
Outcome
System of record
Dashboards, exports, lifecycle tools, and orchestration remain theirs. We do not replace that surface.
AgentStatus
- • External validations of agent behavior on public paths
- • Verdict-tier evidence + conformance outcomes per validate
- • Multi-turn, adversarial validations, no instrumentation required
- • 2,500+ residential devices across 70 countries
Outcome
Outside-in layer
Residential inbound path past CDN/WAF, then gold, consistency, and scoped judges once the agent is reachable.
Proof of scale
Auditable scale metrics
Validations target publicly reachable, customer-visible agent surfaces with conservative rate limits and no attempt to bypass authentication. No tenant data is collected. Retained artifacts are verdict metadata, latency and pass-rate aggregates, short response previews, and structured gold and conformance outcomes, enough to prove behavior, not reconstruct customer records.
What we are not claiming
We are an independent layer that runs alongside your stack.
We are not a replacement for Checkly's general HTTP and browser synthetic monitoring, or for Logfire's inside-out tracing. We are the outside-in layer for AI agents specifically: multi-turn, adversarial, voice-capable, residential.
What we'd like from this conversation
These three asks would move a pilot forward.
Endpoints
Logfire endpoint(s) to monitor, or 'public, no auth, you pick'.
Auth
Auth method if any (bearer / header / none).
Green light
Permission to run conservative-rate validations from residential nodes for two weeks.
Pydantic × AgentStatus.
You said you'd be open to having a look, this is the look. If interesting, reply "go" and we set it up tomorrow.
Contact·dulra@carmel.so·roman@carmel.so
Hypothetical two-week parallel monitoring offer on Logfire's publicly-reachable surfaces. Findings cited are anonymized representative patterns from multi-tenant agent fleets, not specific to Logfire. Validations target publicly-reachable surfaces with conservative rate limits; no tenant data is collected. AgentStatus is independent outside-in production monitoring for AI agents and is not affiliated with Pydantic.