All articles

Agent Status / Field Notes

What is outside-in monitoring (and why inside-out isn't enough)?

If you run an AI agent in production, you already know the uncomfortable truth: your dashboards are almost always green while your users are having a bad time. Traces look clean. Token counts are normal. The health check passes. And somewhere in the middle of a conversation, your agent stalls for eight seconds, returns an empty object, or confidently hallucinates a tool call that never should have fired. You find out about it when someone files a ticket.

That gap - between what your observability tells you and what your users actually experience - is the problem outside-in monitoring is built to solve.

01Section

What "inside-out" monitoring actually measures

Every tracing and observability tool you've ever used is inside-out. OpenTelemetry, Datadog APM, Langfuse, Helicone, Phoenix - they all instrument your code from inside the process and report what the process saw. That data is real and valuable. You get:

  • Server-side latency for each model call and each tool call.
  • Token usage and cost per request.
  • Traces showing the tree of calls in a multi-step reasoning chain.
  • Error logs when something throws inside your code.
  • Retry counts, queue depths, and internal state.

That's the view from your side of the wire. It's what the agent thought it did.

02Section

Why that misses the point for AI agents

Three things about AI agents make inside-out insufficient on its own.

1. The model provider is a third party

When OpenAI, Anthropic, or your inference provider has an outage, a degraded region, or a rate-limit quirk, your own code is still running fine. Your process metrics look healthy. Meanwhile every user request is timing out on the other side of the HTTP call. Inside-out tools can log that a request took 30 seconds or threw, but they can't tell you whether that's your problem or the provider's, or whether it's affecting users in London but not New York.

2. The experience depends on geography

An agent call is rarely one HTTP request. It's a model call plus maybe a vector search plus a tool invocation plus sometimes a second model call to assemble the answer. Every leg has its own network path. A user in Singapore and a user in Toronto can hit your agent at the same second and have completely different experiences. Inside-out metrics are blind to this - they only see what hits your servers.

3. Output quality is part of uptime

When an agent returns a response that's technically valid but semantically wrong - a tool call that doesn't parse, an answer that ignores the question, a JSON blob missing a required field - every inside-out tool reports success. The process returned a 200. The trace closed cleanly. But from the user's side, the agent just failed. Inside-out cannot tell you that the quality of what you shipped through the wire was bad.

03Section

What "outside-in" monitoring does differently

Outside-in monitoring treats your agent like a black box and validations it the way a user would. From machines that sit outside your stack, at real geographic locations, it:

  • Fires real requests at your live agent endpoint, on a schedule.
  • Measures end-to-end latency, including DNS, TLS, and the full round-trip to the model provider.
  • Parses the response and runs assertions on it - is the JSON valid, does the answer contain the expected keys, is the tool call formed correctly?
  • Reports from multiple regions so you see geographic variance, not just server-side averages.
  • Alerts you before your users file a ticket, because the monitor is the first user on every new deploy.

The shorthand is: inside-out tells you what your system thinks happened. Outside-in tells you what actually happened to the user.

04Section

A concrete example

Picture a common scenario we see in our data. An inside-out dashboard shows an average completion latency of 1.8 seconds - well inside the team's SLO. An outside-in validate in South America catches a 14-second spike that's only happening to users routed through that region. The root cause is usually one of a small set: a model-provider PoP with an intermittent degradation, a regional vector-DB timeout, or a CDN routing bug. The team's own servers never see it, because none of their app traffic hits that PoP directly. Their users do.

Without the outside-in validate, the team would have learned about it when a customer complained. With it, they would have had an alert before the first user noticed. For more on why TTFB, not total latency, is the number to watch, see our explainer on TTFB vs latency.

05Section

How to add outside-in to what you already have

Outside-in monitoring is not a replacement for your existing observability stack. It's a complement. The setup is small:

  • Pick one public endpoint per agent to validate. A dedicated /health-agent route that exercises the same code path users hit is ideal.
  • Define 2–4 assertions on the response: JSON parseability, presence of required fields, a semantic check (e.g., response length within bounds), optionally a content match.
  • Configure checks from at least three regions: one near your primary user base, one far away, one at a major cloud region for parity.
  • Set the cadence tight enough to catch incidents early. Agent Status's paid tiers support intervals as short as 5 minutes; free tier runs every 6 hours. Pick the tightest cadence your test budget allows - the shorter the interval, the faster you'll hear about a regression.
  • Route alerts to the same channel as your existing paging, so everything lives in one place.

For framework-specific walkthroughs, see our guides for LangChain, LlamaIndex, and CrewAI - each one has assertions tailored to that framework's output shape.

06Section

When inside-out is still enough

If your agent is internal-only, low-traffic, and has no geographic distribution in its user base, inside-out can carry you. If it's customer-facing, if it depends on third-party models, or if a minute of silent failure costs you anything, you want outside-in running alongside.

07Section

The bottom line

Every mature production system eventually ends up running both. Inside-out tells you why something broke. Outside-in tells you that something broke, and tells you first. For AI agents, where the model provider is a third party, geography matters, and output quality is part of uptime, outside-in is not optional.

Agent Status runs outside-in monitoring for AI agents. It takes about 10 minutes to set up and the free tier covers 30 tests/month on 1 agent from 3 regions - enough to validate the pattern against your real traffic. If you want to see what your users are actually getting from your agent, start there.

Independent monitoring

See your agent the way the world sees it.

Outside-in validations from real residential nodes, evaluation prompts that catch silent-200 failures.