Your Synthetic Monitor Can't Reach
64% of AI Agents

The Study

We tested 6,228 agents from both datacenter and residential IPs.

We ran 1.55 million tests from datacenter IPs and 4.85 million tests from residential consumer devices against the same pool of production AI agents. The datacenter validations used the same IP ranges as Datadog Synthetics, Pingdom, UptimeRobot, and every other traditional monitoring tool. The residential validations came from real consumer devices on home networks across 26 regions.

Both groups used the same evaluation prompts and the same scoring pipeline. Each test produces a transport outcome (reached vs blocked/throttled/empty) and, when a response body exists, can be scored for answer quality. The causal variable this study isolates is network origin. The headline findings are about access asymmetry. Same-reachability gold rates are nearly tied; do not read access failure as origin-caused correctness.

Finding 1

Most AI agents refuse to talk to datacenter IPs.

The most important metric is the rejection rate: how often does the agent see the incoming request and refuse to respond? From a datacenter IP, agents rejected 74.2% of all tests. From a residential IP, that number drops to 22.6%.

The majority of monitoring traffic from datacenter IPs gets rejected before it ever reaches the agent.

Same agents. Different network path.

Datacenter monitors hit the wall most agents put up for cloud IPs.

Datacenter path

74.2%

of requests rejected

67.3%

throttled for sending too many requests

1.2%

fully working (composite; access-dominated)

Outcome

Synthetic “up” is mostly an allowlist illusion.

The majority of monitoring traffic from datacenter IPs gets rejected before it ever reaches the agent.

Residential observers take the inbound path customers actually use.

Residential path

22.6%

of requests rejected

15.8%

throttled for sending too many requests

3.1%

fully working (composite; access-dominated)

Outcome

Reachability is path-dependent.

From a residential IP, rejection drops sharply — because the request looks like a user, not a cloud probe.

If your monitoring tool runs from AWS, GCP, or Azure, most of the agents it tries to test are rejecting it outright. The tool ends up measuring the bot-detection wall your agent puts up, not the agent itself.

Rejection rate by group

Throttle rate by group

How to read the 74% rejection rate

A common reaction is skepticism: if three quarters of datacenter-origin tests were rejected, how are production teams not noticing? The 74% is not “users experience a broken agent 74% of the time.” It is the share of datacenter-origin test attempts that never get a usable agent response — WAF blocks, HTTP 429, auth walls, challenge pages. Residential-origin tests on the same agents were rejected 22.6% of the time. Good teams are not knowingly shipping a 74% user-facing failure rate; they are often measuring from a network position the agent refuses to serve.

Finding 2

Naive correctness rates by origin are access-contaminated — not proof of worse answers.

Early rollups that compared “correct answer rate” across all datacenter probes versus all residential probes showed a gap (on the order of ~7pp in one export). That number is not a same-reachability correctness finding. Blocked and empty datacenter probes fail evaluation by construction. Folding them into a correctness denominator dresses reachability as answer quality.

When both origins actually reached the model, gold evaluation pass rates were nearly identical. The population story this report proves is the access asymmetry in Finding 1. It does not prove that residential IPs make agents smarter, or that datacenter IPs make them give worse answers after HTTP 200.

Finding 3

88% of agents show materially different access depending on where you test from.

We looked at agents where the rejection rate was within 5 percentage points between datacenter and residential — agents that treat both origins roughly equally at the edge. Only 731 out of 6,228 agents met that threshold, which is 12%.

The remaining 88% have materially different rejection profiles by origin. That is an access finding: the wall treats cloud ASNs and residential ASNs differently. It is not a claim that the model returns different gold answers once both sides get HTTP 200.

Why This Happens

AI agents are not websites.

Synthetic monitoring was designed for web pages. Point it at a URL, check if the page loads, measure latency, alert if the server returns an error. That works for static content. It does not work for AI agents.

Agents have bot detection. Your agent sits behind rate limiting, IP reputation checks, and bot protection layers like DataDome, Cloudflare, and HUMAN. A datacenter IP looks like a bot. The edge may block it outright or throttle it aggressively. 74% of datacenter tests were blocked in our dataset. That is reachability, not a different gold answer after HTTP 200.

Answer quality is a separate question. A website either loads or it does not. An agent returns text, and whether that text is correct requires prompts plus evaluation. A successful server response does not mean the answer was right. Our March 2026 report found that 89% of agents with perfect uptime scored 0% on answer quality. Residential vantage helps you retrieve answers to grade on user-class networks; it does not make grading origin-dependent for stably wrong answers.

The coverage gap

Key Takeaways

Four things this data proves.

Methodology

How we collected this data.

This report is based on first-party data from the AgentStatus distributed testing network. No third-party data sources were used.

The datacenter group consists of 1.55 million tests executed from cloud infrastructure IPs, including ranges used by major cloud providers. The residential group consists of 4.85 million tests executed from verified residential consumer devices on home networks across 26 regions.

Both groups tested the same pool of 6,228 production AI agents using identical prompts and the same automated scoring system. Agents were required to have data from both groups to be included in comparative analysis. The rejection rate comparison required at least 20 tests per group per agent. Headline access stats are from the frozen April 2026 cohort export (see backend docs/reports/anti-synthetic-april-2026/). Group A nodes were terminated gaming-farm providers; Group B was all other completed rora jobs in the window — a retrospective cohort, not a live dual-vantage product mode.

Related reading. This report is the controlled population study of datacenter vs residential reachability (rejection, throttle, directional block). The Residentiality Report scopes the architecture: inbound observer identity; tool egress out of scope; same-reachability correctness-by-origin not claimed. Access pairs from this cohort are documented there.