All articles

Agent Status / Field Notes

Geographic Validation: Why Your Agent Works in SF But Fails in Singapore

True story: A team's agent worked perfectly for six months. Then they expanded to Asia.

Support tickets from Singapore users started rolling in. The agent was "slow" or "unresponsive." Engineers couldn't reproduce it from their San Francisco office.

The agent worked perfectly from SF. It was nearly unusable from Singapore.

Their monitoring showed 99.9% uptime. Because their monitoring only checked from US data centers.


01Section

The Physics of Latency

Let's start with basics.

Speed of light: ~300,000 km/s

SF to Singapore: ~13,500 km

Minimum round trip: ~90ms (just physics)

Add in:

  • Routing overhead: +50-100ms
  • TLS handshake: +30-50ms
  • Multiple hops: +20-50ms
  • Actual processing: +variable

Real-world SF → Singapore latency: 200-500ms just to establish connection.

Now add your agent's inference time on top.

An agent that responds in 1 second from SF takes 1.5-2 seconds from Singapore. That's the best case.


02Section

Why Monitoring Misses This

Traditional monitoring architecture:

snippet
[Your US Server] ←→ [Monitoring from US Datacenter]
                          ↓
                    "100% uptime!"

This tests: "Does our agent work from a nearby, well-connected data center?"

It doesn't test: "Does our agent work for actual users?"

Your users aren't in data centers. They're on:

  • Residential connections
  • Mobile networks
  • Corporate firewalls
  • ISPs with weird routing
  • Locations far from your servers

03Section

Real-World Failure Modes

CDN Edge Issues

Your CDN has points of presence (POPs) worldwide. Each is independently configured.

Failure: Singapore POP has stale cache or misconfiguration.

Result: Works in US, fails in Singapore.

Your US monitoring sees: Nothing.

DNS Propagation

You update DNS. It propagates at different speeds globally.

Failure: Asia-Pacific DNS servers still have old record.

Result: Works in US (new IP), fails in APAC (old IP).

Your US monitoring sees: Nothing.

ISP Routing

Some ISPs have poor peering with your hosting provider.

Failure: NTT (major Asian ISP) routes your traffic through congested path.

Result: Works for 90% of users, fails for users on NTT.

Your US monitoring sees: Nothing.

Geographic Rate Limiting

Some providers apply rate limits by region.

Failure: You hit OpenAI's regional rate limit for European IP ranges.

Result: Works in US, rate limited in EU.

Your US monitoring sees: Nothing.

Regional Blocking

Firewall rules or WAF configurations can block by geography.

Failure: Security rule accidentally blocks Asian IP ranges.

Result: Works everywhere except Asia.

Your US monitoring sees: Nothing.


04Section

The Physics Compounds

Geographic latency is not just a convenience issue-it directly impacts reliability.

The HB-Eval research shows that agents are sensitive to perturbations, dropping from 86.9% to 44.0% success rates under stress. Network latency adds timeout pressure that compounds these failure modes.

Example latency progression (US-hosted agent):

RegionBase LatencyWith LLM InferenceUser Experience
US East~50ms~1-2s totalAcceptable
US West~100ms~1.5-2.5s totalAcceptable
EU~150-200ms~2-3s totalNoticeable
APAC~200-300ms~3-5s totalFrustrating
Africa~300-400ms~4-6s totalOften timeouts

Key insight: As latency increases, timeout rates increase-and when agents face network perturbations, their already-stressed architectures fail more frequently.

Regional Detection Gap

When regional issues occur, how long until detection?

Monitoring TypeDetection Time
Single-location monitoringHours to days (customer report)
Geographic monitoring<15 minutes

The difference is whether you find it or your users do.


05Section

Why Residential IPs Matter

Data center IPs often get special treatment at the edge:

  • Different peering and routing than consumer ISPs
  • ASN reputation that bot management and WAFs score aggressively
  • Rate limits tuned against cloud ranges
  • Latency that does not match last-mile user paths

Residential IPs (actual user connections) experience:

  • Standard ISP routing
  • The same bot / WAF policy class as your users
  • Real last-mile latency and packet loss
  • Geo defaults the harness applies from the connecting client IP

Testing from data centers often measures a different access surface — whether the edge lets the probe in, and what latency the path shows. It does not relocate the agent's outbound tools. For reachability, geo-block, and client-IP locale branches, residential vantage is the honest observer. For stably wrong answers after HTTP 200, you still need semantic evaluation.


06Section

Implementation

What Agent Status Does

  1. Global node network: Real devices in 70 countries
  2. Residential IPs: Actual user network conditions
  3. Per-region verdicts: US might be UP while APAC is DOWN
  4. Regional latency tracking: P50/P95/P99 per region
  5. Regional alerting: Know which region failed

What You See

snippet
{
  "overall_verdict": "DEGRADED",
  "by_region": {
    "us": {"status": "UP", "ttfb_p50": 420, "pass_rate": 100},
    "eu": {"status": "UP", "ttfb_p50": 890, "pass_rate": 98},
    "apac": {"status": "DOWN", "ttfb_p50": null, "pass_rate": 15}
  }
}

Immediately clear: US and EU are fine, APAC is broken.


07Section

Geo Context Injection

For agents that should give location-aware responses (weather, local info, search):

snippet
[User location: Singapore]
What's the weather like today?

We prepend the test's location to verify whether your agent gives appropriate regional responses.


08Section

Common Patterns

"Home Region Works, Everything Else Slow"

Pattern:

  • US: 500ms TTFB
  • EU: 1,500ms TTFB
  • APAC: 3,000ms TTFB

Cause: No edge deployment. All traffic routes to origin.

"One Region Down, Others Fine"

Pattern:

  • US: UP
  • EU: UP
  • APAC: DOWN

Cause: Regional infrastructure issue (CDN edge, DNS, routing).

"Intermittent Regional Failures"

Pattern:

  • Sometimes APAC fails
  • No consistent pattern
  • Monitoring shows flaky behavior

Cause: Capacity issues, rate limiting, or network instability.

"Latency Increases Over Time"

Pattern:

  • APAC TTFB: 1,500ms (January)
  • APAC TTFB: 2,200ms (February)
  • APAC TTFB: 3,100ms (March)

Cause: Growing traffic, degrading infrastructure, or routing changes.


09Section

Best Practices

1. Monitor From Where Your Users Are

Not where your servers are. Not where your office is. Where your users are.

If 30% of users are in EU, have 30% of monitoring from EU.

2. Set Regional SLAs

Different regions may have different acceptable thresholds:

RegionMax TTFBMin Pass Rate
US2s95%
EU3s90%
APAC5s85%

Be realistic about physics.

3. Alert on Regional Degradation

Don't just alert on global DOWN. Alert when specific regions degrade.

4. Use Residential Where Possible

Data center checks are supplementary. Residential checks are primary.

5. Track Regional Trends

A region getting slowly worse is a signal to investigate before it becomes an outage.


10Section

The ROI

Cost of regional monitoring: ~$20/month for global coverage.

Cost of 12-hour regional outage discovered by customer complaints:

  • Lost revenue: $X
  • Support costs: $Y
  • Reputation: Priceless (in the bad way)

The math isn't close.

Independent monitoring

See your agent the way the world sees it.

Outside-in validations from real residential nodes, evaluation prompts that catch silent-200 failures.