Standardized classification of agent failure modes for consistent analysis and communication.
Purpose
A common error taxonomy enables consistent incident classification, comparable metrics across agents, clear communication about failures, and actionable debugging information. This taxonomy is designed for AI agents, not traditional web services.
Error Categories
| Code Range | Category | Description |
|---|---|---|
| 1xxx | Reachability | Cannot connect to agent |
| 2xxx | Response | Agent responded with error |
| 3xxx | Semantic | Agent responded but incorrectly |
| 4xxx | Performance | Agent too slow |
| 5xxx | Validation | Monitoring infrastructure issue |
1xxx: Reachability Errors
Agent endpoint cannot be reached.
1001: DNS Resolution Failed
Could not resolve hostname to IP address. Common causes include DNS misconfiguration, provider outage, expired domain, or typo in endpoint URL.
1002: Connection Refused
TCP connection rejected by server. Common causes: service not running, wrong port, firewall blocking, or load balancer misconfigured.
1003: Connection Timeout
TCP connection attempt timed out. Check for network routing issues, server overload, firewall dropping packets, or wrong IP address.
1004: TLS Handshake Failed
SSL/TLS connection could not be established. Check certificate expiration, hostname mismatch, TLS version incompatibility, or untrusted CA.
1005: Host Unreachable
Network indicates host cannot be reached. Server may be down, network segment unavailable, or routing misconfigured.
2xxx: Response Errors
Agent responded but indicated an error condition.
2001: Authentication Failed (401)
Agent rejected authentication credentials. Invalid API key, expired token, or suspended account.
2002: Authorization Failed (403)
Authenticated but not permitted. Insufficient permissions, IP not allowlisted, or rate limit policy block.
2003: Resource Not Found (404)
Requested resource does not exist. Endpoint URL changed, resource deleted, or version mismatch.
2004: Rate Limited (429)
Too many requests, temporarily blocked. Exceeded rate limit or shared quota depleted.
2005: Server Error (500)
Agent returned a server-side error. Unhandled exception, dependency failure, resource exhaustion, or bug in agent code.
2006: Service Unavailable (503)
Agent temporarily unavailable. Maintenance mode, overloaded, or deployment in progress.
2007: Bad Gateway (502)
Upstream service failure. Upstream timeout, upstream crashed, or proxy misconfiguration.
2008: Gateway Timeout (504)
Upstream service did not respond in time. Upstream overloaded, hung, or timeout too short.
2009: Invalid Request (400)
Request was malformed or invalid. Malformed JSON, missing required fields, or encoding issues.
3xxx: Semantic Errors
Agent responded successfully (2xx) but answer was incorrect or unusable.
3001: Gold Prompt Failed - Health Tier
Basic reasoning question answered incorrectly. Example: "What is 2+2?" not answered with "4". Model may be misconfigured, context corrupted, or quality degraded.
3002: Gold Prompt Failed - Contract Tier
Format/schema requirement not met. Asked for JSON, got plain text. System prompt may be altered or parser broken.
3003: Empty Response
Agent returned empty or effectively empty response. Model returned nothing, filtering removed content, or streaming interrupted.
3004: Error Message in Response
Agent returned 200 OK but body contains error message - "I cannot help with that" or "Error: Model unavailable." Safety filter triggered or upstream dependency failed.
3005: Incoherent Response
Response is present but makes no sense. Garbled text, unrelated to query, or completely hallucinated. Context may be corrupted or wrong model loaded.
3006: Schema Validation Failed
Structured response doesn't match expected schema. JSON structure wrong, missing required fields, or wrong data types.
4xxx: Performance Errors
Agent responded but performance was unacceptable.
4001: TTFB Exceeded
Time to first byte exceeded threshold. Cold start, model loading, or queue wait time.
4002: Total Latency Exceeded
Total response time exceeded threshold. Large response, complex processing, or slow dependencies.
4003: Response Timeout
Response did not complete within timeout. Server hung, infinite loop, or massive response.
4004: Streaming Interrupted
Streaming response terminated prematurely. Server crashed mid-response or network interruption.
5xxx: Validation Errors
Issue with monitoring infrastructure, not agent. These are excluded from availability calculations.
5001: Test Network Error
Test could not reach any network. Problem with monitoring node, not indicative of agent status.
5002: Test Configuration Error
Test misconfigured. Invalid settings, can't execute check properly.
5003: Validation Logic Error
Error in validation logic itself. Bug in monitoring code.
5004: Test Timeout
Test itself timed out. Monitoring system issue.
5005: Insufficient Data
Not enough data to make verdict. Too few tests completed.
Error Attribution
All 1xxx, 2xxx, 3xxx, 4xxx errors are attributed to the agent and count against availability. All 5xxx errors are attributed to monitoring infrastructure and excluded.
| Situation | Attribution | Reasoning |
|---|---|---|
| Single timeout, others pass | Agent | Could be legitimate slow request |
| All tests timeout | Investigate | Could be infrastructure |
| One region fails | Agent | Regional agent issue |
| All regions fail | Investigate | Could be infrastructure |
