Thoughts

Evaluation Prompts: How We Detect "Up But Broken" Agents

Here's a question that traditional monitoring can't answer:

"Is my agent working correctly?"

Not "is it responding?" Not "is it fast?" Is it actually, semantically, doing what it's supposed to do?

This is the problem evaluation prompts solve.


The Problem

An AI agent returns HTTP 200. The response is valid JSON. Latency is 1.2 seconds.

Monitoring says: everything's fine.

But the response is: "I'm sorry, I cannot help with that request. Please try again later."

To every single query.

The agent is "up." It's also completely broken.

We call this state "up but broken," and traditional monitoring is blind to it.


What Are Evaluation Prompts?

Evaluation prompts are queries with known-correct answers.

The concept comes from machine learning, where "gold data" or "gold standard" refers to labeled examples used as ground truth.

Applied to agent monitoring:

Gold PromptExpected Answer
"What is 2+2?"Contains "4"
Return JSON: {"x": 1}Valid JSON with x: 1
"Echo exactly: hello"Contains "hello"

These are deliberately simple. Any functioning LLM can answer them. If your agent can't, something is very wrong.


Why This Works

Principle 1: Determinism

The key property of evaluation prompts is determinism. There's exactly one correct answer category, and it's obvious.

This allows automated verification:

def verify_gold_prompt(response, expected):
    return expected.lower() in response.lower()

No human judgment required. No ambiguity. Pass or fail.

Principle 2: Simplicity

Evaluation prompts are trivially easy. This is intentional.

If we used complex prompts, failures could be:

  • Legitimate capability limits
  • Edge case behavior
  • Interpretation differences

With trivial prompts, failures mean one thing: something is broken.

Principle 3: Coverage

Different evaluation prompts test different failure modes:

Prompt TypeTests For
MathBasic reasoning works
EchoInstruction following works
JSONStructured output works
KnowledgeModel is loaded/responding

A comprehensive gold prompt suite catches most failure modes.


The Gold Prompt Taxonomy

Health Tier

Purpose: Verify basic functionality.

Characteristics:

  • Any reasonable LLM can answer
  • Loose matching (answer anywhere in response)
  • Tests core reasoning capability

Examples:

Prompt: "What is 2+2?"
Pass: "4", "The answer is 4", "2+2 equals 4, which is..."
Fail: "I cannot answer that", "Error", "5"
Prompt: "What color is the sky on a clear day?"
Pass: "blue", "The sky is blue", "Blue, typically"
Fail: "I don't know", "Red", "[Error message]"

What failures indicate:

  • Model not responding properly
  • Authentication issues returning error messages
  • Rate limiting returning polite refusals
  • Model degradation

Contract Tier

Purpose: Verify format compliance.

Characteristics:

  • Tests structured output capability
  • Strict matching (exact format required)
  • Tests what API consumers actually need

Examples:

Prompt: "Return JSON: {\"status\": \"ok\"}"
Pass: {"status": "ok"}
Fail: "Here's the JSON: {\"status\":\"ok\"}", {"status": "error"}
Prompt: "Respond with exactly one word: yes or no"
Pass: "yes", "no"
Fail: "Yes, I can help", "Yes" (with punctuation issues)

What failures indicate:

  • Structured output broken
  • Prompt template corrupted
  • Model version changed
  • Function calling broken

Behavioral Tier

Purpose: Verify expected agent behavior.

Characteristics:

  • Tests agent-specific functionality
  • May require configuration
  • Tests what your users need

Examples (for a customer support agent):

Prompt: "What is your company's name?"
Pass: "[Your company name]"
Fail: Generic response, "I don't know"
Prompt: "I need to return a product"
Pass: Contains return policy or starts return flow
Fail: "I cannot help with returns"

What failures indicate:

  • RAG system broken
  • Tool integration failed
  • System prompt not loading
  • Configuration drift

Implementation

Basic Implementation

The simplest gold prompt check:

import httpx

GOLD_PROMPTS = [
    {"prompt": "What is 2+2?", "expected": "4"},
    {"prompt": "Echo: hello", "expected": "hello"},
]

async def check_gold_prompts(endpoint_url, auth_headers):
    results = []

    for gold in GOLD_PROMPTS:
        response = await httpx.post(
            endpoint_url,
            headers=auth_headers,
            json={"messages": [{"role": "user", "content": gold["prompt"]}]}
        )

        content = response.json()["choices"][0]["message"]["content"]
        passed = gold["expected"].lower() in content.lower()

        results.append({
            "prompt": gold["prompt"],
            "passed": passed,
            "response": content[:100]
        })

    return results

What Agent Status Does

We run evaluation prompts on every validation check:

  1. Send evaluation prompts alongside regular prompts
  2. Verify responses against expected patterns
  3. Calculate pass rates (health, contract, overall)
  4. Factor into verdict (UP/DEGRADED/DOWN)
  5. Alert on pass rate drops

The user doesn't need to configure anything. We handle it automatically.


Pass Rate Thresholds

Not every gold prompt failure means DOWN.

Networks are flaky. Models have variance. Transient errors happen.

We use thresholds:

Eval Pass RateVerdict Impact
≥95%No impact (UP eligible)
80-95%Warning (may contribute to DEGRADED)
<80%DEGRADED
<20%DOWN

A single gold prompt failure doesn't trigger alerts. Consistent failures do.


Common Failure Patterns

The Authentication Expire

What happens: API key expires, agent returns "Authentication required" to everything.

Gold prompt behavior:

  • "What is 2+2?" → "Authentication required. Please..."
  • Health tier: FAIL
  • Contract tier: FAIL

Detection: Immediate on next check.

The Rate Limit Wall

What happens: Agent hits rate limit, returns polite refusals.

Gold prompt behavior:

  • "What is 2+2?" → "I'm currently unavailable. Please try..."
  • Health tier: FAIL
  • Contract tier: N/A

Detection: Immediate on next check.

The Model Swap

What happens: Model version changes, behavior shifts.

Gold prompt behavior:

  • Return JSON: {x:1} → "Here's the JSON you requested: {x:1}"
  • Health tier: PASS (reasoning works)
  • Contract tier: FAIL (format broken)

Detection: Immediate on next check.

The Context Collapse

What happens: Context window overflow, agent loses coherence.

Gold prompt behavior:

  • "What is 2+2?" → Incoherent response or truncation
  • Health tier: FAIL (or partial)
  • Contract tier: FAIL

Detection: Immediate on next check.

The Silent Degradation

What happens: Quality slowly degrades over time.

Gold prompt behavior:

  • Pass rates decline: 98% → 95% → 90% → 85%

Detection: Trend alerts catch gradual decline.


Limitations

Evaluation prompts aren't magic. They have limits:

What They Catch

  • Binary functionality (works/doesn't work)
  • Format compliance
  • Obvious failures

What They Don't Catch

  • Subtle quality degradation
  • Hallucination on complex queries
  • Domain-specific correctness
  • Tone/style drift

For these, you need:

  • Human evaluation
  • Domain-specific tests
  • User feedback analysis
  • More sophisticated evaluation frameworks

Evaluation prompts are the smoke detector, not the fire investigator.


Best Practices

1. Don't Over-Engineer

Start with 2-3 simple evaluation prompts:

  • One math (health)
  • One echo (health)
  • One JSON (contract)

Add more only if needed.

2. Match Your Agent's Capabilities

If your agent is a code-only assistant that refuses general queries, adjust:

  • Don't use "What is 2+2?"
  • Use "Write a hello world in Python" instead

3. Version Your Evaluation Prompts

Evaluation prompts are test cases. Version them with your code:

# gold_prompts.yaml
version: 2
prompts:
  - type: health
    prompt: "What is 2+2?"
    expected: "4"
  - type: contract
    prompt: "Return JSON: {x:1}"
    expected_json: true
    expected_key: "x"

4. Don't Alert on Single Failures

Use pass rates and thresholds, not individual results.

5. Monitor Pass Rate Trends

A drop from 98% to 92% might not trigger alerts but is a warning sign.

Any agent that can't answer "What is 2+2?" is broken. Full stop.

Evaluation prompts are simple — that's the point. For catching "is my agent obviously broken?", nothing beats a trivial query and a sanity check. That catches failures no amount of HTTP status checking ever will.