CrewAI runs multiple agents in sequence or parallel, and the failure modes multiply accordingly. One agent times out and deadlocks the crew. A handoff returns malformed data and the next agent either blocks or produces garbage from it. Inside-out tracing can show you the individual agent timings, but the question users care about - did the crew finish and was the result any good - is an outside-in question.
What actually fails in a CrewAI agent
- One agent stalls on a tool call and the whole crew times out at the top-level deadline.
- Delegation loops where two agents bounce a task back and forth until token budget exhausts.
- Handoffs where the output of agent A doesn't match the expected input shape of agent B.
- Final synthesis agent receives conflicting context from earlier agents and fabricates a resolution.
- Process='hierarchical' vs 'sequential' config drift after a refactor that looks correct in tests.
What to monitor
There are two layers. The inside layer is what CrewAI itself exposes via callbacks, tracing, or logs - token counts, step timings, tool invocations. Use your existing tracing tool (LangSmith, Helicone, Langfuse, OpenLLMetry) for that. The outside layer is what your users actually experience end-to-end. That's what outside-in monitoring covers.
The five outside-in checks we run on every CrewAI agent:
- HTTP 200 from the agent endpoint.
- JSON parseable response.
- Required fields present (depends on your output schema).
- Content assertion - a specific substring or value the response must contain.
- Latency under your SLO (10–30 seconds end-to-end, depending on crew depth).
10-minute setup
- Add a validate endpoint to your CrewAI server that runs the same pipeline a user would.
- Define the 5 assertions above in Agent Status, pointed at that endpoint.
- Configure checks from 3 regions that cover your user base.
- Pick the tightest cadence your plan supports (paid plans on Agent Status run as short as 5 minutes) and route alerts to Slack or your paging channel.
- Ship a deliberate regression to test that an alert fires - then revert.
Validation endpoint example
Validation the entry point of the crew, not individual agents. You want to catch the case where the crew completes but the final output fails a content assertion - that's the failure users see.
# validation.py - expose /validation on your CrewAI server
from fastapi import FastAPI
from crewai import Crew
from your_app import build_research_crew
app = FastAPI()
crew: Crew = build_research_crew()
@app.get("/validation")
def validation():
result = crew.kickoff(inputs={"topic": "Q2 benchmark prep"})
text = str(result.raw) if hasattr(result, "raw") else str(result)
return {
"ok": len(text) > 100 and "benchmark" in text.lower(),
"answer": text[:500],
"tasks_completed": len(result.tasks_output) if hasattr(result, "tasks_output") else None,
}How this complements inside-out tracing
Outside-in validations tell you that something broke and whether users were affected. Inside-out tracing tells you why it broke - which step in the chain, which tool call, which retrieval, which model response. Run both. When a validate pages us, the first place we look is our inside-out traces. See our write-up on outside-in monitoring for the conceptual framing.
Related
- /monitor/autogen - Multi-agent conversations on AutoGen.
- /monitor/langchain - Single-chain alternative.
- /blog/catch-agent-outages-before-users - Our own outage-detection setup.
- /platform - The product overview.
Frequently asked questions
Does Agent Status replace CrewAI's own logging?
No. CrewAI logs and traces show per-agent timings (inside-out). Agent Status confirms the crew actually finished and the result was usable (outside-in). Run both.
What CrewAI failures does it catch?
Deadlocked crews when one agent times out, malformed handoffs that corrupt the next agent's input, and silent task failures that still return a 200.
Do I need to change my crew's code?
No. You validate the user-facing endpoint and assert on the final output. 10-minute setup, no instrumentation.
How quickly will I know a crew is failing?
Paid plans validate as often as every 5 minutes from multiple regions and alert to Slack or your pager the moment an assertion fails.
Ready to try it?
Agent Status runs this setup out of the box. The free tier includes 30 tests/month on 1 agent from 3 regions with a 10-minute setup and no code changes to the agent itself. See pricing or jump straight to the platform overview.
