Agent Reliability — Evals, Guardrails, Tracing & Failure Modes

Overview

Chatbots fail by saying the wrong sentence. Agents fail by doing the wrong thing, repeating it, or trusting a tool that lied. Reliability work grades the run, contains the tools, and leaves a trace you can replay.

Trajectory grading

Score the path:

1. Did it call the right tools in a valid order?
2. Were arguments schema-valid and authorized?
3. Did it stop under the budget?
4. Was the final answer faithful to observations? Grounding metrics live on the RAG hub. Use the idea. Do not re-teach the retrieval suite.
5. Were side effects exactly-once?

flowchart TB
  a["1. Goals dataset"]
  b["2. Run in sandbox"]
  c["3. Capture spans"]
  d["4. Grade tools and args"]
  a -->|1. Goals dataset to 2. Run in sandbox| b
  b -->|2. Run in sandbox| c
  c -->|3. Capture spans to 4. Grade tools and args| d

A red trajectory blocks release. A green suite still gets a canary, because fixtures are not production.

Guardrails

Classifiers that route or block at the edge are the System One and hybrid lessons. This page assumes those gates can exist and focuses on the agent loop around them: allowlists, sandboxes, and the eval that proves the gate was consulted.

Failure modes that are specific to agents

Poison tools. Text that came back from a browser, a ticket, or a file is untrusted content. The same rule you use for untrusted web pages applies: do not execute instructions found inside the observation. Keep that rule on this page. Do not turn the answer into a general security survey.

Tracing

Minimum span model:

• agent.run as the root.
• agent.model with a prompt hash, token counts, and whether the prefix cache hit.
• agent.tool with name, latency, error, and the idempotency key.
• agent.guardrail with allow or deny.
• agent.hitl with wait time and the decision.

Correlate them with one run id. Export with OpenTelemetry GenAI conventions where you can. The point of the span is replay: you should see that the refund was proposed, denied, and never executed.

Loop detector

export function isStuck(
  recent: Array<{ name: string; argsKey: string }>,
  n = 3,
): boolean {
  if (recent.length < n) return false;
  const slice = recent.slice(-n);
  return slice.every((item) => item.name === slice[0].name && item.argsKey === slice[0].argsKey);
}

Press Run. Snippets must be self-contained — no network, files, or native modules.

Three identical calls is a stop, not a hint. A successful read that repeats is exactly the case a circuit breaker will not catch, because the calls are not errors.

Circuit breaker

Press Run. Snippets must be self-contained — no network, files, or native modules.

The breaker watches error rate. The scaling lesson decides what the queue does when the breaker is open: wait, shed, or degrade. Cascading failures in the SRE book are the same shape. An agent adds a twist: the calls that succeed can still be the bug.

Rollback

Not every tool has an undo. Design a compensating action (void against a refund request) and store the intent in episodic memory so a later step can see it. If the business step must be atomic across several systems, use a durable workflow with explicit compensations. That is a concept, not a vendor tutorial, and it is not a reason to fan out writes.

Pitfalls

• Shipping on a vibe check of five chats.
• Letting tool text rewrite the system prompt because it was “just context.”
• One metric for prose quality and no metric for “did we email.”
• Opening the email tool to all recipients on day one of the canary.
• Assuming offline green means the live schema still matches the fixture.

Interview Q&A

Go Deeper

• OpenAI evals guide and the Evals repository for harness shape. Your assertions are still trajectory assertions.
• OWASP LLM Top 10 for excessive agency and prompt injection via tool output.
• OpenTelemetry GenAI conventions for span names.
• Addressing cascading failures and CircuitBreaker for the open state. Pair it with the step budget so successful loops still stop.