AI Agent Observability Stops at the Trace

A customer sends a shipping-address correction. An agent reads it, proposes an address, calls update_address, gets a success response, then sends a neat confirmation. The package can still go to the wrong order.

The trace can look like proof. Depending on content-capture settings, it may include the user request, model calls, retrieved documents, tool inputs and results, final answer, latency, and token bill. The agent used the intended tools and got no errors. Someone opens the trace, sees a tidy chain of green boxes, and concludes that the system worked.

That conclusion gets ahead of the evidence. A trace establishes how a run moved through a system. It can make an agent debuggable, explain a cost spike, and show that a tool returned an error. It does not, by itself, establish that the answer was true, that the intended record changed, or that the user got what they needed.

Observability and evaluation solve different problems. Teams that treat one as the other get a beautiful record of failures they have not learned to recognize.

The Trace Is a Reconstruction

An ordinary trace answers operational questions.

Which model handled the request? Which prompt version did it receive? How many tokens did it consume? Which tools did it call, in what order, and how long did each call take? Did a retrieval step return documents? Did the database reject a write? OpenTelemetry’s GenAI conventions provide a common shape for model and token telemetry, with optional capture for prompts, completions, tool calls, and tool results.¹

That record is valuable. Without it, an agent that takes forty-five seconds to answer a question leaves the team guessing whether the time went to the model, a slow tool, or a retry loop. Without tool inputs and results, a bad final answer can look like a model problem when the real problem was an empty search result or an API contract the model misunderstood.

The trace is still an account of process. A tool returning HTTP 200 proves that the server accepted a request. It does not prove that the request carried the right parameter, that the write changed the right customer record, or that the user should have wanted the change in the first place.

That boundary shows up in small ways. An agent can call search_orders, receive ten results, and cite one in a reply. The trace will show a successful tool call. It cannot determine whether the agent searched the right account, interpreted the date range correctly, or chose the order that answers the customer’s question. A run can be perfectly observable and still be wrong.

Five Signals, Five Claims

The useful move is to name each signal by the claim it can support. A dashboard called “agent quality” collapses distinct signals.

The first row is observability. The last row is evaluation. The middle rows connect a particular run to reality.

Take an agent that changes a shipping address. A trace can show that it read the customer message, proposed a new address, called update_address, and received success. A state check reads the stored address back and verifies that it belongs to the named order. An outcome signal arrives later: the package went to the right place, or the customer reopened the issue. An evaluation set tests many past requests, including ambiguous emails, duplicates, international addresses, and orders that have already shipped. Each adds a different kind of evidence.

Production Output Belongs in a Tool Call argues for a typed boundary instead of letting a chat response flow straight into production. That boundary gives the system a place to validate the request. It does not remove the need to validate what the request means. The typed order_id can belong to the wrong order. The schema can be valid while the action harms the customer.

A Successful Tool Call Is a Narrow Fact

Tool telemetry tempts teams into the easiest measurement: count successful calls. It is easy to aggregate, easy to graph, and often beside the point.

Suppose an agent routes support tickets. Its trace says it called assign_ticket on 98 percent of requests without an exception. That metric tells you something useful about the integration. It says the API is available and the agent can form requests the API accepts. It says almost nothing about whether the tickets reached the right team, whether the priority matched the customer’s situation, or whether a resolution followed.

Those facts belong at different boundaries. The assignment service can reject a team identifier that does not exist. A domain rule can reject an enterprise escalation sent to a low-priority queue. A later review can compare the route with the team that actually resolved the ticket. The first two can run immediately. The third may arrive days later. All three are better evidence than a green tool span.

This is why Rules Are Not Controls matters beyond security. A prompt instruction such as “use the billing queue for payment issues” changes the odds. A domain check that knows the account type and payment state decides whether a particular assignment is allowed. The trace should record both. The control belongs in the service that can refuse the bad state.

Evaluation Needs a Task, Not a Dashboard

An evaluation starts with a claim about completed work. “The agent produces useful support routing” is too broad to test. “For tickets in this queue, the agent assigns the correct team and priority, and it escalates the cases that lack evidence” gives you a task, an expected result, and a failure mode.

Build the evaluation set from work that resembles production. Include recently corrected runs, messy inputs, new policy versions, and the cases the agent had to hand back to a person. Keep the expected result close to the actual job. For routing, that may be the resolved destination. For a research agent, it may be an answer whose claims map back to supplied sources. For an action agent, it may be an expected state change plus a list of actions that should have been refused.

LangSmith separates offline evaluations on curated datasets from online evaluations on production traffic. Its own guidance describes production failures and unusual cases feeding the next offline dataset.² That is the loop worth copying. Traces supply the raw material. A human or a domain check decides what failed. The failed case becomes a test before the next prompt, tool, or model change goes out.

The evaluator itself needs scrutiny. A model judge can help score subjective work at a scale a person cannot sustain. It should be calibrated against human decisions before it becomes the release gate. A judge that likes polished, unsupported answers will give the team a quality metric that tracks presentation rather than correctness.

The same caution applies to summary metrics. A single “quality score” may be useful for detecting a sudden change, provided the team can open it up into the task-level signals underneath. It should not settle an argument about a production decision by itself.

Put the Check Beside the Consequence

The highest-value checks sit close to the effect the agent is trying to cause.

If an agent extracts invoice data, reconcile the total against the line items before the record reaches accounting. If it changes an entitlement, read the entitlement back under the user’s identity and check the policy. If it drafts a policy answer, verify the citations against the approved source set. If it plans a database change, check the proposed statement against permissions and invariants before execution.

These are not all easy. A customer-facing answer may require a sampled human review and a later satisfaction signal. A strategic recommendation may be hard to judge for months. That does not make the trace less useful. It changes the decision. The slower the acceptance signal, the less evidence the team has for giving the agent broad autonomy.

We Automate What We Can Verify makes the wider case. A system learns from a result only when something can distinguish an accepted output from a plausible miss. Observability makes the run inspectable. Verification makes a decision about the run. Evaluation shows whether that decision process holds across a real distribution of work.

Good agent systems begin with traces. They name the decision the agent is supposed to make, capture evidence that the decision had the intended effect, and turn failures into evaluation cases.

Footnotes

1. OpenTelemetry, “Inside the LLM Call: GenAI Observability with OpenTelemetry”, May 14, 2026. ↩

2. LangChain, “Evaluation types”, covering offline datasets, online evaluation, and the feedback loop from production cases. ↩