Why Traditional Observability Is Not Enough for AI Agents

Traditional observability answers essential questions: Is the service available? Are requests slow? Did a dependency return an error? Is the queue growing? Those signals remain necessary for an AI-agent system, but they do not establish that the agent understood the task, selected an appropriate action, used tools safely, or produced a useful result.

An agent can complete every HTTP request successfully and still fail its user. That gap is why agent reliability needs an additional outcome layer rather than a replacement for infrastructure observability.

What traditional telemetry can tell us

Metrics, logs, and traces provide a strong operational foundation. They can reveal:

• Model and tool-call latency
• Token, compute, and external API consumption
• Exceptions, timeouts, retries, and rate limits
• Queue depth and worker saturation
• Dependency and network failures
• The sequence and duration of execution steps

OpenTelemetry traces model operations as spans and use semantic attributes to make cross-system analysis possible 1. Emerging generative-AI attributes extend that vocabulary to model operations and tool calls 2. This is established instrumentation practice: represent what executed and attach consistent context.

The limitation is epistemic. A successful span says an operation completed under its technical contract. It does not say the final answer was correct, grounded, complete, safe, or aligned with the user’s intent.

The silent-failure problem

Consider an agent that receives a request to compare two contracts, retrieves only one document, produces a fluent summary, and returns HTTP 200. Infrastructure dashboards may show healthy latency and zero errors. The workflow may even contain a “completed” event. Yet the outcome is incomplete.

Other silent failures include:

• Selecting a plausible but wrong tool
• Supplying validly shaped but semantically wrong arguments
• Ignoring a constraint stated early in a long conversation
• Retrieving stale or irrelevant evidence
• Treating tool output as trustworthy without validation
• Finishing a workflow before required side effects occur
• Producing a confident answer when evidence is insufficient

These are not all model failures. They can originate in orchestration, retrieval, tool schemas, prompts, state management, permissions, or evaluation logic. A useful reliability system must preserve enough evidence to locate the failure boundary.

Observe execution as a decision graph

A flat request trace is rarely enough. Agent execution is a graph of decisions, model calls, retrievals, tool attempts, validations, retries, human interventions, and state transitions.

The trace should answer:

1. What intent and constraints were active?
2. Which component made each decision?
3. What evidence was available at that point?
4. Which tool arguments and results influenced the next step?
5. Which policy, validation, or evaluator accepted the outcome?

Content capture is sensitive. OpenTelemetry’s generative-AI registry warns that tool arguments, results, and instructions may contain sensitive information 2. A production design should default to metadata, hashes, references, redacted excerpts, or opt-in content capture—not indiscriminate prompt logging.

Separate execution health from outcome quality

A practical model keeps independent dimensions instead of compressing them immediately into a single score:

Dimension	Example question	Possible evidence
Execution health	Did the workflow complete within its technical contract?	Exceptions, timeouts, retry exhaustion
Task completion	Were all required steps and side effects completed?	State checks, tool receipts, workflow assertions
Semantic correctness	Is the result supported and appropriate for the task?	Deterministic checks, reference comparison, evaluator judgment
Policy compliance	Did execution respect permissions and safety constraints?	Policy decisions, blocked actions, audit events
Evidence sufficiency	Is there enough evidence to trust the decision?	Evaluator coverage, provenance, confidence bounds

These dimensions can disagree. An execution may be healthy but semantically wrong, or technically degraded yet recover to a correct outcome. Keeping them separate makes alerts and remediation more explainable.

Define SLIs around critical journeys

Traditional SRE recommends user-centered indicators and a small, iteratively refined set of SLOs 3. The same principle can be adapted to agents, with caution.

Start with critical journeys, not a universal “agent accuracy” metric. For a research assistant, one journey might require retrieving every designated source and attaching citations. For an operations agent, a journey might require proposing a change, obtaining approval, applying it through an authorized tool, and verifying the resulting state.

Candidate indicators include:

• Verified completion rate: eligible runs that satisfy deterministic completion checks
• Unsupported-claim rate: sampled outputs containing claims not backed by required evidence
• Tool-selection validity: decisions choosing an allowed tool appropriate to the declared intent
• Argument validity: tool calls that pass both schema and domain validation
• Human-escalation quality: uncertain cases escalated with sufficient context
• Evaluation coverage: eligible outcomes that receive the required evaluator set within a time window

The denominator matters. Exclude cancelled or ineligible runs deliberately and record why. A missing evaluation must not silently become a pass. A three-state result—pass, fail, unknown—often preserves more truth than forcing uncertainty into a binary metric.

SLO targets should follow baseline measurement and risk discussion. This article intentionally proposes no universal percentage: the acceptable objective depends on task consequence, evaluator quality, user expectations, and the cost of false acceptance versus false rejection.

Combine synchronous guardrails with asynchronous evaluation

Some checks must occur before an action:

• Authorization and tool allowlists
• Argument schema and domain constraints
• Data-loss or financial-impact confirmations
• Required evidence presence
• Deterministic output validation

These are guardrails. They should be fast, predictable, and fail safely. They should not rely solely on another probabilistic model for high-consequence controls.

Other evaluations are too expensive or context-heavy for the request path. Asynchronous evaluation can sample completed runs, apply multiple evaluators, compare versions, and discover patterns over time. It introduces its own reliability questions: queue delay, evaluator drift, versioning, partial coverage, and the handling of outcomes later judged unsafe.

The asynchronous result should link to the exact agent, prompt, model, tool, policy, and evaluator versions used. Without versioned provenance, a reliability trend cannot distinguish behavior change from measurement change.

Reliability guardrails are decision points

A guardrail should define its scope, evidence, failure behavior, and owner. Examples:

• Block a tool call outside the agent’s allowed capability set
• Require human approval above a consequence threshold
• Reject arguments that are structurally valid but violate domain constraints
• Stop when required evidence is missing rather than fabricate a completion
• Limit retries to prevent repeated side effects
• Route low-confidence or evaluator-disagreement cases to review

Guardrails can also cause harm through false positives, latency, or brittle rules. Instrument their decisions and review both escapes and unnecessary blocks. Reliability is not maximized by blocking everything; it is improved by making risk-sensitive decisions explicit and observable.

Root-cause investigation needs preserved context

When an outcome fails, investigation should move through layers:

1. Intent: Was the task and success condition represented correctly?
2. Planning: Did the system select a viable path under the available policy?
3. Context: Was relevant, current evidence retrieved and retained?
4. Model behavior: Did the model transform the context appropriately?
5. Tool execution: Were the tool, arguments, permissions, and results correct?
6. State: Did retries or concurrency corrupt workflow state?
7. Evaluation: Did a missing, weak, or drifting evaluator accept the failure?

Correlate the outcome with infrastructure telemetry, but do not stop there. A model timeout may explain an incomplete response; healthy model latency cannot explain whether a recommendation was correct.

A failure taxonomy should remain small enough to be actionable and allow multiple contributing causes. Forcing every incident into one label hides interaction failures, such as stale retrieval combined with an evaluator that checks style but not evidence.

Evaluation is another production system

Evaluators have false positives, false negatives, coverage gaps, latency, and cost. Model-based evaluators may share failure modes with the system they judge. Human review has inconsistency and limited scale. Deterministic checks are precise only for the properties they encode.

A stronger evaluation design combines complementary methods:

• Deterministic invariants for schema, state, permissions, and required evidence
• Reference-based tests where a defensible reference exists
• Model-based judgments for bounded semantic criteria, with explicit rubrics
• Human review for calibration, high-consequence cases, and disagreement
• Regression suites using curated failure examples

Track evaluator version, coverage, disagreement, and calibration evidence. Do not present an evaluator score as ground truth when its own error characteristics are unknown.

Risk management is broader than telemetry

NIST’s AI Risk Management Framework is voluntary guidance for incorporating trustworthiness considerations into the design, use, and evaluation of AI systems 4. It is a useful reminder that observability is one control within a broader lifecycle: governance, risk identification, measurement, and management.

Tracing every call does not establish acceptable risk. Teams still need task boundaries, access control, change review, incident response, data governance, and decisions about where autonomous action is inappropriate.

Limitations and open research questions

This reliability approach has unresolved questions:

• How should semantic SLOs account for evaluator uncertainty?
• Which evidence can be retained without creating unacceptable privacy risk?
• How can outcomes be compared when tasks are open-ended and non-repeatable?
• When do aggregate rates hide severe failures in rare but consequential journeys?
• How should reliability transfer across model, prompt, tool, and policy versions?
• How can teams detect correlated failures between an agent and a model-based evaluator?
• Which guardrails improve safety without making useful tasks impractical?

These are research and engineering questions, not solved properties of the proposed framework. Claims should be validated in the intended domain with representative workloads and an explicit risk model.

Closing perspective

Traditional observability remains the foundation: an agent system cannot be reliable if its infrastructure, dependencies, and execution engine are opaque. But operational health and semantic success are different statements.

Agent observability should connect execution evidence to task outcomes, evaluation coverage, policy decisions, and versioned provenance. Agent reliability should then use that evidence to make bounded decisions—allow, block, escalate, investigate, or improve—without pretending that one score captures every form of correctness.