LLM Observability: What to Log for Every Model Call and Agent Run

LLM observability is the practice of recording what happens in every model call and agent run, so a team can explain, debug and prove what an AI system did. For a model call that means the prompt, the retrieved context and its sources, the model, tokens, latency, cost and a quality score; for an agent it adds every step, tool call, approval and write made to another system.

Traditional application monitoring tells you a request took 900 milliseconds and returned 200. That is not enough for an AI system, where the request "succeeded" and the answer still cited last year's travel policy. The failure lives in the content: what the model was shown, which version of which document it retrieved, and what it did with it. LLM observability exists to make that content inspectable.

What is LLM observability?

LLM observability covers three layers of record:

• The call. One request to one model: input, context, output, model, tokens, latency, cost.
• The answer. What the user saw, which sources it cited, and how it scored on grounding or evaluation checks.
• The run. For agents, the sequence of steps that produced an outcome, including tools called, policies applied, approvals given and writes made.

LLM monitoring is the aggregate view over those records: latency percentiles, error rates, token volume, cost by team, pass rates over time. Monitoring tells you something moved. Observability lets you open one run and find out why.

What to log for every model call

At minimum, each model call should carry:

1. Who and where. The user or service identity, the surface (chat, agent, API, form) and the deployment.
2. Input. The prompt as sent, after masking, and the system instructions version.
3. Retrieved context. The passages included and the document, version and passage each came from.
4. Model. Provider, model name and any routing or fallback decision.
5. Usage. Tokens in and out, latency and cost.
6. Output. The response as delivered, with its citations.
7. Checks. Guardrail results: injection detected or not, PII masked, grounding score, whether the answer was blocked or routed to a person.

The context is the item teams most often skip, and it is the one that explains most bad answers. A wrong answer with the right sources is a model problem. A wrong answer built on a superseded document is a data problem. Without the retrieved context in the record, you cannot tell which one you have.

What to log for every AI agent run

Agent observability is call observability plus the run. Each run should record:

• the trigger (a person, a schedule, an event or a file) and who it ran as;
• every step, in order, with its input, output and duration;
• each tool call with arguments and result;
• each policy decision on an action: run, hold for a person, or refuse;
• each approval, with the approver, time and context;
• each write to another system, with what it did (create, update, close, send) and where it landed.

Joined to one run identifier, those records turn "what did this agent change in Jira and the CRM last Tuesday?" into a query. How writes are made exactly once and recorded is covered in AI agent governance; how multi-step runs are coordinated is in AI orchestration.

AI agent observability: three runs worth replaying

The value of a good record shows up in specific incidents. Here are three illustrative ones, of the kind every team running agents eventually meets:

The duplicate ticket. A triage agent timed out talking to the helpdesk, retried, and a customer received two replies. The run record shows the first call succeeded after the timeout fired. The fix is not in the prompt; it is idempotent execution keyed to the step. Without the per-step record, the team would have blamed the model.

The confident wrong policy. An HR assistant told a manager that international travel needs no pre-approval. The answer cited a document, and the record showed which one: version 3 of the travel policy, superseded by version 4 two months earlier. The model behaved correctly on bad context. The fix was retiring the old version at the source and checking the connector's deletion handling.

The approval nobody remembers. An auditor asked who approved a supplier-record change in March. The run held the write for approval, the approver's decision was recorded with time and context, and the change itself was joined back to the run. The answer took a query, not a week of email archaeology.

Each story needed a different signal: per-step timing, retrieved-source versions, and the approval record. That is why agent observability has to capture the whole run, not just the model calls inside it.

LLM observability signals and who uses them

Developer observability vs. governance-grade audit

Engineering traces and audit evidence overlap in content and differ in purpose.

Developer observability is for debugging. It can be sampled, and engineers can filter, edit or delete it. It sits in the tool the engineering team prefers, and that is fine for its job.

Governance-grade audit is evidence. It has to be complete for material events, append-only, and verifiable by someone who does not trust whoever wrote it. That last property is the one most systems lack. The test is not "do you log everything?" but "can an auditor confirm a record was not changed without asking you?"

The full argument for verifiable records, including signed receipts and tamper-evident exports, is in AI audit trail compliance.

Where OpenTelemetry fits

OpenTelemetry now maintains semantic conventions for generative AI: standard names for spans, metrics and events emitted by GenAI clients, with related conventions for MCP and individual model providers. Adopting them lets AI traces sit beside the rest of your application telemetry and keeps you portable across tracing back ends.

Use them for the engineering layer. They standardize how signals are named; they do not decide what must be kept, who may change it, or how a third party verifies it. Those are governance questions.

Logs must be safe to keep

An observability store full of raw prompts is a second copy of your most sensitive data. Two rules keep it usable:

• Mask before you write. Personal data in prompts and answers should be masked by policy in the request path, before it reaches the model or the log.
• Never store secrets. Keys and tokens pasted into a message should be caught and redacted, so the record notes that a secret was present without holding its value.

If the log is not safe to keep, it gets purged early, and then it is not there when the auditor asks.

How SphereIQ records model calls and agent runs

In SphereIQ, observability and audit come from the same request path, because every call passes through the Governance plane:

• Every model call is written to the audit log with who made it, the model, the tokens in and out, and what it cost. Administrative actions go to the same log, and the application only ever adds to it.
• Guardrails (prompt-injection detection, secret scanning and personal-data masking) run on every chat message before it reaches a model, and their events land on the same ledger as retrievals, policy hits, governed actions and agent publishes.
• Agent runs in the AI Factory are recorded step by step. Scheduled and event runs land in an Output queue with assignment, review state, feedback and cost.
• Evaluations report pass rate, p50 and p95 latency and cost per run, and retrieval is scored separately by citation hit-rate against ground-truth answers.
• Cost is attributed per feature, per user and per session, with chargeback exports by user, team or department.
• Published agents record every run, refusals included, with the surface, the input, the outcome and the cost.
• Evidence is verifiable. Receipts are Ed25519 signatures over canonical JSON with a published public key, and administrators can export records as newline-delimited JSON with a SHA-256 manifest anyone can check offline.

The security page lists these controls as built, including what is not built yet.

How to set up LLM observability in five steps

1. Pick one run identifier and make every call, retrieval, tool call, approval and write carry it.
2. Record context, not just text. Store the source document, version and passage for every retrieved chunk.
3. Mask and redact at the edge so the log is safe to retain for as long as your auditors need.
4. Attach quality signals. Add grounding checks to live traffic and a gating evaluation set to every release; the scoring method in how to score a RAG answer is a practical start.
5. Separate the debugging view from the evidence store. Engineers get searchable traces; risk gets an append-only, verifiable record.

Then test it: pick a bad answer from last week and see how long it takes to explain it. If it takes an afternoon, the record is missing something. The controls that should have stopped the bad answer are covered in AI guardrails, and the 15 enterprise AI agent use cases show which runs will need the deepest records first.

Frequently asked questions