LLM observability adds four signals that application performance monitoring never needed: token counts split by cached and fresh, cost per request, tool-call structure inside a turn, and whether the turn succeeded in a way a status code cannot express. The tooling splits into proxy-based (Helicone, Portkey), SDK-based tracing (Langfuse, LangSmith), and provider-native reporting. OpenTelemetry has GenAI semantic conventions covering agent and tool spans, but as of August 2026 every metric in them is still marked Development stability, so portability is a direction rather than a guarantee.
- The four signals APM lacks: tokens, cost, tool-call structure, and turn outcome. Latency and error rate are the easy part.
- Split tokens into fresh input, cache reads, cache writes, and output. A single "input" number hides the cheapest optimization you have.
- The OTel GenAI conventions define
gen_ai.client.token.usageand agent-level metrics, but all of them are still Development stability. - Proxy tools see every call and add a hop. SDK tools add no hop and miss anything not instrumented. Pick by which blind spot you can live with.
- Continuum reads agent CLI logs locally and prices them. It is not a tracer for your own applications.
What APM does not carry
A traditional APM span tells you an HTTP call took 340ms and returned 200. For a model call that is nearly content-free. The interesting failures return 200: a truncated answer, a tool loop, a refusal, a correct answer that cost eleven dollars.
| Signal | Why APM never needed it | What it tells you |
|---|---|---|
| Token counts, split four ways | HTTP calls have no unit of work inside them | Where the money is, and whether the cache is warm |
| Cost per request | Requests were free at the margin | The only number finance will read |
| Time to first token | A response was atomic | Perceived speed for a streaming UI |
| Tool calls inside a turn | There was no nested agent loop | Whether the agent is working or thrashing |
| Turn outcome | The status code was the outcome | Whether a 200 was actually useful |
The agent-specific part
A chatbot makes one model call per user message. A coding agent makes a variable number, interleaved with tool calls, over a turn that can run for minutes. That structural difference is what the newer conventions are trying to capture.
The OpenTelemetry GenAI semantic conventions, which moved to their own repository in 2026, define both client-level and agent-level instruments:
| Metric | Type | Unit | Stability |
|---|---|---|---|
gen_ai.client.token.usage | Histogram | {token} | Development |
gen_ai.client.operation.duration | Histogram | s | Development |
gen_ai.client.operation.time_to_first_chunk | Histogram | s | Development |
gen_ai.invoke_agent.duration | Histogram | s | Development |
gen_ai.invoke_agent.inference_calls | Histogram | {inference_call} | Development |
gen_ai.invoke_agent.tool_calls | Histogram | {tool_call} | Development |
gen_ai.execute_tool.duration | Histogram | s | Development |
The tool landscape, honestly
Two architectures, and the choice between them is the real decision. A proxy sits between your code and the provider, sees every call including ones you forgot to instrument, and adds a network hop plus a dependency in your critical path. An SDK instruments your code, adds no hop, and is blind to anything that does not go through it.
| Tool | Architecture | Free tier | Paid entry | Notes |
|---|---|---|---|---|
| Helicone | Proxy (async logging also available) | 10,000 requests, 1 seat, 7-day retention | Pro $79/mo, Team $799/mo | Alerts and HQL on Pro; SOC 2 and HIPAA on Team |
| Langfuse | SDK tracing, open source | Hobby: 50k units/mo, 2 users, 30-day access | Core $29/mo (100k units, then $8/100k), Pro $199/mo | Self-host free; Enterprise cloud $2,499/mo adds audit logs and SCIM |
| LangSmith | SDK tracing | Developer: 5k base traces/mo | Plus $39/seat/mo | Overage in compute and storage units ($1.50 LCU, $1.00 LSU); 14-day base retention |
| Portkey | Gateway, open source core | Developer: 10k logs/mo | Production $49/mo (100k logs, $9 per extra 100k) | Observability plus enforcement: granular budget and rate limits are Enterprise |
| Provider-native | Reporting API | Free | Free | Anthropic Usage and Cost Admin API, OpenAI /v1/organization/costs; daily granularity on cost |
The candid version of that table: Langfuse self-hosted is the best value in the category and the most work. Helicone is the fastest thing to stand up if you accept a proxy. LangSmith is priced for LangChain shops and its unit-based overage is genuinely hard to forecast before you run it. Portkey is not really an observability tool that added routing, it is a gateway that logs, which matters because a gateway can also refuse a request.
The free setup that covers coding agents
If the workload you want visibility into is Claude Code rather than your own application, none of the above is required. Claude Code exports OpenTelemetry metrics and events directly, including a cost metric, and the whole thing is environment variables.
export CLAUDE_CODE_ENABLE_TELEMETRY=1
export OTEL_METRICS_EXPORTER=otlp # or prometheus, console
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
# tag every metric for allocation
export OTEL_RESOURCE_ATTRIBUTES="team.id=platform,cost_center=eng-123"
# metrics emitted:
# claude_code.cost.usage (USD)
# claude_code.token.usage (tokens)
# claude_code.session.count
# claude_code.lines_of_code.count
# claude_code.commit.count
# claude_code.pull_request.count
# claude_code.code_edit_tool.decision
# claude_code.active_time.total (seconds)Prompt and response text is redacted by default and only logged if you opt in with OTEL_LOG_USER_PROMPTS or OTEL_LOG_ASSISTANT_RESPONSES. Leave those off unless you have a policy that says otherwise, because turning them on moves your source code into your observability vendor. The full walkthrough lives in the Claude Code metrics guide.
What Continuum covers, and what it does not
Worth being explicit, because this page competes with vendors who would answer the question differently about themselves.
- Covered: cost and token analytics for coding agents, parsed from the transcripts the CLIs already write to local disk. Claude under
~/.claude/projects, Codex under~/.codex/sessions, OpenCode into a local database, and the others into their own state directories. Nothing sits in the request path, so there is no added latency and no proxy to fail. - Covered: history you did not instrument in advance. Because it reads files that already exist, the first parse covers however many months of sessions are on the machine rather than starting from install day.
- Covered: attribution by repo, provider, model, and day, with events priced at each model's own rate.
- Not covered: tracing arbitrary applications. If you want spans for your own LLM-backed product, use Langfuse, LangSmith, or an OTel SDK. That is a different job and we do not do it.
- Not covered: prompt and response content. Direct sessions run laptop to provider and report usage only.
Questions people ask
What is LLM observability?
LLM observability is monitoring and tracing for model calls, covering the signals classic application monitoring does not carry: token counts split into fresh input, cache reads, cache writes and output; cost per request; time to first token; the tool calls made inside an agent turn; and whether the turn produced a useful outcome. The failures worth catching mostly return HTTP 200, which is why status codes and latency are not enough.
What is the difference between LLM observability and APM?
APM measures the transport: latency, throughput, error rate. LLM observability measures the work inside the call. An APM span for a model request tells you it took 340ms and succeeded, which says nothing about whether the answer was truncated, whether the agent looped through eleven tool calls, or whether that one request cost four dollars.
Langfuse or LangSmith: which should I use?
Langfuse if you want to self-host or if you are not a LangChain shop. It is open source, free to self-host, and its cloud Core tier is $29 a month for 100k units with $8 per additional 100k, which is easy to forecast. LangSmith is $39 per seat on Plus and bills overage in compute and storage units at $1.50 and $1.00 respectively, which is harder to predict before you run it, and it fits best if you already use LangChain or LangGraph.
Is Helicone free?
There is a free Hobby tier with 10,000 requests a month, one seat, one organization, 7-day retention and a 10 logs per minute ingestion limit, checked August 2026. Paid tiers are Pro at $79 a month, which adds unlimited seats, alerts and a one-month retention window, and Team at $799 a month, which adds SOC 2 and HIPAA coverage and three-month retention.
Are the OpenTelemetry GenAI semantic conventions stable?
No. As of August 2026 every metric defined in the GenAI semantic conventions, including gen_ai.client.token.usage and the agent-level instruments, is marked Development stability, meaning names and shapes can still change. Adopting them is still the right call because the alternative is a proprietary schema that is equally unstable and not portable, but do not plan a migration around the assumption that the names are frozen.
Do I need an observability tool just to see what Claude Code costs?
No. Claude Code exports OpenTelemetry metrics natively, including claude_code.cost.usage in USD, behind CLAUDE_CODE_ENABLE_TELEMETRY=1. For organizations, Anthropic also ships a free Claude Code Analytics API that returns per-user daily sessions, lines of code, commits, pull requests, and estimated cost by model. Tools that read the local transcripts cover the same ground retroactively with no setup.
Sources
Every figure above was read from these pages on August 2026. Vendors reprice without notice; if you find a stale number, tell us.