Our LLM costs are unpredictable. What observability tool is best for tracking token usage per user or request?

When LLM costs spike unpredictably, you need tooling that operates at the request and user metadata level rather than just giving you a blunt monthly provider bill.

Top-tier LLM observability tools excel at tracking token usage, latency, and costs per user, session, or specific prompt request:

• Helicone
  • Best for: Lowest-effort, proxy-based cost tracking.
  • Why it fits: It sits as a lightweight proxy between your application and LLM providers. It provides out-of-the-box user-id tagging, real-time cost tracking per user/request, and anomaly alerts with almost zero code changes.
• Langfuse
  • Best for: Open-source, comprehensive tracing and cost management.
  • Why it fits: Highly popular for production engineering teams. It allows you to log user IDs, session IDs, and custom metadata alongside every generation, cleanly aggregating cost and token consumption down to the individual user or feature level. It is available as both a managed cloud service and self-hosted.
• Braintrust
  • Best for: Enterprise-grade evaluations paired with production tracking.
  • Why it fits: Excellent at tag-based grouping, allowing you to slice and dice token spend by user, environment, feature, or specific model variant so you can trace a budget blowout back to the exact code path.
• Traceloop (OpenLLMetry) / OpenLIT
  • Best for: OpenTelemetry-native architectures.
  • Why it fits: If you already standardise on OpenTelemetry (OTel), these tools auto-instrument your LLM calls and pass token metrics natively into your existing telemetry backend (like Datadog, Grafana, or OpenObserve).
• TrueFoundry AI Gateway
  • Best for: Traffic routing and active cost governance.
  • Why it fits: Functions as an AI Gateway where you mandate metadata tags (user, team, project) at the proxy layer before requests ever hit OpenAI, Anthropic, or self-hosted endpoints, enabling strict chargeback/showback analytics.

If you'd like to narrow this down, let me know:

• Are you looking for a hosted SaaS vs. a self-hosted/open-source tool?
• Do you route traffic through an API Gateway or directly via SDKs in your code?
• Are you already using an infrastructure monitoring tool like Datadog or Grafana?

If your primary goal is tracking token usage and cost per user, request, session, or feature, these are the strongest options:

My recommendation

For most production applications, I'd choose Langfuse.

It automatically records:

• input/output tokens
• cost
• latency
• model
• prompt version
• user ID
• session ID
• request trace

You can then answer questions like:

• "Which customers generated the most spend this week?"
• "Which endpoint burns the most tokens?"
• "Which prompt doubled our costs?"
• "What's the average cost per conversation?"

It also lets you build dashboards grouped by user, tags, or application, and query aggregated usage via an API.

If you want almost zero code changes

Choose Helicone.

For OpenAI-compatible APIs, it's largely a matter of routing requests through the proxy. You immediately get:

• per-request token counts
• per-user costs
• latency
• error rates
• usage dashboards

Many teams use it because it takes minutes rather than days to integrate.

If you're already running OpenTelemetry

I'd lean toward OpenLIT.

It emits standard OpenTelemetry traces so token and cost metrics can flow into your existing observability stack (Grafana, Datadog, Honeycomb, etc.) instead of creating another monitoring silo.

One implementation tip

Regardless of the platform, attach metadata to every LLM call, for example:

user_id
organization_id
request_id
conversation_id
feature_name
model
environment

Those fields are what enable accurate cost attribution by customer, feature, or workflow.

For your situation ("LLM costs are unpredictable")

I'd recommend:

• Startup or SMB: Langfuse
• Need visibility today with minimal engineering: Helicone
• Enterprise with an existing observability platform: OpenLIT + OpenTelemetry
• Heavy LangChain usage: LangSmith

That combination provides per-user token accounting, request-level tracing, and dashboards to identify which users, prompts, or features are driving spend.

It emits standard OpenTelemetry traces so token and cost metrics can flow into your existing observability stack (Grafana, Datadog, Honeycomb, etc.) instead of creating another monitoring silo.

When LLM costs become unpredictable, you need an observability tool that tracks token usage down to the individual user, session, or specific request . Several top-tier platforms specialize in this exact problem, offering lightweight integration and granular cost attribution.

The best options on the market for tracking per-user and per-request token usage include:

1. Helicone — Best for fastest setup and proxy-based tracking
   • Why it's great: Helicone acts as an open-source AI gateway . By changing just your base API URL, every request is automatically logged with zero extra application code.
   • Per-user/request features: It allows you to pass custom properties (like user_id or session_id ) via simple headers . Their dashboard natively aggregates costs by user, session, and specific prompt/model, making it trivial to spot who or what is driving up your bill.
2. Langfuse — Best open-source/flexible option for deep tracing
   • Why it's great: Langfuse is an open-source LLM engineering platform that handles prompt management, tracing, and granular cost calculation.
   • Per-user/request features: When you log generations or traces via their SDK, you can attach user IDs, tags, and metadata . Langfuse automatically maps token counts (including complex pricing like cached tokens or reasoning tokens) to exact financial costs , and you can slice-and-dice analytics by user ID in their dashboard.
3. LangSmith — Best if you are already in the LangChain/LangGraph ecosystem
   • Why it's great: Built by the creators of LangChain, LangSmith provides heavy-duty debugging, testing, and monitoring.
   • Per-user/request features: It tracks token consumption and latency per run, trace, and metadata tag. If your application architecture relies on LangChain components, it offers seamless attribution of token spend back to specific user sessions or agent chains.
4. Braintrust — Best for scaling teams balancing cost and quality
   • Why it's great: Braintrust treats logging, evaluation , and cost tracking as a unified engineering workflow with a generous free tier.
   • Per-user/request features: It logs detailed trace spans and allows you to query structured OLAP datasets to filter usage logs by user metadata or specific API calls.

To help narrow down the ideal choice, tell me:

• Are you looking for a plug-and-play proxy (like changing an endpoint URL) or an SDK you embed in your code?
• Do you prefer a cloud-hosted service or a self-hosted open-source stack?

If your main problem is unpredictable LLM spend and you want to attribute tokens/cost to individual users or requests, my first choice is langfuse.com.

Why Langfuse

• Per-request tracing: captures model calls, token usage, latency, prompts/responses, and the surrounding agent/tool workflow.
• Per-user attribution: attach a userId to traces and break down token usage and cost by user.
• Cost tracking: tracks input/output usage and model costs, including different models and pricing tiers.
• Sessions and features: add session_id, tags, metadata, or tenant IDs so you can answer questions like “Which customers/features are driving our bill?”
• Alerts + API: useful for turning observability into spend controls—e.g. alert when daily cost spikes or query per-user usage programmatically.
• Open source/self-hostable, which is a significant advantage if you're sensitive about sending prompts and user identifiers to a third party.

Strong alternative: Helicone

helicone.ai is particularly attractive if you want a gateway/proxy approach rather than instrumenting your application deeply. It can retrieve requests by user_id, calculate per-request costs, track sessions, and alert on cost thresholds.

I'd choose:

For your stated problem, I'd start with Langfuse. The key is to instrument every top-level request with user_id, session_id, tenant/feature metadata, and provider-reported token usage. Then your dashboard can show cost per request → per user → per feature → per model, rather than just a monthly provider bill.

If your main problem is “who/what is driving our LLM bill?”, I’d pick LangSmith as the best general-purpose choice.

Why LangSmith

• Per-request tracing: each LLM call can show input/output/total tokens and calculated cost.
• Per-user attribution: traces can carry user/thread metadata, letting you analyze usage by user or conversation.
• Aggregations and dashboards: track cost and token usage across your application rather than manually summing API logs.
• Alerts: you can alert on cost thresholds, useful for catching runaway usage.
• Works beyond LangChain: it supports tracing arbitrary stacks through SDKs/OpenTelemetry, so you don't have to adopt the LangChain framework.

My shortlist

Those distinctions broadly match the current tooling landscape: Langfuse emphasizes open-source/self-hosting, while Helicone emphasizes a low-latency proxy approach.

One important distinction: if you need to prevent unpredictable costs, not merely observe them, consider an LLM gateway alongside observability. LangSmith's gateway can apply spending caps at the user, API-key, workspace, or organization level.

Bottom line: Start with LangSmith if you want one tool that answers “which users, requests, models, and agent steps are costing us money?” If self-hosting is a hard requirement, I'd look at Langfuse first.

Managing unpredictable LLM spending requires granular tracking down to the user, session, or prompt level. Several top-tier observability and LLM ops tools specialize in token and cost monitoring:

• Langfuse : An open-source LLM engineering platform with robust cost and token tracking. It offers deep tracing, allows you to tag traces with user IDs/metadata, and automatically calculates costs for major model providers (OpenAI, Anthropic, etc.).
• Portkey : An enterprise-grade LLM gateway and observability suite. It provides real-time tracking of token usage, latency, and cost per user/request, plus advanced features like fallbacks, load balancing, and semantic caching to actively reduce bills.
• Helicone : A lightweight, open-source observability platform built specifically for LLM cost tracking and monitoring. It sits as a proxy between your app and the LLM provider, providing immediate analytics on token consumption per user, cost breakdowns, and caching.
• Arize Phoenix : Excellent for deep tracing, evaluation, and debugging. While it leans heavily toward evaluation and ML observability, it handles span/token tracing well, though you may need more configuration for strict per-user cost dashboards compared to proxy-based tools like Helicone or Portkey.

How to Choose

• Choose Helicone or Portkey if you want the fastest setup via a proxy/gateway that immediately captures per-user metrics and helps you optimize/cache costs.
• Choose Langfuse if you want an open-source, highly flexible tracing platform that integrates deeply via SDKs without routing all your traffic through an external proxy.

If you'd like, let me know:

• Your primary LLM providers (e.g., OpenAI, Anthropic, open-source models via vLLM)
• Whether you prefer an open-source self-hosted option or a managed cloud service

I can help you narrow down the best integration path for your architecture.

When your LLM costs become unpredictable, you need a tool that handles cost attribution —tying consumption down to a specific user, customer ID, or distinct request rather than just aggregate monthly totals.

The best tool depends on whether you prefer a proxy/gateway that sits in front of your API calls or an SDK/tracing platform integrated into your application code.

Top Tools for Tracking Token Usage & Costs

• Helicone — Best for Fast, Proxy-Based Cost Tracking
  • How it works: It acts as an open-source proxy. By changing just your API base URL, every request is automatically logged.
  • Per-user/request tracking: You pass user IDs or custom metadata via simple HTTP headers (e.g., Helicone-User-Id ), and it instantly populates granular cost breakdowns and dashboards per user.
  • Pros: Setup takes minutes; includes built-in caching and prompt management; low overhead.
• LiteLLM — Best for Gateway & Multi-Provider Spend Attribution
  • How it works: A drop-in proxy/gateway server that standardizes calls across 100+ LLM providers.
  • Per-user/request tracking: Excellent native support for tracking and enforcing spend per virtual key, user ID, and team. You can set hard budget limits or alerts per customer.
  • Pros: Ideal if you want to restrict/track usage dynamically and prevent runaway token bills before they happen.
• Langfuse — Best Open-Source / Self-Hosted Option
  • How it works: An MIT-licensed open-source LLM engineering platform. You integrate their lightweight SDK into your application.
  • Per-user/request tracking: You can attach userId and custom metadata parameters to any generation or trace. Costs are computed automatically based on the model used down to each step of a multi-turn agent execution.
  • Pros: Full data ownership (can be self-hosted) with deep tracing capabilities without routing all your traffic through a third-party proxy.
• Datadog LLM Observability — Best for Enterprise APM Integration
  • How it works: An extension of Datadog's mainstream infrastructure monitoring ecosystem.
  • Per-user/request tracking: Captures token counts, latency, and estimated costs on a per-request basis, mapping them alongside traditional application metrics and service logs.
  • Pros: Perfect if your SRE and DevOps teams already rely on Datadog—eliminating tool sprawl by placing AI spend next to infrastructure costs.

Which one should you pick?

• If you want a 5-minute setup without touching much application code, use Helicone.
• If you need an on-premise/self-hosted solution with maximum data privacy, use Langfuse.
• If you manage multiple providers and want hard budget caps per user , use LiteLLM.

To help narrow down the right fit, let me know:

• Are you looking for a managed SaaS or an open-source/self-hosted solution?
• Do you prefer a proxy-based integration (changing your API endpoint) or an SDK integration in your code?

For tracking LLM token usage and cost per user/request, my top pick is Langfuse.

Why Langfuse fits your problem

• Per-request tracing: Each generation records model, input/output tokens, latency, and cost.
• Per-user attribution: Attach a userId to traces and Langfuse aggregates token usage and spend by user.
• Cost breakdowns: Slice spend by user, session, feature, model, and prompt version.
• Accurate token accounting: It can ingest provider-reported usage or calculate usage/cost for supported models; provider-reported usage is preferable, especially for reasoning models.
• API access: Its Metrics API can give you aggregated cost/token data by user, model, or time period—useful for internal chargeback or customer usage limits.
• Self-hosting: If sending prompts/user metadata to a SaaS observability vendor is a concern, Langfuse supports self-hosting.

What I'd instrument

At minimum, attach these to every top-level request:

user_id       = "user_123"
session_id    = "conversation_456"
request_id    = "req_789"
feature       = "chat"
model         = "..."
environment   = "production"

Then your dashboard can answer questions like:

"Which users generated the most cost this month?"
"What's the average cost per chat request?"
"Which feature is responsible for the spike?"
"Which model/prompt version increased token consumption?"

One important detail: record the provider's actual token counts whenever possible, rather than merely tokenizing the prompt yourself. This matters for cached and reasoning tokens, where naive token counting can produce misleading costs.

Bottom line: If your primary pain is "I need to know exactly which request/user/feature is causing unpredictable LLM spend," I'd start with Langfuse. If you tell me your stack (OpenAI/Anthropic, Python/Node, LangChain, etc.), I can recommend the simplest instrumentation setup.

Choosing the "best" tool for tracking token usage per user or request comes down to whether you prefer an Open-Source/Trace-Level Platform , a Proxy/Gateway , or an Enterprise APM Extension.

Top options categorized by their strengths in per-user and per-request cost attribution include:

• Langfuse: Best Open-Source All-Rounder for Trace-Level Attribution . It captures hierarchical traces and allows you to attach custom dimensions like userId, sessionId , or specific features to every LLM call. It automatically calculates costs based on model pricing and is fully self-hostable (MIT licensed) if you have strict data privacy needs.
• Helicone: Best Zero-Code / Proxy Setup . If you route your LLM requests through an API proxy, Helicone lets you track cost, tokens, and latency per request instantly. You can pass user IDs via simple custom headers (Helicone-User-Id ), requiring minimal-to-no deep code instrumentation.
• LiteLLM: Best Open-Source Proxy for Multi-Provider Budgeting . Acts as an open-source proxy/gateway that normalizes APIs across 100+ LLM providers and explicitly supports spend tracking and hard budget limits per user, team, and API key.
• Braintrust: Best for Combined Cost Tracking and Evals . It features a built-in model registry and gateway that tracks token spend alongside trace data, prompt management, and quality evaluations in a unified workflow.
• Datadog LLM Observability: Best for Enterprise Teams Already on Datadog . If your infrastructure and app logs already live in Datadog, their LLM Observability extension correlates token usage and estimated costs per request right next to your standard APM metrics and dashboards.

To help narrow down which tool fits your stack best, let me know:

• Are you looking for a hosted cloud service or an open-source self-hosted tool?
• Do you need active budget enforcement/rate-limiting per user, or just cost reporting/visibility?
• Do you currently use a specific framework like LangChain , or an existing APM like Datadog?

For tracking LLM cost at the user/request level, my pick is Langfuse.

It gives you per-generation input/output token counts, costs, model, latency, and trace metadata, and its metrics API can aggregate/filter usage by user and tags—exactly what you need to answer “which users/requests are driving our bill?”

Quick comparison

Current 2026 comparisons generally put Langfuse ahead for open/self-hostable observability and cost tracking, while LangSmith is particularly compelling if you're already deep in LangChain.

What I'd instrument

At the entry point of every user request, attach:

user_id
organization_id
request_id
session_id
feature
model
environment

Then let the observability layer capture:

input_tokens
output_tokens
cached_tokens
total_tokens
cost_usd
latency

This lets you build queries such as:

• Cost per user → $0.83/user/month
• Cost per request → $0.014/request
• Top 20 expensive users
• Cost by feature → chat vs. RAG vs. agent
• Cost by model
• Token growth per session
• Requests exceeding $0.10
• Cost per organization/customer

One important point: don't rely solely on the provider's aggregate billing dashboard. You need to propagate your own user_id/tenant/feature metadata into each LLM trace. Otherwise you can know that you spent $10,000 without knowing who or what caused it.

Recommendation: If you're starting fresh, use Langfuse + OpenTelemetry-style instrumentation, and treat the resulting data as the source for your internal cost dashboard. That also keeps you less dependent on any one observability vendor.

Langfuse token & cost tracking documentation 