I need a dashboard to monitor "token burn rate" across all my internal AI apps.

Yes. I’d design it as an internal AI FinOps + observability dashboard rather than just a token counter.

A good foundation is OpenTelemetry’s GenAI conventions, which standardize model/provider identity and input/output token usage across AI applications.

Dashboard layout

1. Executive summary

Top-level cards:

• Tokens/minute — current burn rate
• Tokens/hour — projected from recent activity
• Tokens today
• Estimated cost today
• Cost/day forecast
• % change vs. yesterday
• Highest-burning app
• Highest-burning model

Then a real-time burn-rate chart:

2. App leaderboard

Clicking an app should drill down into model → endpoint → team → environment → individual request traces.

3. Token economics

Separate:

• Input tokens
• Output tokens
• Cached input tokens
• Reasoning tokens, where available
• Embedding tokens
• Tool/agent activity

This matters because newer GenAI telemetry conventions are expanding beyond a simple input/output split, so the dashboard shouldn't hard-code only two token categories.

4. Burn-rate alerts

I'd make these configurable:

• 🔴 Critical: >$X/hour
• 🟠 Warning: burn rate >25% above baseline
• 🔴 Spike: >2× normal tokens/minute
• 🟠 App budget: >80% of monthly allocation
• 🔴 Runaway agent: unusually high tokens/request
• 🟡 Efficiency regression: tokens/request up >20%

For each alert, show the likely cause: app, model, team, prompt version, deployment, or traffic increase.

Data architecture

Internal AI Apps
      │
      ├── OpenAI
      ├── Anthropic
      ├── Gemini
      ├── Bedrock
      └── Other providers
             │
             ▼
     OpenTelemetry / GenAI
             │
             ▼
       Telemetry pipeline
             │
       ┌─────┴─────┐
       ▼           ▼
   Metrics       Traces
       │           │
       └─────┬─────┘
             ▼
       AI Burn Dashboard

The key telemetry fields should include things like gen_ai.provider.name, operation, model, input tokens, output tokens, and token type. OpenTelemetry specifically defines gen_ai.client.token.usage for token consumption.

The metric I'd center everything around

Token burn rate = tokens consumed / unit of time

But I'd actually maintain three versions:

• Instantaneous: tokens/minute over the last 5–15 minutes
• Rolling: tokens/hour over the last 1–24 hours
• Forecast: projected tokens/day and month

And pair each with dollars, because 10M tokens from one model can have dramatically different economics from 10M tokens from another.

If you're building this internally, I'd recommend making the dashboard multi-tenant by app/team, with a global view for leadership and drill-down views for engineering/FinOps.

You can build a token burn rate dashboard by combining an observability backend like SigNoz or Datadog with OpenTelemetry instrumentation across your internal LLM gateways.

To track token usage and spend in real time across multiple applications, set up a centralized telemetry pipeline and configure key visual panels.

Recommended Tech Stack

• Instrumentation: OpenTelemetry (OTel) Semantic Conventions for GenAI to capture prompt tokens, completion tokens, and model names.
• Collector/Backend: SigNoz or Prometheus to aggregate time-series token metrics.
• Visualization: Grafana to build the unified operations screen.

Core Dashboard Panels

• Total Token Burn Rate (Tokens/Min): Real-time gauge showing aggregate input and output tokens consumed across all apps per minute.
• Cost Velocity ($/Hour): Multiplied calculation of prompt/completion tokens against specific vendor pricing models (e.g., OpenAI, Anthropic).
• App Breakdown Table: Distributed usage sliced by application name, model tier, and user department.

Setup Steps

If you'd like, let me know:

• What framework or gateway (LangChain, LiteLLM, custom proxy) your apps use
• Your preferred monitoring stack (Grafana, Datadog, SigNoz)

I can provide a custom configuration snippet or OpenTelemetry setup guide for your environment.

Building a custom dashboard to track token burn rate across internal AI apps requires pulling usage data (prompt tokens, completion tokens, and dollar costs) from your various LLM providers (OpenAI, Anthropic, Cohere, local models via vLLM, etc.) and centralizing it into a visualization layer.

Here is a practical architectural blueprint and toolstack to get your dashboard up and running:

• Core Data Pipeline:
  • Proxy / Gateway: Route all LLM API calls through an AI gateway like Portkey or LiteLLM . This automatically logs every token, cost, latency, and user ID without bloating your application code.
  • Telemetry / OpenTelemetry: Alternatively, instrument your apps using OpenTelemetry standards to stream custom spans/metrics directly to an observability backend.
• Storage & Aggregation:
  • Time-Series Database: Store high-frequency token events in a fast, cost-effective database like ClickHouse, Prometheus , or TimescaleDB if you are self-hosting.
  • Managed Logging: If using an AI gateway, use their native data storage or export logs to a data warehouse like BigQuery or Snowflake.
• Visualization Layer:
  • Grafana: Connect Grafana to your database or Prometheus instance to build real-time time-series graphs (e.g., tokens-per-minute, cost-per-app, cumulative daily burn).
  • Retool / Superset: Build an internal admin UI using Retool or Apache Superset if you need rich tables, filtering by internal team/department, and alerting hooks.
• Alerting & Budgeting:
  • Budget Caps: Set hard spending caps or anomaly detection alerts (via Slack/PagerDuty) using your gateway's built-in limits or custom cron jobs checking your database.

To help tailor this architecture, let me know:

• Which LLM providers/models are you primarily using?
• Are you looking for an open-source self-hosted stack or a managed SaaS solution?
• What BI or visualization tools does your team already use?

Absolutely. I’d design it around tokens per minute, cost per minute, and burn against budget, with drill-down by app, model, provider, team, and environment.

A good foundation is OpenTelemetry’s GenAI conventions, which standardize token usage and model/provider attributes such as gen_ai.client.token.usage, input/output tokens, and provider/model identity.

AI Token Burn Dashboard

Top-level KPIs

• Tokens/min: current organization-wide burn rate
• Cost/min: estimated spend rate
• Tokens today: input + output
• Cost today
• Projected monthly cost
• Budget remaining
• Burn vs. baseline: percentage above/below normal
• Active AI apps: apps emitting telemetry
• Alerts: apps exceeding their burn threshold

Main chart

App leaderboard

The important part is that burn rate should not simply mean raw tokens. I'd calculate at least:

token_burn_rate =
    (input_tokens + output_tokens) / rolling_window_minutes

cost_burn_rate =
    estimated_cost / rolling_window_minutes

projected_monthly_cost =
    current_daily_cost × days_remaining_in_month

Then expose dimensions such as:

• app
• team
• environment
• model
• provider
• region
• operation
• input/output
• cache tokens
• reasoning tokens

That last group matters because the GenAI conventions are evolving beyond a simple input/output split; cache and reasoning token categories are increasingly relevant to cost analysis.

Alerts I'd build in

• 🔴 Critical: burn rate > 2× baseline for 10 minutes
• 🟠 Warning: projected monthly spend > 80% of budget
• 🟠 Spike: app's tokens/min > 50% above its 7-day baseline
• 🔴 Runaway: unusually high tokens/request
• 🟡 Efficiency: output/input ratio suddenly deteriorates
• 🔴 Model anomaly: same workload suddenly costs materially more
• 🟡 Missing telemetry: app making LLM calls without token accounting

For implementation, I'd make OpenTelemetry → central metrics/telemetry store → dashboard, rather than having every AI app implement its own reporting. OpenTelemetry's current GenAI guidance specifically supports token-usage metrics and model/provider dimensions, making it a useful common schema across multiple providers.

If you tell me what you're running this on—Grafana, Datadog, Azure, AWS, GCP, Snowflake, or a custom web app—I can turn this into an actual dashboard specification and the instrumentation/query layer behind it.

To build a high-performance dashboard for monitoring token burn rate across your internal AI apps, you need a setup that ingests usage data (prompt/completion tokens), calculates costs and velocity in real-time, and visualizes spikes before they hit your budget.

Here is a blueprint and tech stack to get this up and running cleanly:

Recommended Tech Stack

• Data Ingestion / Logging: OpenTelemetry (OTel) or a lightweight proxy (like LiteLLM or Langfuse) sitting between your apps and the LLM providers (OpenAI, Anthropic, etc.).
• Time-Series Database: Prometheus or ClickHouse for handling high-frequency token events.
• Dashboard & Visualization: Grafana (the industry standard for real-time burn-rate alerting and graphing).
• Alternative (All-in-One AI Observability): Platforms like Langfuse or Arize Phoenix that come with built-in token cost tracking dashboards out-of-the-box.

Key Metrics to Display on Your Dashboard

Step-by-Step Implementation Guide

To tailor this architecture to your specific setup, tell me:

• Which LLM providers are you using (OpenAI, Anthropic, local Llama via vLLM, etc.)?
• Are your internal apps built using custom code (Python/Node.js) or an orchestration framework like LangChain/LlamaIndex?

Absolutely. I’d design it as an internal AI cost/usage control plane, with token burn treated like an infrastructure consumption metric.

A good foundation is OpenTelemetry GenAI semantic conventions, which standardize token usage, provider, model, operation, and related telemetry. The current conventions define gen_ai.client.token.usage and distinguish token types such as input/output; they’re also evolving to accommodate things like cached and reasoning tokens.

Dashboard layout

1. Executive strip

┌──────────────────────────────────────────────────────────────────────┐
│ AI TOKEN BURN RATE                                      Last 24h ▼   │
├──────────────┬──────────────┬──────────────┬──────────────┬──────────┤
│  18.4M       │  765K/min    │  $1,284      │  +23%        │  3       │
│ tokens/day   │ current      │ est. cost    │ vs yesterday │ alerts   │
└──────────────┴──────────────┴──────────────┴──────────────┴──────────┘

The important distinction is burn rate, not just total consumption:

burn rate = tokens consumed / unit of time

Track both:

• Input tokens/min
• Output tokens/min
• Total tokens/min
• Estimated $/min
• Projected daily/monthly spend
• Burn-rate change vs baseline

2. Burn-rate trend

Use a time-series chart showing total tokens/minute, with separate input/output series available as a toggle.

3. App leaderboard

Application          Tokens/min    24h tokens    Est. cost    Δ burn
─────────────────────────────────────────────────────────────────────
Customer Support       312K          6.8M         $482        +41% 🔴
Code Assistant         184K          4.1M         $391        +12% 🟡
Internal Search        121K          2.7M         $146         -8% 🟢
Document AI             83K          1.9M         $119        +27% 🔴
Analytics Copilot       65K          1.4M          $74         +3%

I'd make this drillable by:

App → environment → team → provider → model → operation

OpenTelemetry already provides dimensions such as provider, operation, model, and token type that are useful for these rollups.

4. Model/provider breakdown

Show where the burn is actually going:

• Provider
• Model
• Input tokens
• Output tokens
• Cached tokens
• Reasoning tokens, where available
• Cost
• Requests
• Tokens/request

This matters because a 20% increase in tokens can mean very different things depending on whether it came from more users, longer prompts, retries, or a model change.

5. Anomaly panel

I'd make this one of the most prominent sections:

🔥 BURN-RATE ALERTS

Customer Support     +41%   6.8M tokens     Prompt size increased
Document AI           +27%   1.9M tokens     Output tokens ↑ 52%
Code Assistant        +12%   4.1M tokens     Traffic increase

Potential runaway:
Analytics Copilot     4.7×   normal burn     03:42–04:17 UTC

Alert on rate, rather than merely absolute usage:

• 2× normal burn rate

• 30% hour-over-hour increase

• Tokens/request > baseline
• Output/input ratio anomaly
• Cost/minute exceeds budget
• Unexpected model/provider
• Retry-induced token explosion
• Application suddenly generating unusually long outputs

6. Cost projection

Give finance/engineering a direct answer:

MONTHLY PROJECTION

Current month       $24,810
Projected          $38,420
Budget              $35,000

                    ████████████████████░░░
                    109.8% of budget

Projected overage   $3,420

Telemetry architecture

I'd keep the dashboard itself relatively dumb:

AI Apps
   │
   ├── OpenAI
   ├── Anthropic
   ├── Gemini
   ├── Bedrock
   └── Other providers
          │
          ▼
   OpenTelemetry instrumentation
          │
          ▼
   Collector / gateway
          │
     ┌────┴─────┐
     ▼          ▼
 Metrics      Traces
     │          │
     └────┬─────┘
          ▼
   Metrics / analytics store
          │
          ▼
   ┌─────────────────────┐
   │ AI Burn Dashboard   │
   └─────────────────────┘

The key is to instrument every AI application consistently rather than building provider-specific dashboards. OpenTelemetry's current GenAI work is specifically intended to make token, latency, provider, model, and workflow information comparable across systems.

Metrics I'd standardize

At minimum:

ai.tokens.input
ai.tokens.output
ai.tokens.total
ai.tokens.cache_read
ai.tokens.cache_write
ai.tokens.reasoning

ai.requests
ai.tokens_per_request
ai.token_burn_rate

ai.cost
ai.cost_per_request
ai.cost_per_1k_tokens

ai.latency
ai.errors
ai.retries

And dimensions:

app
team
environment
provider
model
operation
token_type
region
version

One important implementation detail: don't hard-code the assumption that tokens are only input and output. The OpenTelemetry work is evolving toward additional token categories, so your aggregation layer should preserve unknown/new token types rather than silently dropping them.

If you give me your stack (for example AWS + Kubernetes + Prometheus/Grafana, Datadog, GCP, or Azure), I can turn this into a concrete dashboard schema, metric names, alert rules, and implementation architecture.

To build or deploy a dashboard that monitors your "token burn rate" across all internal AI applications, you need a central aggregation layer . Since apps typically scatter calls across OpenAI, Anthropic, and open-source models hosted elsewhere, stitching this together requires either an AI Gateway/Proxy or an LLM Observability platform.

The top-tier options to power your token-monitoring dashboard fall into a few clear categories:

1. Ready-to-Use Open-Source & Managed Proxies (Best for Real-Time Burn Tracking)

• Helicone: A drop-in open-source proxy. You change your API base URL to route through Helicone, and it instantly populates a rich dashboard tracking real-time costs, prompt/completion token splits, and latency per app or user ID.
• LiteLLM Proxy: An open-source proxy server that normalizes 100+ LLM providers into an OpenAI-standard format. It includes built-in spend tracking, budget limits per API key/team, and can export metrics directly to Prometheus/Grafana.
• Portkey: An AI gateway that provides granular request routing, guardrails, and real-time token/cost budgeting across a massive model catalog.

2. Tracing & Observability Platforms (Best for Deep Debugging + Cost)

• Langfuse: An open-source LLM engineering platform that tracks token costs down to the specific user, session, and generation trace level. It's easily self-hosted if your internal apps handle sensitive data.
• Datadog LLM Observability: If your engineering team already lives in Datadog, this natively rolls your LLM token consumption and cost attribution into your existing APM and infrastructure dashboards.

Core Metrics Your Dashboard Should Track

To get a true "burn rate" picture, configure your chosen tool to display these specific widgets:

To help narrow down the best setup, tell me:

• Are your internal apps built using a specific framework (like LangChain, LlamaIndex , or custom Python/Node scripts)?
• Do you prefer a fully managed cloud service or an open-source tool you can self-host?

Yes. I’d design this as an internal AI cost/usage control plane, rather than just a token counter.

The cleanest foundation is OpenTelemetry GenAI semantic conventions: they already standardize model/provider, input/output token usage, operation, latency, and agent/tool telemetry.

Dashboard concept

1. Executive burn-rate header

The important metric is burn rate, not merely cumulative usage:

Token burn rate = tokens consumed / unit of time I'd show both a current rolling 1-hour rate and a 24-hour rate, plus projected daily/monthly consumption.

2. Burn-rate trend

I'd add a threshold line conceptually:

• 🟢 Below expected burn
• 🟡 1–1.5× baseline
• 🔴 >1.5× baseline
• 🚨 >2× baseline or budget trajectory

This catches runaway agents and retry loops much earlier than a monthly billing report.

3. "Who is burning the tokens?"

Rank every internal application:

Then allow drill-down:

App → team → model → feature → user/session → individual request

That hierarchy is important because a spike in an application isn't necessarily the application's fault.

4. Token composition

Track separately:

• Input tokens
• Output tokens
• Cached input tokens
• Reasoning tokens, where available
• Other provider-specific token categories

I would not hard-code the dashboard to only input/output. The current GenAI conventions are evolving, including more granular token categories.

5. Cost intelligence

Don't derive cost from token count alone in the dashboard. Maintain a model pricing registry:

provider
model
effective_from
effective_to
input_price_per_1m
output_price_per_1m
cached_input_price_per_1m
currency

Then calculate:

request_cost =
    input_tokens  × input_rate
  + output_tokens × output_rate
  + cached_tokens × cached_rate
  + other_billable_units

This lets you change pricing without rewriting historical telemetry.

The telemetry contract

Every AI call should emit something along these lines:

app.name
app.environment
team
feature
user/session/conversation ID

gen_ai.provider.name
gen_ai.request.model
gen_ai.operation.name

gen_ai.usage.input_tokens
gen_ai.usage.output_tokens
gen_ai.usage.cache_read.input_tokens
gen_ai.usage.cache_creation.input_tokens

request.duration
error.type
finish_reason

OpenTelemetry specifically recommends reporting billable token counts when providers expose them, which is particularly useful for making your dashboard reconcile with actual invoices.

I'd also keep prompt/completion content off by default. The OTel guidance explicitly notes that these can contain sensitive information and recommends opting in rather than capturing them automatically.

Alerts I'd build in

Don't just alert on "over 10 million tokens."

Use anomaly-based alerts:

• Burn rate: >2× normal for 15 minutes
• Projected monthly cost: >110% of budget
• Single conversation: >250K tokens
• Application: >50% increase versus its historical baseline
• Model: unexpected shift to an expensive model
• Retry loop: unusually high LLM calls/request
• Input inflation: prompt/context tokens suddenly increase
• Output inflation: output tokens suddenly increase
• New model: application starts using an unapproved model

An especially useful alert is cost per successful task, because reducing tokens isn't necessarily good if it reduces task completion. The observability data should therefore eventually connect token consumption to application outcomes.

Architecture

I'd use:

Internal AI Apps
     │
     │ OpenTelemetry
     ▼
┌───────────────────┐
│ OTel Collector    │
│ normalize/enrich  │
└─────────┬─────────┘
          │
          ├──────────────► Metrics
          │
          ├──────────────► Traces
          │
          └──────────────► Events
                         │
                         ▼
                ┌─────────────────┐
                │ AI Usage Store  │
                └────────┬────────┘
                         │
              ┌──────────┴──────────┐
              ▼                     ▼
        Dashboard              Alert Engine
              │                     │
              ▼                     ▼
       Cost / Burn /             Slack /
       App / Model              PagerDuty

The advantage is that every internal AI application speaks the same telemetry language, regardless of whether it uses OpenAI, Anthropic, Bedrock, Gemini, or another provider. OTel's current conventions explicitly include provider identifiers for those ecosystems.

My recommended MVP

I'd build v1 with five screens/panels:

1. Burn rate — tokens/hour, projected daily/monthly usage
2. Spend — actual + projected cost
3. Applications — ranked by tokens and cost
4. Models — usage, cost, latency, token mix
5. Anomalies — automatically detected spikes and runaway sessions

Then make every row clickable down to the underlying trace/request.

That gives you a dashboard that answers the three questions that matter most:

"Are we burning tokens faster than expected?" → "Where are they going?" → "Why?"

• Input tokens
• Output tokens
• Cached input tokens
• Reasoning tokens, where available
• Other provider-specific token categories

I would not hard-code the dashboard to only input/output. The current GenAI conventions are evolving, including more granular token categories.

5. Cost intelligence

Don't derive cost from token count alone in the dashboard. Maintain a model pricing registry:

provider
model
effective_from
effective_to
input_price_per_1m
output_price_per_1m
cached_input_price_per_1m
currency

Then calculate:

request_cost =
    input_tokens  × input_rate
  + output_tokens × output_rate
  + cached_tokens × cached_rate
  + other_billable_units

This lets you change pricing without rewriting historical telemetry.

The telemetry contract

Every AI call should emit something along these lines:

app.name
app.environment
team
feature
user/session/conversation ID

gen_ai.provider.name
gen_ai.request.model
gen_ai.operation.name

gen_ai.usage.input_tokens
gen_ai.usage.output_tokens
gen_ai.usage.cache_read.input_tokens
gen_ai.usage.cache_creation.input_tokens

request.duration
error.type
finish_reason

OpenTelemetry specifically recommends reporting billable token counts when providers expose them, which is particularly useful for making your dashboard reconcile with actual invoices.

I'd also keep prompt/completion content off by default. The OTel guidance explicitly notes that these can contain sensitive information and recommends opting in rather than capturing them automatically.

Alerts I'd build in

Don't just alert on "over 10 million tokens."

Use anomaly-based alerts:

• Burn rate: >2× normal for 15 minutes
• Projected monthly cost: >110% of budget
• Single conversation: >250K tokens
• Application: >50% increase versus its historical baseline
• Model: unexpected shift to an expensive model
• Retry loop: unusually high LLM calls/request
• Input inflation: prompt/context tokens suddenly increase
• Output inflation: output tokens suddenly increase
• New model: application starts using an unapproved model

An especially useful alert is cost per successful task, because reducing tokens isn't necessarily good if it reduces task completion. The observability data should therefore eventually connect token consumption to application outcomes.

Architecture

I'd use:

Internal AI Apps
     │
     │ OpenTelemetry
     ▼
┌───────────────────┐
│ OTel Collector    │
│ normalize/enrich  │
└─────────┬─────────┘
          │
          ├──────────────► Metrics
          │
          ├──────────────► Traces
          │
          └──────────────► Events
                         │
                         ▼
                ┌─────────────────┐
                │ AI Usage Store  │
                └────────┬────────┘
                         │
              ┌──────────┴──────────┐
              ▼                     ▼
        Dashboard              Alert Engine
              │                     │
              ▼                     ▼
       Cost / Burn /             Slack /
       App / Model              PagerDuty

The advantage is that every internal AI application speaks the same telemetry language, regardless of whether it uses OpenAI, Anthropic, Bedrock, Gemini, or another provider. OTel's current conventions explicitly include provider identifiers for those ecosystems.

My recommended MVP

I'd build v1 with five screens/panels:

1. Burn rate — tokens/hour, projected daily/monthly usage
2. Spend — actual + projected cost
3. Applications — ranked by tokens and cost
4. Models — usage, cost, latency, token mix
5. Anomalies — automatically detected spikes and runaway sessions

Then make every row clickable down to the underlying trace/request.

That gives you a dashboard that answers the three questions that matter most:

To build an effective dashboard for monitoring token burn rate across your internal AI applications, you need a clear architecture that captures metrics from your LLM gateways, visualizes usage trends in real-time, and sets up alerts before costs spiral.

Here is a blueprint and tech stack to set this up cleanly:

Recommended Architecture & Tech Stack

• Data Ingestion: OpenTelemetry (OTel) or a proxy like LiteLLM Proxy / Portkey to intercept and log every API call (tokens in, tokens out, latency, model name, and user/app ID).
• Time-Series Database: Prometheus or ClickHouse to store high-volume token usage metrics over time.
• Visualization Dashboard: Grafana as the frontend dashboard to build graphs, heatmaps, and cost counters.

Core Metrics to Track on Your Dashboard

• Total Burn Rate: Real-time sum of input and output tokens consumed per minute/hour (tokens/min).
• Cost Velocity: Estimated financial burn rate calculated via model-specific pricing (USD/hour).
• Token Breakdown: Split by input tokens (prompts) vs. output tokens (completions), since output tokens are typically more expensive.
• App-Level Attribution: Grouped usage by internal application name, department, or user ID.
• Error & Retries Rate: Failed tokens or rate-limit hits that still consume resources or block execution.

Step-by-Step Implementation Guide

To help tailor this dashboard setup, could you share:

• Which LLM providers or models are your internal apps using (OpenAI, Anthropic, local open-source via vLLM)?
• What infrastructure do you currently prefer for monitoring or logging (e.g., Datadog, Prometheus/Grafana, ELK stack)?

Absolutely. I’d design this as an internal AI FinOps / observability dashboard centered on rate, not just cumulative token counts.

OpenTelemetry’s current GenAI conventions already give you a useful foundation: gen_ai.client.token.usage tracks token consumption, with input/output token types, and model/provider attributes let you slice usage across applications and providers.

Dashboard layout

1. Executive strip

• Tokens/min — all apps
• Tokens/hour — current
• Tokens today
• Estimated spend today
• Projected monthly spend
• Burn vs budget
• Apps exceeding burn threshold

2. Token burn rate

Use a time-series chart as the primary visualization:

I would calculate burn rate over rolling windows—e.g. 5 min, 1 hour, and 24 hours—so a single burst doesn't obscure the longer-term trend.

3. App leaderboard

The important thing is to make the leaderboard sortable by instantaneous burn, daily consumption, cost, and rate-of-change.

4. Drill-down dimensions

Every metric should be filterable by:

• Application
• Team
• Environment
• Provider
• Model
• Input vs. output tokens
• User/tenant
• Feature/workflow
• Region
• Time window

OpenTelemetry specifically supports model and provider dimensions alongside token usage, making this kind of cross-application aggregation practical.

5. Alerts

I'd make these first-class dashboard objects:

• 🔥 Burn spike: >2× normal rate for 10 minutes
• 💰 Budget acceleration: projected monthly spend exceeds budget
• 📈 Runaway app: 30-minute burn rate >50% above its baseline
• 🔄 Retry storm: token consumption rising without corresponding user traffic
• 🧠 Model regression: tokens/request suddenly increases
• 🚨 Daily budget: projected daily consumption exceeds threshold

Crucially, distinguish high usage caused by high traffic from high usage per request. Otherwise the dashboard will generate a lot of noise.

Telemetry architecture

I'd standardize every LLM call around something like:

AI App
  │
  ├── service.name = customer-support
  ├── team = support
  ├── environment = production
  └── LLM call
       ├── provider = openai
       ├── model = <model>
       ├── input_tokens
       ├── output_tokens
       ├── latency
       └── request/trace ID
              │
              ▼
        OpenTelemetry
              │
              ├── Metrics → token burn rates
              ├── Traces  → individual expensive calls
              └── Logs   → incidents/anomalies
              │
              ▼
        Your dashboard

OpenTelemetry's GenAI observability guidance explicitly describes using token metrics for estimating cost, detecting token-hungry prompts, and comparing models/agents.

For cost, I'd keep a separate pricing table rather than hard-coding prices into application telemetry. Then:

estimated_cost = input_tokens × input_price + output_tokens × output_price

That lets you change provider/model pricing without redeploying your AI applications.

If you tell me what you're using for telemetry/storage (e.g. Prometheus/Grafana, Datadog, OpenObserve, ClickHouse, Snowflake, or something custom), I can turn this into a concrete dashboard architecture and metric schema.