Skip to content
Telemetry
Dashboard example

LLM usage and unit economics dashboard

Attach workflow and account identifiers to model calls. Compare their token usage, latency, failures, and cost.

Reviewed by the Telemetry product team on . We checked what each event and metric represents, the SQL and sample results, and the assumptions and data freshness needed to interpret them. Who reviews this page

What one row represents

One completed model request before grouping by model.

Decision this helps you make

Choose a workflow to review for model, cache, retry, or prompt changes. Check that any savings preserve output quality.

Metric definitions

Put the denominator and units beside the chart

Use your own field names, but keep the same definition of one row. Check the linked schema before mapping your production events.

  • Requests
  • Input and output tokens
  • Estimated cost
  • Cost per successful request
Inspect the event schema
Complete DataFusion SQL

Review the query before adapting the fields

Adapt this query to your events before using it in production. Set a time range and environment filter, decide how to handle incomplete time buckets, and require enough events for a useful comparison.

SELECT
  model,
  COUNT(*) AS requests,
  SUM(input_tokens) AS input_tokens,
  SUM(output_tokens) AS output_tokens,
  ROUND(SUM(cost_usd), 4) AS estimated_cost_usd,
  ROUND(
    SUM(cost_usd)
    / NULLIF(SUM(CASE WHEN status = 'success' THEN 1 ELSE 0 END), 0),
    4
  ) AS cost_per_success_usd
FROM llm_requests
GROUP BY model
ORDER BY estimated_cost_usd DESC;
Data assumptions

Check these before publishing the result

  • Record token usage and status after the provider finishes responding.
  • Event-level cost uses a versioned price allocation.
  • The provider invoice remains the financial source of truth.
Query review

Check the query and metric definitions

  • Add workflow and account fields when allocating shared model spend.
  • Reconcile aggregate estimates to the invoice for the same complete period.
  • Compare cost with quality and success; lower cost alone is not an outcome.

What the results mean

What this result cannot prove by itself

Use the provider invoice as the financial source of truth. Event-level cost is an allocation estimate unless reconciled to billing.

Related dashboard examples

SaaS health overview

Did account adoption, application reliability, or represented revenue change enough to require a deeper review?

Inspect example

API reliability and latency

Which endpoints have rising error rates or p95 latency, and enough requests to trust the comparison?

Inspect example

Incident customer impact

Which accounts and operations were directly represented in a declared incident window?

Inspect example

Validate the definition with known data

Test the query with known successes, failures, missing values, duplicates, and values at its limits. Check the results before using it in a production dashboard or alert.

Open the SQL LabRead the dashboard guideStart with Telemetry