Telemetry
Dashboard example

LLM usage and unit economics dashboard

Connect model calls to product workflows and accounts so token volume, latency, failures, and cost can be reviewed together.

Reviewed by the Telemetry product team on . Decision, event grain, metrics, DataFusion SQL, synthetic result, assumptions, freshness, and interpretation boundary. Review standards and ownership

Row grain

One terminal model-provider request before grouping by model.

Decision supported

Choose a workflow for quality-preserving model, cache, retry, or prompt-efficiency review.

Metric contract

Put the denominator and units beside the chart

Adapt these names to your contract while preserving the declared grain. Inspect the linked event schema before mapping fields from production producers.

  • Requests
  • Input and output tokens
  • Estimated cost
  • Cost per successful request
Inspect the event schema
Complete DataFusion SQL

Review the query before adapting the fields

This public query is a starting definition. Add a bounded time filter, complete-bucket policy, environment, and minimum volume appropriate to your production event contract.

SELECT
  model,
  COUNT(*) AS requests,
  SUM(input_tokens) AS input_tokens,
  SUM(output_tokens) AS output_tokens,
  ROUND(SUM(cost_usd), 4) AS estimated_cost_usd,
  ROUND(
    SUM(cost_usd)
    / NULLIF(SUM(CASE WHEN status = 'success' THEN 1 ELSE 0 END), 0),
    4
  ) AS cost_per_success_usd
FROM llm_requests
GROUP BY model
ORDER BY estimated_cost_usd DESC;
Data assumptions

Prove these before publishing the result

  • Token usage and status are final at the provider-response boundary.
  • Event-level cost uses a versioned price allocation.
  • The provider invoice remains the financial source of truth.
Query review

Checks that keep the dashboard trustworthy

  • Add workflow and account fields when allocating shared model spend.
  • Reconcile aggregate estimates to the invoice for the same complete period.
  • Compare cost with quality and success; lower cost alone is not an outcome.

Interpretation boundary

What this result cannot prove by itself

Use the provider invoice as the financial source of truth. Event-level cost is an allocation estimate unless reconciled to billing.

Related dashboard examples

SaaS health overview

Did account adoption, application reliability, or represented revenue change enough to require a deeper review?

Inspect example

API reliability and latency

Which sufficiently busy endpoint has the broadest error or tail-latency regression?

Inspect example

Incident customer impact

Which accounts and operations were directly represented in a declared incident window?

Inspect example

Validate the definition with known data

Run a fixture with known success, failure, missing, duplicate, and boundary cases before connecting the query to a production dashboard or alert.

Open the SQL LabRead the dashboard guideStart with Telemetry