OpenAI cost monitoring dashboard and alerts

What to record and check

Calculate the cost of a completed task

Record the pricing version used for each cost estimate. Join costs to retries and final task results to see what each completed task cost.

1
Model request
Capture provider, model, feature, tokens, latency, cache, and retry context.
2
Cost estimate
Apply a dated price source and retain its version beside the estimate.
3
Product outcome
Connect the request to accepted, resolved, saved, escalated, or discarded work.
4
Unit economics
Compare cost per reviewed outcome, not token or request volume alone.

Use case versus template

Choose what to measure

Use this guide to choose what to measure and when to log it. For a shorter setup prompt, open the matching template.

Open LLM cost tracker

Agent prompt

Paste this into your coding agent

Replace YOUR_API_KEY after signup, then ask the agent to run the product flow and verify the first events.

agent prompt

OpenAI cost monitoring setup prompt

text

Instrument this project with Telemetry so we can understand OpenAI and LLM usage.

Use /skill.md and this Telemetry API key: YOUR_API_KEY

Please log:
1. Every model request with model, provider, route, feature, input_tokens, output_tokens, total_tokens, estimated_cost_usd, latency_ms, status, and error_type when relevant.
2. Tool calls made by the agent or assistant, including tool_name, status, latency_ms, and result_category.
3. User-facing AI workflow outcomes, including feature, status, retry_count, and whether the user accepted, copied, saved, or discarded the result.
4. A dashboard with daily cost, cost by feature, failures by model, p95 latency, and accepted output rate.

Keep prompts and raw completions out of telemetry unless I explicitly approve storing them.

Setup steps

1Log each model request without storing raw prompts by default.
2Record tokens, estimated cost, latency, provider, model, and workflow.
3Connect product outcome events like saved, copied, accepted, or retried.
4Create dashboards for model costs, failure rates, and accepted outputs.

Events to capture

llm_request_completedllm_request_failedassistant_tool_calledai_output_acceptedai_output_discardedai_feature_retained

Questions you can answer

Which AI features cost the most per activated user?
Which model has the best accepted-output rate per dollar?
Where are retries or timeouts damaging conversion?

LLM cost and unit economics

Estimate what each accepted output costs

An inexpensive request can still be wasteful if you retry it or discard the answer. Estimate cost per accepted output here, then use the SQL Lab to compare costs by account and feature.

Estimate monthly cost

The price inputs are examples, not current provider prices. Replace them with the rates and billable token categories from your provider agreement.

Logical requests / monthInput tokens / requestOutput tokens / requestInput USD / 1M tokensOutput USD / 1M tokensAutomatic retry rate, percentAccepted output rate, percent

Provider attempts

105,000

Estimated monthly cost

$304.50

Estimated retry cost

$14.50

Accepted outputs

60,000

Cost per accepted output

$0.0051

This browser-only estimate is not sent to Telemetry and is not a provider invoice.

Use consistent fields across providers

Store each provider's usage in the same event fields. Your dashboard can then compare models and prices without a separate query for each provider.

Normalized fields	Why they belong together
provider, model, feature	Record the provider and the application action that made the request.
input_tokens, output_tokens	Record the token counts returned by the provider.
cached_input_tokens, reasoning_tokens	Keep optional billable categories separate when available.
estimated_cost_usd, pricing_version	Record which prices you used so you can recalculate the estimate later.
retry_count, cache_hit, latency_ms	Check whether retries or cache misses account for higher cost or latency.
accepted, saved, discarded, human_handoff	Record whether the user accepted, saved, or discarded the output, or needed human help.

Run the joined analysis

Use the shared accounts and llm_requests tables to calculate cost per accepted output by plan.

Open SQL Lab

Break cost down by feature

See the event fields and DataFusion query behind the example chart. Use the result to plan your dashboard.

Open cost recipe

Measure retry and cache cost

Check how retries and caching affect the cost of each user request.

Open retry recipe

Check estimates against your invoice

Your provider invoice determines what you owe. Use event cost estimates to compare features, investigate margins, and find unexpected spending.

Read the OpenAI API cost tracking implementation guide

Example event schemas

Check what each event records, when to send it, and which field types it needs. Review the example payload and privacy checklist before using it in production.

AI operations10 fields