Telemetry
Copy-paste instrumentation template

LLM Cost Tracker

Measure model spend, token usage, latency, failure rate, and value signals by feature, user, and account.

Get an API keyCopy template
Questions this unlocks
  • Which features cost the most per retained account?
  • Which models have the best accepted-output rate per dollar?
  • Where do retries or timeouts hide margin risk?

Template

Paste this into your coding agent

Replace YOUR_API_KEY, run the flow locally, then verify the generated events and dashboards.

llm-cost-tracker

LLM Cost Tracker

text
Instrument LLM usage and cost with Telemetry.

Use /skill.md and this Telemetry API key: YOUR_API_KEY

Log each model request with:
provider, model, route, feature, user_id, team_id, input_tokens, output_tokens, total_tokens, estimated_cost_usd, latency_ms, status, retry_count, and error_type.

Connect outcome events such as copied, saved, accepted, retried, regenerated, or discarded.

Create dashboards for daily spend, cost by feature, cost by account, failures by model, p95 latency, and accepted-output rate.

Do not store raw prompts or completions by default.

Events to capture

llm_request_completedllm_request_failedllm_stream_completedllm_output_savedllm_output_discarded

More templates

AI Agent Observability Template

Track agent runs, tool calls, retries, model latency, errors, and accepted outcomes with structured SQL-ready events.

Open template

Background Job Failure Monitor

Capture queue, cron, import, billing sync, and webhook job health from the first run through retries and failures.

Open template

API Error And Latency Monitor

Watch request volume, status codes, route latency, failed endpoints, and customer-impacting API incidents.

Open template