Measure Accepted AI Outputs per Dollar

Event contract

Fields the query expects

Field	Type	Why it exists
timestamp_utc	Timestamp	When the workflow outcome was recorded.
feature	Utf8	Stable product feature.
model	Utf8	Provider model identifier.
accepted	Boolean	Whether the user accepted or saved the result.
estimated_cost_usd	Float64	Cost of the model request.

DataFusion SQL

Copy the query

sql

SELECT
  feature,
  model,
  COUNT(*) AS completed_outputs,
  SUM(CASE WHEN accepted = true THEN 1 ELSE 0 END) AS accepted_outputs,
  100.0 * SUM(CASE WHEN accepted = true THEN 1 ELSE 0 END)
    / NULLIF(COUNT(*), 0) AS acceptance_rate_pct,
  SUM(estimated_cost_usd) AS cost_usd,
  SUM(CASE WHEN accepted = true THEN 1 ELSE 0 END)
    / NULLIF(SUM(estimated_cost_usd), 0) AS accepted_outputs_per_dollar
FROM llm_requests
WHERE timestamp_utc >= now() - INTERVAL '30 days'
GROUP BY feature, model
HAVING COUNT(*) >= 50
ORDER BY accepted_outputs_per_dollar DESC;

This recipe is read-only and reviewed against Telemetry's DataFusion query patterns. Sample output is deterministic and synthetic; validate field types, thresholds, and business definitions against your own data.

Query result

Accepted outputs per dollar

The largest model wins on acceptance rate but loses on cost-adjusted throughput.

support_draft

49.33 / USD

document_summary

17.83 / USD

research_report

2.16 / USD

feature	model	completed_outputs	accepted_outputs	acceptance_rate_pct	cost_usd	accepted_outputs_per_dollar
support_draft	fast-small	4,200	3,108	74	63	49.33
document_summary	balanced-medium	1,700	1,394	82	78.2	17.83
research_report	reasoning-large	510	464	90.98	215	2.16

Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.

How the SQL works

1Acceptance rate measures quality only among completed outputs. Accepted outputs per dollar adds economic efficiency.
2The minimum sample threshold avoids declaring a winner from a handful of requests.
3Acceptance must be defined from real product behavior such as saved, sent, applied, or copied—not an assumed model-quality label.

Edge cases to decide

A discarded output may still be useful if it informed a later action; choose the outcome signal with product context.
Compare models within the same feature because workflow difficulty changes acceptance.
Use confidence intervals before making costly model migrations from small differences.

Recommended dashboard

Bar chart: accepted_outputs_per_dollar by feature and model
Line chart: acceptance_rate_pct over time
Table: cost, latency, and outcome for model experiments

Alert guidance

Alert on sharp acceptance-rate drops after a model, prompt, or tool change; review cost efficiency as a dashboard rather than a paging alert.

Read alert setup

Put the recipe to work

Related instrumentation and guides

AI agent use case LLM cost recipe OpenAI Agents integration

Continue the analysis

Calculate LLM Cost by Feature and Model

Attribute model spend, tokens, request volume, and cost per request to product features.

Open recipe

Measure LLM Cache Savings and Retry Cost

Compare cached requests, retry volume, and estimated spend by model to find avoidable AI cost.

Open recipe

Run it on real events

Create a table, adapt the fields, and save the result

Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.

Get an API key