Event schema
Fields the query expects
| Field | Type | Why it exists |
|---|---|---|
| timestamp_utc | Timestamp | When the workflow outcome was recorded. |
| feature | Utf8 | Stable product feature. |
| model | Utf8 | Provider model identifier. |
| accepted | Boolean | Whether the user accepted or saved the result. |
| estimated_cost_usd | Float64 | Cost of the model request. |
Copy the query
SELECT
feature,
model,
COUNT(*) AS completed_outputs,
SUM(CASE WHEN accepted = true THEN 1 ELSE 0 END) AS accepted_outputs,
100.0 * SUM(CASE WHEN accepted = true THEN 1 ELSE 0 END)
/ NULLIF(COUNT(*), 0) AS acceptance_rate_pct,
SUM(estimated_cost_usd) AS cost_usd,
SUM(CASE WHEN accepted = true THEN 1 ELSE 0 END)
/ NULLIF(SUM(estimated_cost_usd), 0) AS accepted_outputs_per_dollar
FROM llm_requests
WHERE timestamp_utc >= now() - INTERVAL '30 days'
GROUP BY feature, model
HAVING COUNT(*) >= 50
ORDER BY accepted_outputs_per_dollar DESC;This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. We review the synthetic sample output separately. Check field types, thresholds, and counting rules against your own data. Read the testing methodology.
Query result
Accepted outputs per dollar
The largest model wins on acceptance rate but loses on cost-adjusted throughput.
| feature | model | completed_outputs | accepted_outputs | acceptance_rate_pct | cost_usd | accepted_outputs_per_dollar |
|---|---|---|---|---|---|---|
| support_draft | fast-small | 4,200 | 3,108 | 74 | 63 | 49.33 |
| document_summary | balanced-medium | 1,700 | 1,394 | 82 | 78.2 | 17.83 |
| research_report | reasoning-large | 510 | 464 | 90.98 | 215 | 2.16 |
Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.
Reproduce the example
Download the sample data
The JSON bundle includes the event schema with field types, illustrative input rows, exact SQL, expected output, review notes, and engine version. The CSV contains the displayed result.
How the SQL works
- 1Acceptance rate measures quality only among completed outputs. Accepted outputs per dollar adds economic efficiency.
- 2The minimum sample threshold avoids declaring a winner from a handful of requests.
- 3Define acceptance using product behavior, such as whether someone saved, sent, applied, or copied the result. A model-quality label does not measure that behavior.
Edge cases to check
- A discarded output may still be useful if it informed a later action; choose the outcome signal with product context.
- Compare models within the same feature because workflow difficulty changes acceptance.
- Use confidence intervals before making costly model migrations from small differences.
Recommended dashboard
- Bar chart: accepted_outputs_per_dollar by feature and model
- Line chart: acceptance_rate_pct over time
- Table: cost, latency, and outcome for model experiments
Alert guidance
Alert on sharp acceptance-rate drops after a model, prompt, or tool change; review cost efficiency as a dashboard rather than a paging alert.
Read alert setupSet up the events this query needs
Related instrumentation and guides
Continue the analysis
Calculate LLM cost by feature and model
Attribute model spend, tokens, request volume, and cost per request to product features.
Open recipeMeasure LLM cache savings and retry cost
Compare cached requests, retry volume, and estimated spend by model to find avoidable AI cost.
Open recipeMeasure LLM time to first token
Compare streaming responsiveness and total generation latency by model and feature.
Open recipeRun it on your events
Create a table, adapt the fields, and save the result
Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.