Event contract
Fields the query expects
| Field | Type | Why it exists |
|---|---|---|
| timestamp_utc | Timestamp | Model request completion time. |
| model | Utf8 | Provider model identifier. |
| feature | Utf8 | Product feature making the request. |
| status | Utf8 | Final request outcome. |
| time_to_first_token_ms | Float64 | Milliseconds until the first streamed token. |
| latency_ms | Float64 | Total request duration in milliseconds. |
Copy the query
SELECT
feature,
model,
COUNT(*) AS requests,
approx_percentile_cont(time_to_first_token_ms, 0.50) AS p50_ttft_ms,
approx_percentile_cont(time_to_first_token_ms, 0.95) AS p95_ttft_ms,
approx_percentile_cont(latency_ms, 0.95) AS p95_total_latency_ms
FROM llm_requests
WHERE timestamp_utc >= now() - INTERVAL '7 days'
AND status = 'success'
AND time_to_first_token_ms IS NOT NULL
GROUP BY feature, model
HAVING COUNT(*) >= 30
ORDER BY p95_ttft_ms DESC;This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. The deterministic sample output is synthetic and reviewed separately; validate field types, thresholds, and business definitions against your own data. Read the testing methodology.
Query result
p95 time to first token
Report generation has both the longest startup pause and the slowest total completion.
| feature | model | requests | p50_ttft_ms | p95_ttft_ms | p95_total_latency_ms |
|---|---|---|---|---|---|
| report_generation | large-reasoning | 842 | 1,280 | 4,620 | 18,400 |
| draft_reply | small-fast | 6,810 | 310 | 920 | 3,840 |
| search_summary | balanced | 2,420 | 480 | 1,380 | 6,210 |
Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.
How the SQL works
- 1Successful streaming requests are grouped by the product decision and model.
- 2p50 describes the typical pause while p95 exposes the tail users complain about.
- 3Total p95 latency remains beside TTFT so optimization does not improve startup while making completion worse.
Edge cases to decide
- Non-streaming requests do not have a meaningful first-token measurement.
- Provider-reported timing and client-observed timing can differ; document the measurement boundary.
- Compare models within features because prompt size and requested output length affect both metrics.
Recommended dashboard
- Grouped bars: p50_ttft_ms and p95_ttft_ms
- Trend: p95 TTFT by model and feature
- Table: slow requests with token counts and retry context
Alert guidance
Alert when a high-volume feature's p95 TTFT exceeds its user-experience objective for consecutive buckets.
Read alert setupPut the recipe to work
Related instrumentation and guides
Continue the analysis
Calculate LLM Cost by Feature and Model
Attribute model spend, tokens, request volume, and cost per request to product features.
Open recipeMeasure Accepted AI Outputs per Dollar
Connect model spend to accepted, saved, or otherwise useful product outcomes.
Open recipeMeasure LLM Cache Savings and Retry Cost
Compare cached requests, retry volume, and estimated spend by model to find avoidable AI cost.
Open recipeRun it on real events
Create a table, adapt the fields, and save the result
Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.