Telemetry
API reliability SQL recipe

Calculate p50, p95, and p99 API Latency

Compare median and tail latency by endpoint with DataFusion-compatible percentile SQL.

Intermediateapi_requestsReviewed 2026-07-27

Question answered

Which endpoints have the worst tail latency?

Averages hide the slow requests users remember. Comparing p50, p95, and p99 reveals whether an endpoint is consistently slow or has a long tail caused by a smaller set of requests.

Event contract

Fields the query expects

FieldTypeWhy it exists
timestamp_utcTimestampRequest completion time in UTC.
route_templateUtf8Stable route template.
latency_msFloat64Request duration in milliseconds.
status_codeInt64HTTP response status code.
DataFusion SQL

Copy the query

sql
SELECT
  route_template,
  COUNT(*) AS requests,
  approx_percentile_cont(latency_ms, 0.50) AS p50_ms,
  approx_percentile_cont(latency_ms, 0.95) AS p95_ms,
  approx_percentile_cont(latency_ms, 0.99) AS p99_ms
FROM api_requests
WHERE timestamp_utc >= now() - INTERVAL '24 hours'
  AND status_code < 500
GROUP BY route_template
HAVING COUNT(*) >= 50
ORDER BY p95_ms DESC
LIMIT 10;

This recipe is read-only and reviewed against Telemetry's DataFusion query patterns. Sample output is deterministic and synthetic; validate field types, thresholds, and business definitions against your own data.

Query result

p95 latency by route

Export requests have the slowest tail and the largest gap between typical and worst-case performance.

route_templaterequestsp50_msp95_msp99_ms
/api/reports/export2846404,8208,110
/api/projects/:id/sync8603182,1403,910
/api/search2,41092410760

Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.

How the SQL works

  1. 1approx_percentile_cont uses DataFusion's t-digest implementation, which is efficient for large event tables.
  2. 2Filtering out 5xx responses makes this a successful-request latency view. Create a separate failed-request view when failure duration matters.
  3. 3The difference between p50 and p95 is often more useful than either number alone: a wide gap points to intermittent slow paths.

Edge cases to decide

  • Do not compare endpoints with fundamentally different work without preserving route context.
  • Streaming routes and long-running exports may need separate service-level objectives.
  • Keep latency in a consistent unit and use a numeric field.

Recommended dashboard

  • Grouped bars: p50_ms and p95_ms by route_template
  • Line chart: p95_ms over time for the slowest routes
  • Table: individual requests above the route's p99

Alert guidance

Alert when a high-volume route's p95 exceeds its service objective for three complete time buckets.

Read alert setup

Put the recipe to work

Related instrumentation and guides

Continue the analysis

Run it on real events

Create a table, adapt the fields, and save the result

Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.

Get an API key