Skip to content
Telemetry
Data quality SQL recipe

Measure late-arriving events

Measure event delivery delay by source and identify producers that send stale or out-of-order data.

Intermediateingestion_eventsReviewed 2026-07-28Tested with Apache DataFusion 45.2.0

Reviewed by the Telemetry product team on . We checked the SQL syntax, required event fields, sample results, and limits on using the query. Who reviews this page

Question answered

Which event producers deliver data late enough to distort analysis?

Event freshness and delivery latency are different problems. Comparing event time with receive time shows whether a live source is delivering old data that can rewrite recent dashboards.

Event schema

Fields the query expects

FieldTypeWhy it exists
timestamp_utcTimestampWhen the source says the event occurred.
received_atTimestampWhen Telemetry received the event.
sourceUtf8Producer or ingestion path.
event_nameUtf8Stable event contract name.
DataFusion SQL

Copy the query

sql
SELECT
  source,
  event_name,
  COUNT(*) AS events,
  AVG(date_part('second', received_at - timestamp_utc)) / 60.0
    AS average_delay_minutes,
  approx_percentile_cont(
    date_part('second', received_at - timestamp_utc) / 60.0,
    0.95
  ) AS p95_delay_minutes,
  SUM(CASE
    WHEN received_at - timestamp_utc > INTERVAL '15 minutes' THEN 1
    ELSE 0
  END) AS events_over_15_minutes
FROM ingestion_events
WHERE received_at >= now() - INTERVAL '24 hours'
  AND received_at >= timestamp_utc
GROUP BY source, event_name
HAVING COUNT(*) >= 20
ORDER BY p95_delay_minutes DESC;

This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. We review the synthetic sample output separately. Check field types, thresholds, and counting rules against your own data. Read the testing methodology.

Query result

p95 event delivery delay

Offline mobile synchronization is live but regularly delivers old events that can change recent product cohorts.

sourceevent_nameeventsaverage_delay_minutesp95_delay_minutesevents_over_15_minutes
mobile_offline_syncfeature_used4,26018.496.21,088
billing_webhookinvoice_updated1,3801.24.811

Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.

p95 event delivery delay: static chart of synthetic p95_delay_minutes values from the Measure late-arriving events example result
Download this SVG chart of the sample results for an article, runbook, or design review. Please credit Telemetry.

Reproduce the example

Download the sample data

The JSON bundle includes the event schema with field types, illustrative input rows, exact SQL, expected output, review notes, and engine version. The CSV contains the displayed result.

How the SQL works

  1. 1Receive time bounds the operational window, while event time measures how stale each delivered record is.
  2. 2Average and p95 delay separate a generally delayed producer from a long-tail problem.
  3. 3The explicit late-event count makes the impact legible without relying only on a percentile.

Edge cases to check

  • Clock skew can produce negative or exaggerated delay; monitor and correct producer clocks separately.
  • Offline clients may be expected to deliver late data and need a different threshold from server events.
  • Decide whether dashboards restate recent buckets when late events arrive.

Recommended dashboard

  • Bars: p95_delay_minutes by source
  • Trend: late-event count by receive hour
  • Table: newest late events with source and event time

Alert guidance

Alert only for sources with a documented delivery target and a minimum event count. Track expected offline synchronization on a dashboard.

Read alert setup

Set up the events this query needs

Related instrumentation and guides

Continue the analysis

Run it on your events

Create a table, adapt the fields, and save the result

Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.

Get an API key