Telemetry
Data quality SQL recipe

Measure Late-Arriving Events

Measure event delivery delay by source and identify producers that send stale or out-of-order data.

Intermediateingestion_eventsReviewed 2026-07-28Tested with Apache DataFusion 45.2.0

Question answered

Which event producers deliver data late enough to distort analysis?

Event freshness and delivery latency are different problems. Comparing event time with receive time shows whether a live source is delivering old data that can rewrite recent dashboards.

Event contract

Fields the query expects

FieldTypeWhy it exists
timestamp_utcTimestampWhen the source says the event occurred.
received_atTimestampWhen Telemetry received the event.
sourceUtf8Producer or ingestion path.
event_nameUtf8Stable event contract name.
DataFusion SQL

Copy the query

sql
SELECT
  source,
  event_name,
  COUNT(*) AS events,
  AVG(date_part('second', received_at - timestamp_utc)) / 60.0
    AS average_delay_minutes,
  approx_percentile_cont(
    date_part('second', received_at - timestamp_utc) / 60.0,
    0.95
  ) AS p95_delay_minutes,
  SUM(CASE
    WHEN received_at - timestamp_utc > INTERVAL '15 minutes' THEN 1
    ELSE 0
  END) AS events_over_15_minutes
FROM ingestion_events
WHERE received_at >= now() - INTERVAL '24 hours'
  AND received_at >= timestamp_utc
GROUP BY source, event_name
HAVING COUNT(*) >= 20
ORDER BY p95_delay_minutes DESC;

This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. The deterministic sample output is synthetic and reviewed separately; validate field types, thresholds, and business definitions against your own data. Read the testing methodology.

Query result

p95 event delivery delay

Offline mobile synchronization is live but regularly delivers old events that can change recent product cohorts.

sourceevent_nameeventsaverage_delay_minutesp95_delay_minutesevents_over_15_minutes
mobile_offline_syncfeature_used4,26018.496.21,088
billing_webhookinvoice_updated1,3801.24.811

Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.

How the SQL works

  1. 1Receive time bounds the operational window, while event time measures how stale each delivered record is.
  2. 2Average and p95 delay separate a generally delayed producer from a long-tail problem.
  3. 3The explicit late-event count makes the impact legible without relying only on a percentile.

Edge cases to decide

  • Clock skew can produce negative or exaggerated delay; monitor and correct producer clocks separately.
  • Offline clients may be expected to deliver late data and need a different threshold from server events.
  • Decide whether dashboards restate recent buckets when late events arrive.

Recommended dashboard

  • Bars: p95_delay_minutes by source
  • Trend: late-event count by receive hour
  • Table: newest late events with source and event time

Alert guidance

Alert only for sources with a documented delivery objective and meaningful volume; expected offline synchronization usually belongs on a dashboard.

Read alert setup

Put the recipe to work

Related instrumentation and guides

Continue the analysis

Run it on real events

Create a table, adapt the fields, and save the result

Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.

Get an API key