Telemetry
Data quality SQL recipe

Measure Event Ingestion Freshness

Find event sources that stopped delivering data or are arriving substantially later than they occurred.

Beginnertelemetry_eventsReviewed 2026-07-27Tested with Apache DataFusion 45.2.0

Question answered

Which production event sources are stale or delayed right now?

A healthy dashboard can become stale without showing an error. Comparing event time, receive time, and the current clock separates a silent source from delayed delivery.

Event contract

Fields the query expects

FieldTypeWhy it exists
timestamp_utcTimestampWhen the event occurred in the producer.
received_atTimestampWhen the ingestion service accepted the event.
sourceUtf8Stable producer or workflow name.
environmentUtf8Deployment environment.
DataFusion SQL

Copy the query

sql
SELECT
  source,
  MAX(timestamp_utc) AS latest_event_at,
  MAX(received_at) AS latest_received_at,
  date_part('second', now() - MAX(received_at)) / 60.0
    AS minutes_since_receive,
  AVG(date_part('second', received_at - timestamp_utc)) / 60.0
    AS average_delivery_delay_minutes
FROM telemetry_events
WHERE received_at >= now() - INTERVAL '24 hours'
  AND environment = 'production'
GROUP BY source
ORDER BY minutes_since_receive DESC;

This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. The deterministic sample output is synthetic and reviewed separately; validate field types, thresholds, and business definitions against your own data. Read the testing methodology.

Query result

Minutes since the newest received event

Billing sync is stale even though its normal delivery delay is small.

sourcelatest_event_atlatest_received_atminutes_since_receiveaverage_delivery_delay_minutes
billing_sync2026-07-27 12:112026-07-27 12:12481.4
api_gateway2026-07-27 12:592026-07-27 12:5910.1
queue_workers2026-07-27 12:582026-07-27 12:5910.8

Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.

How the SQL works

  1. 1MAX(received_at) measures whether each source is still delivering data.
  2. 2The average difference between received_at and timestamp_utc measures delivery delay separately from source silence.
  3. 3The production filter prevents development traffic from making a stopped production source look healthy.

Edge cases to decide

  • A source that emits only once per day needs a different freshness threshold from an API source.
  • Producer clock skew can make delivery delay negative; monitor and correct clock synchronization.
  • A 24-hour lookback cannot return sources that have been silent longer than the window. Keep an expected-source registry for absence monitoring.

Recommended dashboard

  • Bar chart: minutes_since_receive by source
  • Trend: average delivery delay by source
  • Table: expected sources missing from the current window

Alert guidance

Alert when minutes_since_receive exceeds the source schedule plus a documented grace period.

Read alert setup

Put the recipe to work

Related instrumentation and guides

Continue the analysis

Run it on real events

Create a table, adapt the fields, and save the result

Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.

Get an API key