Telemetry
Data quality SQL recipe

Measure Telemetry Volume by Event Name

Rank event types by payload bytes, average event size, and rejection rate before changing retention or collection policy.

Beginnertelemetry_ingestion_eventsReviewed 2026-07-28Tested with Apache DataFusion 45.2.0

Reviewed by the Telemetry product team on . SQL compatibility, event contract, synthetic output, and operational caveats. Editorial ownership

Question answered

Which event contracts create the most ingestion volume?

Event count alone can hide oversized payloads. This query keeps count and byte volume together, making noisy or unexpectedly large event contracts visible without pretending that payload bytes equal a vendor bill.

Event contract

Fields the query expects

FieldTypeWhy it exists
timestamp_utcTimestampIngestion attempt time in UTC.
event_nameUtf8Controlled event name.
sourceUtf8Bounded producer or SDK source.
payload_bytesInt64Serialized event size before transport compression.
acceptedBooleanWhether ingestion accepted the event.
environmentUtf8Deployment environment.
DataFusion SQL

Copy the query

sql
SELECT
  event_name,
  source,
  COUNT(*) AS events,
  SUM(payload_bytes) AS total_payload_bytes,
  AVG(payload_bytes) AS avg_payload_bytes,
  SUM(CASE WHEN accepted THEN 0 ELSE 1 END) AS rejected_events,
  100.0 * SUM(CASE WHEN accepted THEN 0 ELSE 1 END)
    / NULLIF(COUNT(*), 0) AS rejection_rate_pct
FROM telemetry_ingestion_events
WHERE timestamp_utc >= now() - INTERVAL '24 hours'
  AND environment = 'production'
GROUP BY event_name, source
ORDER BY total_payload_bytes DESC, event_name;

This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. The deterministic sample output is synthetic and reviewed separately; validate field types, thresholds, and business definitions against your own data. Read the testing methodology.

Query result

Payload volume by event contract

The synthetic debug trace has the largest byte volume despite having fewer events than page views.

event_namesourceeventstotal_payload_bytesavg_payload_bytesrejected_eventsrejection_rate_pct
debug_traceworker416,0004,000125
llm_request_completedapi36,0002,00000
page_viewedweb63,000500116.67

Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.

Payload volume by event contract: static chart of synthetic total_payload_bytes values from the Measure Telemetry Volume by Event Name example result
Indexable SVG of the deterministic example output. Download it for an article, runbook, or design review with attribution.

Reproduce the example

Download the public fixture

The JSON bundle includes the typed event contract, reproducible input rows, exact SQL, expected output, review notes, and engine version. The CSV contains the displayed result.

How the SQL works

  1. 1Total bytes and average bytes distinguish a high-frequency event from an oversized contract.
  2. 2Source keeps ownership visible when the same event name can be emitted by multiple producers.
  3. 3Rejection rate turns collection changes into a data-quality review instead of a cost-only exercise.

Edge cases to decide

  • Serialized payload size is not stored size, scan bytes, network egress, or invoice cost.
  • Measure dimension cardinality separately; a small event can still create expensive or unusable groups.
  • Exclude synthetic and development traffic before changing production retention.

Recommended dashboard

  • Bars: total_payload_bytes by event_name
  • Trend: event count and bytes by source
  • Table: average size, rejection rate, retention owner, and schema version

Alert guidance

Alert on sustained contract-size or volume changes relative to a reviewed baseline, not on a fixed global byte threshold.

Read alert setup

Put the recipe to work

Related instrumentation and guides

Continue the analysis

Run it on real events

Create a table, adapt the fields, and save the result

Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.

Get an API key