Event schema
Fields the query expects
| Field | Type | Why it exists |
|---|---|---|
| timestamp_utc | Timestamp | UTC finish-event time; use a fixed analysis window in production. |
| job_id | Utf8 | One fictional completed logical job per row. |
| schema_version | Int64 | Version of the event contract; 1 uses duration_ms and 2 uses latency_ms. |
| duration_ms | Int64 | Release-1 duration, null on release 2. |
| latency_ms | Int64 | Release-2 duration, null on release 1. |
Copy the query
WITH normalized AS (
SELECT
job_id,
schema_version,
duration_ms,
CASE schema_version
WHEN 1 THEN duration_ms
WHEN 2 THEN latency_ms
END AS measured_ms
FROM job_finished
)
SELECT
COUNT(*) AS completed_jobs,
COUNT(duration_ms) AS jobs_in_old_average,
ROUND(AVG(duration_ms), 1) AS old_average_ms,
COUNT(measured_ms) AS jobs_with_duration,
SUM(CASE WHEN measured_ms IS NULL THEN 1 ELSE 0 END) AS unmeasured_jobs,
ROUND(100.0 * COUNT(measured_ms) / NULLIF(COUNT(*), 0), 1)
AS duration_coverage_pct,
ROUND(AVG(measured_ms), 1) AS schema_aware_average_ms
FROM normalized;This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. We review the synthetic sample output separately. Check field types, thresholds, and counting rules against your own data. Read the testing methodology.
Query result
A plausible average can cover only half the jobs
The old field averages two fictional jobs at 475 ms. The version-aware query covers all four at 580 ms.
| completed_jobs | jobs_in_old_average | old_average_ms | jobs_with_duration | unmeasured_jobs | duration_coverage_pct | schema_aware_average_ms |
|---|---|---|---|---|---|---|
| 4 | 2 | 475 | 4 | 0 | 100 | 580 |
Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.
Reproduce the example
Download the sample data
The JSON bundle includes the event schema with field types, reproducible input rows, exact SQL, expected output, review notes, and engine version. The CSV contains the displayed result.
How the SQL works
- 1The old query AVG(duration_ms) still returns 475 ms because SQL averages ignore nulls. Only two of four completed jobs contribute to it.
- 2The version-aware CASE maps release 1 to duration_ms and release 2 to latency_ms, giving 580 ms across all four fictional jobs.
- 3Keep duration coverage beside the average. An unknown version or a missing field can leave the average unchanged while coverage falls.
Edge cases to check
- Use a fixed UTC time window when comparing releases; this four-row lesson leaves all fictional rows in scope.
- This example assumes one finish row per completed logical job. Deduplicate repeated finish events by job_id before using it for a real job count.
- A field rename is comparable only if both versions time the same start and stop points in the same unit.
- A missing or unknown schema_version maps to null and lowers coverage; do not silently coalesce it to an old field.
- The four rows are fictional and do not describe a customer, a real incident, or measured product performance.
Recommended dashboard
- Table: completed jobs, duration coverage, and average by release
- Trend: duration coverage percentage over deploy time
Alert guidance
Alert on falling duration coverage only after establishing an expected minimum event count and documenting the version mapping.
Read alert setupSet up the events this query needs
Related instrumentation and guides
Continue the analysis
Count sensor deliveries and measurements separately
Run SQL on six fictional sensor deliveries. A network retry adds rows without adding measurements, changing a row-weighted average.
Open recipeMeasure event ingestion freshness
Find event sources that stopped delivering data or are arriving substantially later than they occurred.
Open recipeFind duplicate event ids
Identify event identifiers delivered more than once and measure whether duplicate handling is working.
Open recipeRun it on your events
Create a table, adapt the fields, and save the result
Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.