Event contract
Fields the query expects
| Field | Type | Why it exists |
|---|---|---|
| timestamp_utc | Timestamp | Job lifecycle event time. |
| job_id | Utf8 | Stable logical job identifier. |
| queue_name | Utf8 | Queue that owns the job. |
| event_name | Utf8 | dead_letter_created or dead_letter_resolved. |
| error_type | Utf8 | Categorized terminal failure. |
Copy the query
SELECT
date_trunc('day', timestamp_utc) AS day,
queue_name,
SUM(CASE
WHEN event_name = 'dead_letter_created' THEN 1 ELSE 0
END) AS created_jobs,
SUM(CASE
WHEN event_name = 'dead_letter_resolved' THEN 1 ELSE 0
END) AS resolved_jobs,
SUM(CASE
WHEN event_name = 'dead_letter_created' THEN 1
WHEN event_name = 'dead_letter_resolved' THEN -1
ELSE 0
END) AS net_queue_change
FROM job_events
WHERE timestamp_utc >= now() - INTERVAL '30 days'
AND event_name IN ('dead_letter_created', 'dead_letter_resolved')
GROUP BY date_trunc('day', timestamp_utc), queue_name
ORDER BY day, queue_name;This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. The deterministic sample output is synthetic and reviewed separately; validate field types, thresholds, and business definitions against your own data. Read the testing methodology.
Query result
Dead-letter jobs created and resolved
Billing accumulated 22 additional dead-letter jobs on July 26 while imports reduced its queue.
| day | queue_name | created_jobs | resolved_jobs | net_queue_change |
|---|---|---|---|---|
| 2026-07-25 | billing | 18 | 16 | 2 |
| 2026-07-26 | billing | 31 | 9 | 22 |
| 2026-07-27 | imports | 12 | 14 | -2 |
Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.
How the SQL works
- 1Created and resolved events remain separate so the same net change cannot hide very different operational workloads.
- 2The signed net change exposes days when unresolved work accumulated.
- 3Grouping by queue keeps ownership and escalation paths clear.
Edge cases to decide
- A retry is not a resolution until the logical job reaches a documented terminal success or discard outcome.
- Backfilled lifecycle events can change historical net movement.
- Track the current queue stock separately or calculate a cumulative sum from a known opening balance.
Recommended dashboard
- Stacked bars: created_jobs and resolved_jobs by queue
- Trend: cumulative dead-letter stock
- Table: oldest unresolved jobs by error_type
Alert guidance
Alert when net_queue_change remains positive for consecutive buckets or the oldest unresolved job breaches its response target.
Read alert setupPut the recipe to work
Related instrumentation and guides
Continue the analysis
Measure Background Job Retry and Failure Rate
Find unreliable jobs by comparing successful runs, retries, failures, and tail duration.
Open recipeFind Stalled Background Jobs With SQL
Join job start and finish events to identify work that exceeded its expected completion window.
Open recipeMeasure Queue Wait Time by Job
Separate time spent waiting in a queue from execution duration and compare p50 and p95 delay by job name.
Open recipeRun it on real events
Create a table, adapt the fields, and save the result
Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.