Follow one logical job across attempts
Keep a stable job identifier and separate attempt-level events from the final outcome so retry recovery remains measurable.
Selection framework
Choose the boundary that knows the result
Separate queue wait from execution
Scheduled, enqueued, started, and completed timestamps distinguish backlog pressure from slow worker code.
Define terminal states
Document completed, permanently failed, cancelled, discarded, and dead-letter outcomes before writing stalled-job queries or alerts.
Shared event contract
Keep the first schema interoperable
- job_id, job_name, queue_name, status, attempt, and release
- scheduled_at, started_at, completed_at, duration_ms, and wait_ms
- error_type, worker_name, item_count, dead_lettered, and account_id
Adapt these fields to the workflow, but keep units, status categories, identifiers, and ownership explicit before several services depend on the same table.
Verification
Exercise failure paths before alerts
- 1Run one success, retry recovery, permanent failure, and cancellation fixture.
- 2Confirm the same logical job remains correlated across attempts.
- 3Reconcile queue wait, execution duration, and end-to-end duration.
- 4Test duplicate delivery and idempotent downstream behavior.
Implementation guides
Match the framework, provider, or runtime
Sourced implementation guide
Inngest And Trigger.dev Job Telemetry
Instrument async workers, scheduled jobs, retries, failures, and dead-letter events with SQL-ready logs.
Open guideSourced implementation guide
Django and Celery Structured Event Monitoring
Connect Django request outcomes and Celery task lifecycles with safe identifiers, latency, retries, and terminal status.
Open guideSourced implementation guide
Temporal Workflow Observability
Track Temporal workflow outcomes, activity retries, schedule-to-start latency, and business milestones without recording workflow payloads.
Open guideSourced implementation guide
n8n Workflow Telemetry
Send n8n workflow completion, failure, retry, item-count, and downstream delivery outcomes to a bounded Telemetry event through the HTTP Request node.
Open guideSourced implementation guide
BullMQ Queue Monitoring
Measure BullMQ queue wait, execution duration, retries, failures, and dead-letter growth with SQL-ready lifecycle events.
Open guideSourced implementation guide
AWS SQS Queue Observability
Track SQS message age, receive attempts, processing outcomes, batch behavior, and dead-letter routing without storing message bodies.
Open guideSourced implementation guide
RabbitMQ Queue Telemetry
Track RabbitMQ publish confirmation, delivery, acknowledgement, redelivery, queue wait, retries, and dead-letter outcomes with structured events.
Open guideSourced implementation guide
Google Cloud Pub/Sub Telemetry
Track Pub/Sub publish, delivery, acknowledgement, redelivery, ordering, queue age, and dead-letter outcomes with stable message identifiers.
Open guideSourced implementation guide
Azure Service Bus Telemetry
Track Azure Service Bus sends, receives, lock renewal, settlement, redelivery, deferral, and dead-letter outcomes without collecting message bodies.
Open guideSourced implementation guide
Kafka Consumer Lag and Processing Analytics
Monitor Kafka consumer outcomes, partition lag, processing latency, retries, and poison-message handling with structured events.
Open guideSourced implementation guide
Sidekiq Job Telemetry
Track Sidekiq execution outcomes, attempts, queue wait, errors, and releases from server middleware without copying job arguments.
Open guideSourced implementation guide
Redis and node-redis Telemetry
Measure Redis command outcomes, latency, cache behavior, reconnects, and bounded error categories alongside node-redis.
Open guideSourced implementation guide
AWS Lambda Structured Event Monitoring
Track Lambda invocations, cold starts, duration, retries, and business outcomes with compact structured events.
Open guideSourced implementation guide
Stripe Webhook Telemetry
Debug payment, subscription, invoice, retry, and downstream billing sync behavior without storing webhook bodies.
Open guideSourced implementation guide
GitHub Actions Workflow Telemetry Integration
Send a bounded terminal workflow outcome from GitHub Actions with run, attempt, job, conclusion, duration, and release context.
Open guideVerify one end-to-end workflow first
Create a free API key, choose the closest runtime guide, and send one known success and failure before expanding coverage.