1. Event schema
One row in service_heartbeats, with the types used by the query made explicit.
- timestamp_utc
- Timestamp
- service_name
- Utf8
- environment
- Utf8
- status
- Utf8
How it works
How to set it up
1
Choose a signal with an owner
Choose a metric someone can act on, such as failed checkouts or API errors. Decide who should receive the alert.
2
Account for low volume and late events
Require enough events for a useful rate. Allow for normal scheduling delays and exclude time buckets that are still receiving events.
3
Test the response path
Trigger the condition with synthetic data, confirm delivery, and include enough context for the recipient to open the right query or dashboard.
Creating an alert from an Explore result
The Explore menu for adding a result to a dashboard or creating an alert.
Boundaries
What this does not replace
- Set thresholds against a baseline you have checked. Use enough events and complete time buckets to avoid misleading alerts. Telemetry does not detect anomalies automatically.
- Email delivery is not a substitute for a complete incident-management and escalation process.
- Give each alert an owner and write down what they should do when it fires.
Try the example
Read the schema, SQL, and sample result
This example includes the schema, read-only SQL, and synthetic results. Use it to check how the query works. It does not measure customer results.
2. Read-only SQL
Which expected telemetry sources have stopped sending heartbeats?
SELECT
service_name,
environment,
MAX(timestamp_utc) AS last_seen_at,
COUNT(*) AS heartbeats_in_window
FROM service_heartbeats
WHERE timestamp_utc >= now() - INTERVAL '24 hours'
AND environment = 'production'
GROUP BY service_name, environment
HAVING MAX(timestamp_utc) < now() - INTERVAL '10 minutes'
ORDER BY last_seen_at ASC;3. Synthetic result
The oldest last_seen_at value should be investigated first.
| service_name | environment | last_seen_at |
|---|---|---|
| billing_sync | production | 2026-07-27 15:04:00Z |
| email_worker | production | 2026-07-27 15:11:00Z |
Capabilities
What is included
Count, sum, average, min, max, and percentile conditions
Greater-than and less-than comparisons
Configurable evaluation interval and recent-point window
Optional exclusion of the newest incomplete bucket
Multiple email recipients and alert history
See the analysis
SQL recipes that use this capability
Detect missing service heartbeats
Find services, workers, or scheduled tasks that stopped reporting before a failure event appeared.
Open recipeFind stalled background jobs with SQL
Join job start and finish events to identify work that exceeded its expected completion window.
Open recipeCalculate API error rate by route
Rank API routes by 5xx error rate. Exclude routes with fewer than 20 requests so one failure does not dominate the results.
Open recipeCustomer evidence
How teams use this workflow
Related capabilities
Related guides and examples
Start with one production workflow
Use a focused prompt, send synthetic events, and verify the first useful query before expanding coverage.