Telemetry cost and volume management
To understand telemetry cost, measure event frequency, payload size, retention, data scanned by queries, exports, and the number of distinct field values. Total bytes alone cannot explain a vendor bill. Check these measurements before cutting collection so you know which data you would lose and which costs you might reduce.
Measure the event contract first
At ingestion, record a compact operational event with the controlled event name, bounded producer source, serialized payload bytes, schema version, acceptance outcome, and environment. Do not duplicate the original payload into the measurement event.
The telemetry volume SQL recipe groups count and payload bytes by event contract. It also calculates average event size and rejection rate. This distinguishes a high-frequency small event from a lower-frequency oversized event and prevents a cost change from silently degrading data quality.
Serialized bytes are not the same as compressed storage, scanned bytes, network egress, or invoice cost. Connect the aggregate with your actual retention and pricing model before forecasting money.
Review cardinality and usefulness
A small event can still create an unusable dashboard if fields such as raw URL, request ID, error message, or user-generated label become grouping dimensions. Follow the high-cardinality fields guide and keep identifiers for targeted drill-down rather than default charts.
For each large contract, document:
- The decisions, dashboard, alert, or investigation it supports.
- Required fields and fields that can be removed or categorized.
- Production sampling rules and the questions sampling would prevent.
- Retention and deletion requirements.
- The owner who reviews schema and volume changes.
Change collection safely
Start with development noise, accidental duplicates, and fields that have no reader. Test schema changes and rejection behavior before reducing production data. Compare complete daily buckets after the rollout so a traffic change is not mistaken for an instrumentation win.
Use the data quality recipe collection to monitor freshness, duplicates, null fields, and schema adoption while changing volume. The data retention guide covers policy and deletion decisions that should remain separate from query optimization.