Schema Evolution

Schema evolution is the controlled process of changing an event contract while producers, stored rows, SQL, dashboards, alerts, and exports continue to work. Telemetry can accept additive JSON fields without an up-front migration, but it cannot make a semantic or type change safe for you.

The central rule is simple: keep an existing field's meaning and type stable. Introduce a new field or version when either must change.

Compatibility matrix

Adding data is technically easy. Compatibility also depends on every downstream definition, especially controlled values, units, row grain, identity, and time semantics.

Add a field without breaking consumers

Suppose api_request_completed already records:

{
  "event_id": "evt_api_01",
  "route": "/v1/query/:id",
  "status_code": 200,
  "latency_ms": 184,
  "release": "2026.07.2"
}

You want to add a bounded failure category:

{
  "event_id": "evt_api_02",
  "route": "/v1/query/:id",
  "status_code": 503,
  "latency_ms": 921,
  "release": "2026.07.3",
  "error_type": "upstream_unavailable"
}

Deploy in stages:

1. Document the allowed values, privacy class, owner, and branch that sets the field.
2. Add a fixture for a success without error_type and each expected failure category.
3. Release the producer while existing queries still ignore the field.
4. Measure field coverage by release and status.
5. Update dashboards and alerts only after enough relevant rows contain it.
6. Keep queries tolerant of older rows for at least the retained migration window.

The Log API removes null values, empty objects, and empty arrays. “Not present” is therefore the expected stored state for an optional field with no value.

Measure adoption before depending on a field

Use release or an explicit producer version to find partially migrated code:

SELECT
  release,
  COUNT(*) AS failed_requests,
  SUM(CASE WHEN error_type IS NULL THEN 1 ELSE 0 END) AS missing_error_type,
  100.0 * SUM(CASE WHEN error_type IS NULL THEN 1 ELSE 0 END)
    / NULLIF(COUNT(*), 0) AS missing_rate_pct
FROM api_request_completed
WHERE status_code >= 500
  AND timestamp_utc >= now() - INTERVAL '24 hours'
GROUP BY release
ORDER BY release;

Do not use COALESCE(error_type, 'none') unless “none” is the intended category for every older and missing value. It can hide a broken producer rollout.

Rename, move, or redefine a field

Renaming latency_ms to duration_ms is not an in-place rename in stored event data. Use a dual-write migration:

{
  "latency_ms": 184,
  "duration_ms": 184,
  "schema_version": 2
}

During the migration window, make the precedence explicit:

SELECT
  route,
  approx_percentile_cont(
    COALESCE(duration_ms, latency_ms),
    0.95
  ) AS p95_duration_ms
FROM api_request_completed
WHERE timestamp_utc >= now() - INTERVAL '7 days'
GROUP BY route;

Then:

1. update every saved query, dashboard, alert, export, and consumer;
2. verify no current producer sends only the old field;
3. wait through the agreed compatibility period;
4. stop writing the old field;
5. keep historical-query behavior documented for the retained old rows.

Use schema_version when a query must distinguish definitions, not as a substitute for stable field names. A version is especially useful when row grain, outcome meaning, or a nested structure changes together.

Never change a field's type in place

This change is unsafe:

{ "account_id": 8421 }

{ "account_id": "acct_8421" }

The second producer conflicts with a numeric account_id established by the first. Even when the storage layer could represent both values separately, joins and filters would no longer share one reliable type.

Add a new string field such as account_key, backfill only if you have a reviewed and deterministic mapping, and migrate consumers. The same rule applies to:

• numeric durations sent as formatted strings;
• booleans replaced by "yes" and "no";
• timestamps replaced by locale-specific strings;
• one nested path changing from an object to a scalar;
• identifiers changing from one entity type to another.

Treat status values as schema

Adding cancelled to a field previously documented as success or failed does not change its string type, but it can still break logic:

SUM(CASE WHEN status = 'failed' THEN 1 ELSE 0 END)

The query silently treats cancelled as not failed. Decide whether the new value belongs in a failure, exclusion, or separate outcome before emitting it. Search the query registry for exhaustive status filters and update fixtures first.

Validate the rollout and rollback

Test representative events before production:

After deployment, compare accepted event volume, required-field coverage, controlled-value distribution, and key query results by release. A rollback is safe only if the old producer can still write the established schema and new consumers tolerate its missing fields.

Historical data and backfills

Schema evolution changes future events; it does not automatically rewrite retained history. Before a backfill:

• define the exact source of truth and deterministic transformation;
• preserve original event time and stable identifiers;
• prevent duplicates with an event or migration identifier;
• test row counts and aggregate totals on a bounded interval;
• record which dates and versions were rewritten;
• decide whether dashboards should show a mixed or backfilled history.

If the old data cannot support the new meaning, leave it missing and show a coverage boundary. Inventing a value produces a cleaner chart but less trustworthy analysis.

Schema-change checklist

1. State the current and proposed row grain, types, units, and meanings.
2. Inventory producers and every downstream query, dashboard, alert, and export.
3. Prefer an additive field; use a new event or version for a grain change.
4. Add success, failure, missing, retry, and rollback fixtures.
5. Deploy the compatible reader before the new writer when both must change.
6. Measure adoption by release instead of assuming a completed rollout.
7. Keep a documented dual-read or dual-write window.
8. Remove the old path only after usage and retained-history behavior are understood.

Continue with event data types and nullability, querying nested JSON, the event schema catalog, and the required-field null-rate recipe.