Saltar al contenido
Calidad de datos receta SQL

Catch event-field renames that hide rows from averages

Compare an old timing-field average with schema-aware coverage so an event rename cannot silently drop completed jobs.

Intermediojob_finishedRevisado 2026-09-30Probado con Apache DataFusion 45.2.0

Revisado por el equipo de producto Telemetry en . Revisamos la sintaxis SQL, los campos necesarios, los resultados de ejemplo y las limitaciones de la consulta. Quién revisa esta página

Pregunta respondida

Did a field rename make the duration average ignore completed jobs?

A query can keep returning a plausible average after instrumentation changes. Count all completed events, map each documented schema version to its timing field, and show the share with a usable value beside the average.

Esquema del evento

Campos que espera la consulta

CampoTipoPor que existe
timestamp_utcTimestampUTC finish-event time; use a fixed analysis window en production.
job_idUtf8One fictional completed logical job per row.
schema_versionInt64Version of the event contract; 1 uses duration_ms and 2 uses latency_ms.
duration_msInt64Release-1 duration, null on release 2.
latency_msInt64Release-2 duration, null on release 1.
DataFusionSQL

Copiar la consulta

sql
WITH normalized AS (
  SELECT
    job_id,
    schema_version,
    duration_ms,
    CASE schema_version
      WHEN 1 THEN duration_ms
      WHEN 2 THEN latency_ms
    END AS measured_ms
  FROM job_finished
)
SELECT
  COUNT(*) AS completed_jobs,
  COUNT(duration_ms) AS jobs_in_old_average,
  ROUND(AVG(duration_ms), 1) AS old_average_ms,
  COUNT(measured_ms) AS jobs_with_duration,
  SUM(CASE WHEN measured_ms IS NULL THEN 1 ELSE 0 END) AS unmeasured_jobs,
  ROUND(100.0 * COUNT(measured_ms) / NULLIF(COUNT(*), 0), 1)
    AS duration_coverage_pct,
  ROUND(AVG(measured_ms), 1) AS schema_aware_average_ms
FROM normalized;

Esta consulta de solo lectura se planifica y ejecuta en una tabla escrita vacía con Apache DataFusion 45.2.0. El resultado de la muestra determinista es sintético y se revisa por separado; valide tipos de campos, umbrales y definiciones comerciales con sus propios datos. Lea la metodología de prueba.

Resultado de la consulta

A plausible average can cover only half the jobs

The old field averages two fictional jobs at 475 ms. The version-aware query covers all four at 580 ms.

completed_jobsjobs_in_old_averageold_average_msjobs_with_durationunmeasured_jobsduration_coverage_pctschema_aware_average_ms
4247540100580

Salida de ejemplo sintético. Ejecute la consulta con su propio esquema de eventos y umbrales antes de utilizarla para decisiones operativas.

A plausible average can cover only half the jobs: gráfico estático de valores sintéticos de jobs_with_duration del resultado del ejemplo Catch event-field renames that hide rows from averages
SVG indexable de la salida del ejemplo determinista. Descárguelo para ver un artículo, un runbook o una revisión de diseño con atribución.

Reproducir el ejemplo

Descarga los datos de ejemplo

El paquete JSON incluye el contrato de evento escrito, reproducible filas de entrada, SQL exacto, resultado esperado, notas de revisión y versión del motor. El CSV contiene el resultado mostrado.

Cómo funciona el SQL

  1. 1The old query AVG(duration_ms) still returns 475 ms because SQL averages ignore nulls. Only two of four completed jobs contribute to it.
  2. 2The version-aware CASE maps release 1 to duration_ms and release 2 to latency_ms, giving 580 ms across all four fictional jobs.
  3. 3Keep duration coverage beside the average. An unknown version or a missing field can leave the average unchanged while coverage falls.

Casos límite que debes comprobar

  • Use a fixed UTC time window when comparing releases; this four-row lesson leaves all fictional rows en scope.
  • This example assumes one finish row per completed logical job. Deduplicate repeated finish events by job_id before using it for a real job count.
  • A field rename is comparable only if both versions time the same start and stop points en the same unit.
  • A missing or unknown schema_version maps to null and lowers coverage; do not silently coalesce it to an old field.
  • The four rows are fictional and do not describe a customer, a real incident, or measured product performance.

Panel recomendado

  • Table: completed jobs, duration coverage, and average by release
  • Trend: duration coverage percentage over deploy time

Guía de alerta

Alert on falling duration coverage only after establishing an expected minimum event count and documenting the version mapping.

Leer configuración de alerta

Configura los eventos que necesita esta consulta

Instrumentación y guías relacionadas.

Continuar el análisis

Ejecútalo con tus eventos

Crea una tabla, adapta los campos y guarda el resultado.

Comience gratis, envíe eventos estructurados y utilice el resultado de la consulta como un gráfico, un widget de panel compartido o una entrada de alerta.

Obtén una clave API