Skip to content
Telemetry
AI quality

rag_retrieval_evaluated event schema

A versioned RAG evaluation event that separates retrieval scores, answer review, latency, and cost without storing source text, prompts, or generated answers.

Reviewed by the Telemetry product team on . We checked what each event represents, which service sends it, field types, sensitive data, the example, and validation steps. Who reviews this page

Test this schema

Question this event answers

Which retrieval and prompt versions produce relevant context and grounded accepted answers?

Fields
11
Required
10
Send and check an event

Record and query rag_retrieval_evaluated

Document what each rag_retrieval_evaluated row represents and which service sends it. Send test events to check the fields, then verify that the query answers your question.

  1. 1

    Outcome becomes final

    RAG evaluation pipeline emits only after the retrieval and answer evaluators finish or the candidate becomes ineligible.

  2. 2

    Choose the fields

    10 required fields preserve the declared grain: One evaluator result per eligible query, retrieval version, evaluator, and candidate run.

  3. 3

    Check the test event

    Check types, UTC time, alternate outcomes, idempotency, and every pseudonymous or review-classified field.

  4. 4

    Query the result

    Which retrieval and prompt versions produce relevant context and grounded accepted answers?

Grain

One evaluator result per eligible query, retrieval version, evaluator, and candidate run.

Owner

RAG evaluation pipeline

Emit when

After the retrieval and answer evaluators finish or the candidate becomes ineligible.

Field contract

Field types and data to exclude

Keep field names and types stable once production queries depend on them. Document optional fields and add them only when they answer a specific question.

FieldTypeRequiredPrivacyMeaning
timestamp_utctimestampyesnon-sensitiveUTC time when the operation finishes.
event_idstringyesnon-sensitiveStable unique identifier used for deduplication.
releasestringyesnon-sensitiveApplication or service version that emitted the event.
account_idstringyespseudonymousStable internal account identifier, never an email or name.
evaluation_idstringyespseudonymousStable identifier for the evaluation case.
retrieval_versionstringyesnon-sensitiveVersion of the index, retriever, and ranking contract.
evaluator_versionstringyesnon-sensitiveVersioned evaluator and rubric identifier.
retrieval_scorenumberyesnon-sensitiveNormalized retrieval relevance score.
groundedbooleanyesnon-sensitiveWhether the reviewed answer met the grounding rule.
acceptedbooleannonon-sensitiveReviewed downstream acceptance when available.
duration_msnumberyesnon-sensitiveEnd-to-end evaluated workflow duration.

Synthetic JSON event

{
  "timestamp_utc": "2026-07-29T10:12:43Z",
  "event_id": "evt_rag_eval_01",
  "account_id": "acct_8f31",
  "release": "2026.07.3",
  "evaluation_id": "eval_72ab",
  "retrieval_version": "hybrid_v4",
  "evaluator_version": "grounding_rubric_v2",
  "retrieval_score": 0.86,
  "grounded": true,
  "accepted": true,
  "duration_ms": 1280
}

Privacy review

Review identifiers before ingestion

This example uses synthetic identifiers. Pseudonymous values can still be personal data, and review fields can expose business or provider context. Apply your own consent, retention, access, residency, and deletion requirements.

  • account_id: pseudonymous
  • evaluation_id: pseudonymous

Validation checklist

Test the schema before building a dashboard

  • Send one known rag_retrieval_evaluated fixture after the documented outcome boundary.
  • Verify all 10 required fields arrive with the documented types.
  • Retry the same event identifier and confirm the chosen deduplication behavior.
  • Send a controlled failure or alternate outcome when the workflow supports one.
  • Run the related SQL over a fixed window and reconcile the result to the fixture.

Common mistakes

Record one result per row

  • Emitting rag_retrieval_evaluated before rag evaluation pipeline knows the final outcome.
  • Mixing different kinds of results in one table, which makes counts and rates ambiguous.
  • Replacing controlled categories with raw URLs, payloads, prompts, or error text.
  • Changing a field type in place after saved queries and dashboards depend on it.
  • Adding identifiers without a documented investigation, access, and retention need.

Use the contract

Query the event and set up monitoring

RAG quality SQLRAG evaluation guideEvent schema design guide

Related contracts

user_signed_up

Which signup cohorts activate and retain?

Review schemaproduct_milestone_completed

Which accounts complete the milestones you use to define activation?

Review schemaapi_request_completed

Where do errors and slow requests affect customers?

Review schema

Send a test event before production traffic

Create a free API key, send the synthetic event, and inspect the inferred table before connecting a live workflow.

Test this schema