Connect OpenTelemetry GenAI Traces to Outcome Events

OpenTelemetry traces and Telemetry structured events solve different parts of an AI-agent investigation. Keep detailed model, agent, and tool spans in an OTLP-compatible observability backend. Send a smaller terminal outcome event to Telemetry when you want inspectable SQL over success, cost, latency, release, account, handoff, or reviewed product value.

Telemetry does not expose an OTLP endpoint and is not an OpenTelemetry trace backend. The connection is an application-owned correlation identifier, not a second export of every span.

Use each system for its intended job

An OpenTelemetry trace is well suited to answering:

• which span or tool dominated one slow run;
• how control moved between agent, model, retrieval, and tool operations;
• which exception or dependency explains an individual failure;
• what detailed attributes were present inside an approved trace-retention boundary.

A compact outcome event is well suited to answering:

• which release has the highest terminal task-success rate;
• which workflow costs the most per accepted result;
• whether tool retries or human handoffs increased this week;
• which customer tier was affected by a bounded error category;
• whether an evaluation score changed after a prompt or model version.

Do not copy all span attributes into the event. Decide which aggregate questions need a durable column and allowlist those fields.

Map semantics deliberately

The OpenTelemetry generative AI semantic conventions define evolving attributes and span conventions for model, agent, and tool operations. Pin the semantic-convention and instrumentation-library versions used by your service, then review the mapping during upgrades.

The semantic conventions can change while they mature. Do not assume an attribute observed in one SDK or framework has the same stability or availability everywhere. Treat the exact mapping as versioned application code.

Emit one terminal outcome beside the trace

This JavaScript wrapper reads the current trace ID and records an application outcome after the agent run. The trace continues to be exported by the configured OpenTelemetry SDK; only the selected fields go to Telemetry.

import { trace } from "@opentelemetry/api";
import telemetry from "telemetry-sh";

telemetry.init(process.env.TELEMETRY_API_KEY);

export async function runSupportAgent({
  agent,
  input,
  operationId,
  accountId,
  release,
}) {
  const startedAt = Date.now();
  let status = "success";
  let errorType;
  let result;

  try {
    result = await agent.run(input);
    return result;
  } catch (error) {
    status = "failed";
    errorType = classifyAgentError(error);
    throw error;
  } finally {
    const activeSpan = trace.getActiveSpan();
    const traceId = activeSpan?.spanContext().traceId;

    await telemetry.log("agent_run_completed", {
      operation_id: operationId,
      trace_id: traceId,
      workflow: "support_resolution",
      account_id: accountId,
      status,
      error_type: errorType,
      duration_ms: Date.now() - startedAt,
      human_handoff: result?.handoffRequired ?? false,
      tool_call_count: result?.toolCallCount ?? 0,
      release,
    });
  }
}

Make event delivery failure non-fatal unless the business workflow explicitly requires otherwise. Use a short timeout, bounded retries, graceful shutdown, and the delivery guidance in batching, backpressure, and shutdown.

Add model usage only once

If the OpenTelemetry instrumentation already observes provider requests, that does not automatically put request usage in Telemetry. Decide whether an aggregate SQL question requires a separate llm_request_completed event.

When it does, emit one event per billable request with:

• operation_id, run_id, and an optional approved trace_id;
• provider, requested_model, response_model, and service_tier;
• input, cached-input, output, and other separately defined token categories;
• estimated_cost_usd and a versioned pricing source;
• latency_ms, status, attempt, feature, and release.

Do not derive cost from span duration. Use provider-reported usage and a reviewed rate table, then reconcile estimates to the provider invoice. The OpenAI request-cost guide shows that pattern.

Protect the data boundary

Generative AI telemetry can contain unusually sensitive data. Do not send these fields to Telemetry by default:

• prompt or completion content;
• system instructions or chain-of-thought content;
• retrieved document text or embeddings;
• tool arguments, tool responses, shell output, or file contents;
• authorization headers, cookies, API keys, database credentials, or connection strings;
• unrestricted exception messages or OpenTelemetry baggage;
• personal data that has not passed the product’s collection and retention review.

Prefer categories such as input_category, output_category, tool_name, error_type, policy_result, and review_outcome. Apply the allowlist before calling either exporter; removing a field from a dashboard does not remove it from stored data.

Query outcomes and preserve the trace link

Use SQL for aggregate comparison while retaining the correlation pointer responders need:

SELECT
  release,
  workflow,
  COUNT(*) AS completed_runs,
  SUM(CASE WHEN status = 'success' THEN 1 ELSE 0 END) AS successful_runs,
  SUM(CASE WHEN human_handoff THEN 1 ELSE 0 END) AS handoffs,
  ROUND(AVG(duration_ms), 0) AS average_duration_ms,
  MAX(trace_id) AS example_trace_id
FROM agent_run_completed
WHERE timestamp_utc >= now() - INTERVAL '7 days'
GROUP BY release, workflow
ORDER BY completed_runs DESC;

MAX(trace_id) is only an example pointer from the group, not a representative trace. For an investigation, open the underlying rows, select the relevant run, and paste its trace_id into the trace backend. If that backend supports a stable URL template, build the link in an access-controlled internal tool rather than sending vendor credentials or private hostnames as event fields.

Validate the connection

Before production rollout:

1. Generate one successful run, one tool failure, one recovered retry, and one terminal failure.
2. Confirm the trace backend contains the expected span tree.
3. Confirm Telemetry contains one terminal event per run and the intended request or tool events.
4. Compare the trace and event correlation identifiers.
5. Verify prompts, completions, arguments, credentials, and private content are absent.
6. Test behavior when either exporter is slow or unavailable.
7. Document owners, retention, semantic-convention version, sampling, and the investigation handoff.

For general architecture and SDK examples, read Telemetry with OpenTelemetry. For agent-specific outcome design, continue to evaluate AI agents with SQL and the AI agent monitoring product boundary.