AI Agent Instrumentation Template

Template evidence path

AI Agent Observability Template: implementation to decision

Treat the prompt as an implementation brief. The useful artifact is not copied code alone, but a reviewed event contract that produces a trustworthy answer.

1
Select the boundary
Instrument the point where agent_run_started becomes final.
2
Create the contract
Start with agent_run_started, agent_tool_called, agent_run_completed and keep every field typed, bounded, and privacy-reviewed.
3
Run a fixture
Exercise known success, failure, retry, and empty-result cases before relying on aggregate results.
4
Answer the question
Which tools fail most often?

Template versus use case

This page is the implementation brief

Copy this template when the measurement goal is already clear. Use the matching use-case guide to review event boundaries, success definitions, and the decisions the resulting SQL should support.

Read AI Agent Quality and Evaluation

Template

Paste this into your coding agent

Replace YOUR_API_KEY, run the flow locally, then verify the generated events and dashboards.

ai-agent-observability

AI Agent Observability Template

text

Add Telemetry to this AI agent workflow.

Use /skill.md and this Telemetry API key: YOUR_API_KEY

Log:
1. agent_run_started with agent_name, route, user_id, team_id, model, provider, and input_category.
2. agent_tool_called with tool_name, status, latency_ms, retry_count, and error_type.
3. agent_run_completed with status, duration_ms, total_tokens, estimated_cost_usd, output_category, and accepted when available.
4. agent_run_failed with error_type, failed_step, retry_count, and duration_ms.

Create SQL queries for run volume, failure rate by tool, p95 duration, model cost by feature, and accepted-output rate.

Do not log raw prompts, completions, secrets, auth headers, or personal data unless explicitly approved.

Events to capture

agent_run_startedagent_tool_calledagent_run_completedagent_run_failedai_output_accepted

Verification checklist

What a complete instrumentation pass leaves behind

Events

Synthetic events reach the intended table with stable names and field types.

Queries

The first SQL queries return plausible rows with an explicit time window.

Views

A dashboard uses the real fields and includes enough context to explain a change.

Safety

Prompts, bodies, credentials, signatures, and private content were checked for redaction.

Event schema starting points

Review the row grain, emit boundary, required types, privacy classes, example payload, and validation checklist before adapting a query or snippet to production.

AI operations10 fields