Skip to content
Telemetry
Structured events SQL recipe

Query nested AI tool-call events

Filter dotted nested fields and rank failing tools without flattening the original event payload.

Intermediateagent_eventsReviewed 2026-07-27Tested with Apache DataFusion 45.2.0

Reviewed by the Telemetry product team on . We checked the SQL syntax, required event fields, sample results, and limits on using the query. Who reviews this page

Question answered

Which AI tools and arguments are associated with the most failed calls?

Telemetry exposes nested tool fields as dotted columns. Query those columns directly without flattening the JSON before ingestion.

Event schema

Fields the query expects

FieldTypeWhy it exists
timestamp_utcTimestampWhen the tool call completed.
event_nameUtf8agent_tool_called for these events.
data.tool_nameUtf8Nested tool identifier exposed as a dotted column.
data.statusUtf8Nested success or failed status.
data.args.operationUtf8Safe, categorized operation rather than raw arguments.
data.latency_msFloat64Nested tool-call latency.
DataFusion SQL

Copy the query

sql
SELECT
  "data.tool_name" AS tool_name,
  "data.args.operation" AS operation,
  COUNT(*) AS calls,
  SUM(CASE
    WHEN "data.status" = 'failed' THEN 1 ELSE 0
  END) AS failures,
  100.0 * SUM(CASE
    WHEN "data.status" = 'failed' THEN 1 ELSE 0
  END) / NULLIF(COUNT(*), 0) AS failure_rate_pct,
  approx_percentile_cont("data.latency_ms", 0.95) AS p95_latency_ms
FROM agent_events
WHERE event_name = 'agent_tool_called'
  AND timestamp_utc >= now() - INTERVAL '7 days'
GROUP BY "data.tool_name", "data.args.operation"
HAVING COUNT(*) >= 20
ORDER BY failure_rate_pct DESC, calls DESC;

This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. We review the synthetic sample output separately. Check field types, thresholds, and counting rules against your own data. Read the testing methodology.

Query result

Tool-call failure rate

CRM lookup is both the least reliable tool operation and the slowest at p95.

tool_nameoperationcallsfailuresfailure_rate_pctp95_latency_ms
crm_lookupsearch_contact842617.242,180
order_apifetch_order2,210441.99940
knowledge_searchsemantic_search4,510190.42720

Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.

Tool-call failure rate: static chart of synthetic failure_rate_pct values from the Query nested AI tool-call events example result
Download this SVG chart of the sample results for an article, runbook, or design review. Please credit Telemetry.

Reproduce the example

Download the sample data

The JSON bundle includes the event schema with field types, illustrative input rows, exact SQL, expected output, review notes, and engine version. The CSV contains the displayed result.

How the SQL works

  1. 1Dotted field names are double-quoted because the dots are part of the inferred column name.
  2. 2The query groups by both tool and categorized operation, preserving enough context to distinguish different behaviors of one tool.
  3. 3Raw tool arguments should not be logged by default. Store safe categories, identifiers, and outcome fields instead.

Edge cases to check

  • Keep nested field types stable as schemas evolve.
  • Different tools may have different success semantics; normalize status values at instrumentation time.
  • High-cardinality raw arguments can create privacy, cost, and usability problems.

Recommended dashboard

  • Bar chart: failure_rate_pct by tool_name
  • Line chart: p95_latency_ms over time split by tool
  • Table: recent failures with error_type and agent workflow

Alert guidance

Alert when a production tool's failure rate crosses its threshold at meaningful volume, and route the alert to the owning integration.

Read alert setup

Set up the events this query needs

Related instrumentation and guides

Define the source data

Event schemas for this analysis

Continue the analysis

Run it on your events

Create a table, adapt the fields, and save the result

Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.

Get an API key