Skip to content
For teams governing AI agents that can call tools or change external state

AI agent security monitoring

Monitor tool authorization, policy denials, approval queues, risky action classes, and release changes without storing prompts, credentials, or tool payloads.

Reviewed by the Telemetry product team on . We checked the event fields, suggested queries, and data to exclude. Who reviews this page

Why this works
  • Keep the agent's requested action separate from the policy engine's decision.
  • Measure denied, allowed, and approval-required outcomes by tool and risk class.
  • Build an investigation timeline from authorization decisions and final tool outcomes. Use a fixed set of decision values.
How to test this use case

Measure AI agent security monitoring and check the results

To measure ai agent security monitoring, choose one workflow and its owner. Define the events, test them with known inputs, and write a query that answers a specific question.

  1. 1

    Choose when to log

    Inventory tools that read data, write data, execute code, or affect external systems.

  2. 2

    Capture the outcome

    Begin with agent_tool_authorization_decided, agent_tool_completed, agent_policy_changed and document the grain of each event.

  3. 3

    Check the stored rows

    Review policy-version changes, denial spikes, and approval backlogs before enabling alerts.

  4. 4

    Make the decision

    Which high-risk tools are denied or routed to approval most often?

Use case versus template

Choose what to measure

Use this guide to choose what to measure and when to log it. For a shorter setup prompt, open the matching template.

Open AI agent security audit template

Related use cases

Check which events this guide covers

  • Use this page for agent tool authorization, human approval, and policy-decision analysis.
  • Use AI agent observability for latency, cost, retries, and end-to-end run reliability.
  • Use security audit-log analytics for authentication and privileged actions elsewhere in your application.

Agent prompt

Paste this into your coding agent

Replace YOUR_API_KEY after signup, then ask the agent to run the product flow and verify the first events.

agent prompt

AI agent security monitoring setup prompt

text
Instrument AI agent security decisions with Telemetry.

Use /skill.md and this Telemetry API key: YOUR_API_KEY

Before every consequential tool call, log agent_tool_authorization_decided with event_id, run_id, tool_call_id, workflow, tool_name, action_class, risk_level, decision, reason_code, policy_version, release, environment, and timestamp_utc. Use only bounded values. Log the terminal tool outcome separately with the same run_id and tool_call_id.

Create dashboards for decisions by tool and risk level, denial rate, approval-required volume, policy-version changes, and allowed actions that later fail. Keep counts beside rates and require a minimum sample size before alerting.

Never log raw prompts, completions, credentials, authorization headers, tool arguments, tool results, retrieved documents, customer content, or free-form policy explanations. Do not treat these analytics as a replacement for least privilege, sandboxing, authorization enforcement, or an immutable security evidence store.

Setup steps

  1. 1Inventory tools that read data, write data, execute code, or affect external systems.
  2. 2Define controlled action, risk, decision, and reason fields with the security owner.
  3. 3Record the authorization decision before each consequential tool call. Send a separate event with its final result afterward.
  4. 4Review policy-version changes, denial spikes, and approval backlogs before enabling alerts.

Events to capture

agent_tool_authorization_decidedagent_tool_completedagent_policy_changedagent_approval_resolvedagent_run_completed

Questions you can answer

  • Which high-risk tools are denied or routed to approval most often?
  • Did a policy or agent release change the authorization mix?
  • Which allowed actions later fail or require an incident review?

Example event schemas

Check what each event records, when to send it, and which field types it needs. Review the example payload and privacy checklist before using it in production.

Use these queries in Telemetry

Learn about AI agent monitoring

Query agent events to compare tool use, model costs, and outcomes for each run.

Related SQL recipes

More SQL recipes

Run the query using this workflow's event fields and check the example result. Save the result to a dashboard or set up an alert.

Browse all recipes

Next step

Create the API key your agent will use

The free plan is enough to run the prompt, send test events, and review the first dashboard.

Related pages