Measure AI agent security monitoring and check the results
To measure ai agent security monitoring, choose one workflow and its owner. Define the events, test them with known inputs, and write a query that answers a specific question.
- 1
Choose when to log
Inventory tools that read data, write data, execute code, or affect external systems.
- 2
Capture the outcome
Begin with agent_tool_authorization_decided, agent_tool_completed, agent_policy_changed and document the grain of each event.
- 3
Check the stored rows
Review policy-version changes, denial spikes, and approval backlogs before enabling alerts.
- 4
Make the decision
Which high-risk tools are denied or routed to approval most often?
Use case versus template
Choose what to measure
Use this guide to choose what to measure and when to log it. For a shorter setup prompt, open the matching template.
Related use cases
Check which events this guide covers
- Use this page for agent tool authorization, human approval, and policy-decision analysis.
- Use AI agent observability for latency, cost, retries, and end-to-end run reliability.
- Use security audit-log analytics for authentication and privileged actions elsewhere in your application.
Agent prompt
Paste this into your coding agent
Replace YOUR_API_KEY after signup, then ask the agent to run the product flow and verify the first events.
AI agent security monitoring setup prompt
Instrument AI agent security decisions with Telemetry.
Use /skill.md and this Telemetry API key: YOUR_API_KEY
Before every consequential tool call, log agent_tool_authorization_decided with event_id, run_id, tool_call_id, workflow, tool_name, action_class, risk_level, decision, reason_code, policy_version, release, environment, and timestamp_utc. Use only bounded values. Log the terminal tool outcome separately with the same run_id and tool_call_id.
Create dashboards for decisions by tool and risk level, denial rate, approval-required volume, policy-version changes, and allowed actions that later fail. Keep counts beside rates and require a minimum sample size before alerting.
Never log raw prompts, completions, credentials, authorization headers, tool arguments, tool results, retrieved documents, customer content, or free-form policy explanations. Do not treat these analytics as a replacement for least privilege, sandboxing, authorization enforcement, or an immutable security evidence store.Setup steps
- 1Inventory tools that read data, write data, execute code, or affect external systems.
- 2Define controlled action, risk, decision, and reason fields with the security owner.
- 3Record the authorization decision before each consequential tool call. Send a separate event with its final result afterward.
- 4Review policy-version changes, denial spikes, and approval backlogs before enabling alerts.
Events to capture
Questions you can answer
- Which high-risk tools are denied or routed to approval most often?
- Did a policy or agent release change the authorization mix?
- Which allowed actions later fail or require an incident review?
Example event schemas
Event schemas for this workflow
Check what each event records, when to send it, and which field types it needs. Review the example payload and privacy checklist before using it in production.
Use these queries in Telemetry
Learn about AI agent monitoring
Query agent events to compare tool use, model costs, and outcomes for each run.
Related SQL recipes
More SQL recipes
Run the query using this workflow's event fields and check the example result. Save the result to a dashboard or set up an alert.
Audit AI agent tool authorization decisions
Which agent tools are denied or routed to human approval most often?
Open recipeAudit privileged actions with SQL
Which privileged actions fail or require human review most often?
Open recipeAnalyze access-policy denials
Which access policies deny the most actors and actions?
Open recipeQuery nested AI tool-call events
Which AI tools and arguments are associated with the most failed calls?
Open recipeDetect repeating AI agent tool loops
Which agent runs appear stuck in a repetitive tool loop?
Open recipeReconstruct a correlated workflow timeline
What happened, in order, during the latest failed workflow?
Open recipeNext step
Create the API key your agent will use
The free plan is enough to run the prompt, send test events, and review the first dashboard.
Related pages
AI agent telemetry and observability
Record agent runs and tool calls in Telemetry tables. Query retries, latency, cost, failures, and whether users accepted the results.
Open pageSecurity and audit-log analytics
Record authentication failures, access changes, and privileged actions. Use these events to reconstruct an incident timeline, and exclude credentials and private payloads.
Open pageCodex instrumentation prompt
Ask Codex to add product events, verify that they arrive, and report which workflows still need logging.
Open page