Telemetry gives coding agents a feedback loop: capture agent telemetry, query real outcomes, and decide what to improve next—with inspectable SQL.
Real outcomes · Inspectable SQL · Human-reviewable decisions
Use telemetry.sh to instrument, deploy, find an opportunity, fix it, then verify the change helped.
The decision loop
The agent instruments before it optimizes. Then it deploys, queries the outcome, finds an opportunity, makes the fix, redeploys, and confirms the result actually improved.
Give the agent a goal and guardrails: improve checkout completion without increasing latency or errors.
It instruments the outcome, tests delivery, and deploys the event contract before changing behavior.
It queries the results, finds an opportunity, ships a fix, and confirms the outcome improved after redeploy.
Decision proof
In this loop, the agent compares checkout releases, finds a slow path, ships the fix, and checks that completion improved while p95 fell. The same pattern works for growth, reliability, AI quality, and cost.
Better decisions
A faster run is not better if acceptance falls. A cheaper model is not better if handoffs rise. Telemetry lets an agent reason across the outcome and the cost of producing it.
Track OpenAI costs with SQLCompare releases, prompt versions, models, routes, and workflows against the result users actually received.
Keep quality, success, latency, and cost in the same query so an optimization cannot hide its downside.
The event contract, SQL, result, and caveats stay visible to the human responsible for the decision.
Decisions worth measuring
Start where guessing is expensive. Each workflow becomes a repeatable evidence loop, not another dashboard to maintain.
Did this release improve successful runs without raising handoffs or cost?
Which model produces the most accepted results per dollar?
Did the new release reduce errors without increasing tail latency?
Which workload is stalling, and what changed before retries rose?
Which onboarding change moved activation and retained usage?
Which access pattern changed enough to deserve human review?
Give the agent stable outcome, release, duration, and workflow fields it can compare before and after a change. Keep secrets and raw payloads out so the evidence stays safe and legible.
telemetry.log("checkout_attempt_completed", {
attempt_id,
status: "completed",
duration_ms,
step,
release,
});Humans in the loop
“Telemetry makes it super easy to track, analyze, and visualize what’s going on in my business.”
“One of the easiest ways to go from dumping in your data to actually understanding it.”
“It’s straightforward, the SQL workbench is fast, and I don’t have to worry about the schema upfront.”
Teach the loop
Start with sample outcomes. Review the query. Connect real traffic when the decision is useful.