Batching, Backpressure, and Graceful Shutdown

Batching can reduce request overhead, but it also changes failure impact and delivery latency. A process that queues events without limits can consume memory during an outage. A process that exits without accounting for pending work can silently lose its newest events.

Telemetry's Log API accepts either one JSON object or an array of JSON objects. The API capability does not require every producer to maintain an in-memory background queue.

Start with direct, awaited delivery

For low-volume workflows, send one compact outcome event and await the request with a bounded timeout. This is the easiest delivery path to reason about and test.

Batch when measurement shows that request overhead is material or when an existing worker already owns a durable collection buffer. Prefer small, time-bounded batches over a large queue that waits indefinitely for a size threshold.

await telemetry.log("api_request_completed", [
  {
    event_id: "evt_101",
    route_template: "/api/projects/:id",
    status: "success",
    latency_ms: 84,
  },
  {
    event_id: "evt_102",
    route_template: "/api/projects/:id",
    status: "failed",
    latency_ms: 912,
    error_type: "database_timeout",
  },
]);

Keep every object in a batch compatible with the same table schema. One invalid or incompatible item can complicate the outcome for the whole request.

Bound every queue

If the application buffers events, define:

1. Maximum queued event count and serialized bytes.
2. Maximum age before a partial batch is sent.
3. Maximum batch size.
4. Request timeout and retry budget.
5. Overflow behavior.
6. Shutdown behavior.

Overflow behavior must be explicit. Dropping the oldest low-priority diagnostic event, sampling a noisy success path, blocking a dedicated telemetry worker, and persisting to a durable outbox have different operational consequences. Do not let an unbounded queue decide by exhausting process memory.

Record queue depth, oldest-event age, batches sent, events accepted, events dropped, and permanent delivery failures through a safe diagnostic path. Avoid recursively sending the telemetry queue's own failure event through the same failing queue.

Separate backpressure from customer work

For most product analytics, telemetry delivery should not consume the entire latency budget of a customer request. Use a short bounded handoff or an application-owned worker. If the queue is full, apply the documented overflow policy instead of waiting indefinitely.

For billing or approved audit events, use a durable outbox and backpressure the outbox worker rather than losing the business event. The event delivery and idempotency guide explains stable event identity across attempts.

Handle shutdown deliberately

On a graceful shutdown signal:

1. Stop accepting new work.
2. Allow in-flight application work to reach a defined boundary.
3. Send or persist the remaining telemetry batch.
4. Enforce a shutdown deadline.
5. Record the count that could not be delivered.

Do not claim a client flush guarantee unless the specific SDK exposes and documents one. When a client sends each call immediately, awaiting all tracked calls is different from flushing an internal queue. Serverless and edge runtimes may freeze execution after a response, so use the runtime's supported background-work mechanism or deliver before returning when the event is required.

Verify overload behavior

Test a full batch, partial time-triggered batch, 429, 503, connection timeout, queue overflow, and process termination with pending events. Confirm that retry attempts preserve event IDs and that no failure path prints credentials or raw payloads.

Use the event-ingestion troubleshooting guide to verify responses and the telemetry volume guide to measure whether batching or sampling is actually necessary.