All posts

Deep dive

Capturing LLM calls automatically

The Wytness Team
·4 min read

The most consequential calls your agents make are model calls — and in most stacks they're also the easiest to lose. Manual instrumentation depends on every engineer remembering to wrap every call site, forever. The Wytness SDK takes the opposite approach: initialise once, and the model calls your code already makes are recorded automatically.

One init, zero per-call code

When your process starts, the SDK wraps the Anthropic and OpenAI client libraries you already use. Your application code doesn't change — the same calls run as before, and each one now produces a signed audit event carrying the model used, the full token accounting, the duration, and the outcome. The events ride the same envelope-signed, PII-pseudonymised path as everything else on the tape, and the token counts are what power the Models & cost view.

Why "automatic" is the load-bearing word: manual wrapping fails in a specific, predictable way. It starts complete, then a new engineer adds a call site without the wrapper, a refactor moves a call out from under it, a library update changes a method — and each miss is silent, discovered months later as a gap in the record precisely where something interesting happened. Wrapping the client once at initialisation means new call sites are captured because they exist, not because someone remembered.

Streaming: where naive capture loses the plot

Streaming is where hand-rolled instrumentation quietly fails, because the interesting numbers — how many tokens this call actually consumed — only exist when the stream finishes. Wrap the request and you'll record that a stream started; the cost, the completion, and the outcome all live at the far end, after your instrumentation has already reported in. The SDK accounts for the stream's whole life: one event per call, recorded when the stream ends, with the final usage figures. Helper-style streams are captured when their context closes. And there is one documented edge we'd rather you read here than discover: breaking out of a raw stream loop without closing it skips capture — close your streams, which was already the provider's advice.

Cache tokens count too

Modern LLM bills are increasingly shaped by prompt caching — input served from the vendor's cache at a discount, and written into it at a premium. Captured events carry that accounting, so the trail reflects what the call actually cost, not a simplification of it. This matters beyond the bill: an audit that asks "what context did the agent have?" gets a truthful answer only if cached context is counted as the input it genuinely was.

Capture is a contract, not a vibe

The uncomfortable truth about automatic capture is that "automatic" tempts vendors into vagueness — we capture your LLM calls, unspecified. We treat the capture scope as a stated contract: the docs name which provider surfaces are wrapped, and other providers are not captured — said in exactly those words. Provider SDKs also move fast — new endpoints, new streaming shapes, whole new API styles — so the scope is a maintained contract, tested against the real provider packages, rather than a claim from the month the feature shipped.

A model call outside the recorded scope is absent from your trail, not silently half-recorded, and absence you know about is something you can fix. Vagueness you believed is an audit finding waiting to happen. If your stack leans on a provider we don't wrap, the honest path is the one the docs describe — not a quiet assumption that "automatic" covered it.

Why this matters beyond cost

The model call is where the agent's decision actually happened — which model, with how much context, at what expense, succeeding or failing. When an auditor asks "what was the AI doing in this workflow?", the LLM events are the answer's backbone: the reasoning step sits recorded between the tool calls it caused, and the run reads as a whole. Getting model calls recorded by default, rather than by discipline, is the difference between a trail you have and a trail you meant to have.

Setup is part of the standard quickstart — auto-capture is on when you init. Questions about your provider mix? Ask us.

ShareXLinkedIn

Keep reading

We set no cookies. Sign-in and preferences use essential first-party browser storage only — no tracking, advertising, or third-party analytics. Privacy Policy

One init, zero per-call code
TABLE OF CONTENTS