A canary can look trivial: ask an AI worker to return one line, then check that it is alive. Yet a short visible answer says little about the request size. A wrapper may attach policy, task context, tool descriptions, conversation history, schemas, retry state and routing instructions before the request reaches the model.
View the full infographic → · View this article
The useful question is not how many words the canary returned, but what exact input the provider counted after orchestration assembled the request.
The one-line output is only the visible edge
| Input component | Why it may be present | How to check |
|---|---|---|
| System/developer instructions | Persistent behaviour rules | Inspect serialized request or provider usage |
| Task prompt | The apparent canary | Record exact submitted text |
| Conversation/history | Prior turns, summaries or retries | Record messages and tokens by role |
| Tool definitions | Available names and schemas | Compare with unused tools excluded |
| Format constraints | JSON mode, response format or grammar | Record exact configuration |
| Wrapper and retry context | Workflow state, failed output or repair message | Instrument the final request boundary and count every attempt |
Provider-reported input may include more than visible text. Local string tokenization may not reproduce provider accounting.
How a small canary reaches a large context
A one-sentence health check launched through a production agent wrapper may load a system prompt, tool catalog and prior transcript. A retry may append a validator message and the previous response. The user sees one line; the model sees the assembled messages.
A bounded OpenCode auth canary on 26 September 2026 requested GPT-6 Luna (openai/gpt-6-luna). Provider-reported usage was 13,289 input tokens, zero cached input, and 10 output tokens. The response was exactly KD_OPENCODE_AUTH_OK and the CLI exited successfully. This is one invocation, not a representative task benchmark. It does not establish how many tokens each wrapper component contributed and must not be generalized to other agents, providers, models or runs.
For any canary claim, preserve run-specific provider usage and request-boundary evidence. Keep observation separate from causal explanation unless each request component has been instrumented.
Measure at the request boundary
- Record run ID, timestamp, provider, endpoint and exact model.
- Capture final serialized messages and tool/response-format definitions, with sensitive data removed.
- Record provider-reported input, cached-input and output counts where available.
- Record attempt number, retry reason, finish reason and wrapper outcome.
- Track the actual health signal separately from progress and acceptance.
Separate input, cached input and output
Use provider categories as returned. Do not double-count cached input if it is already included in input; document definitions. Include retries. Mark unavailable fields as unavailable rather than estimating from characters.
Find the source of overhead
| Variant | Change | Question |
|---|---|---|
| A — baseline | Current canary path | What does the full wrapper send? |
| B — fresh context | Remove unrelated history | How much does history contribute? |
| C — minimal tools | Exclude unused tools | Are schemas material? |
| D — dedicated probe | Small purpose-built prompt and contract | Can liveness be tested without the full agent? |
Repeat comparisons when the path is nondeterministic; hold model, endpoint, output cap and settings fixed. Report sample size and do not generalize from one run.
A heartbeat is not model progress
A wrapper heartbeat shows process or supervisor liveness, not that a model call is advancing or a task passed verification. Track process liveness, model activity, named task progress and acceptance separately. A heartbeat should not indefinitely reset a task-progress deadline.
Reduce waste without weakening the canary
- Use a dedicated short prompt for liveness-only checks.
- Do not carry unrelated history into a fresh probe.
- Expose only necessary tools.
- Set a small output limit and stable response contract.
- Use deterministic process or HTTP checks when suitable.
- Log per-attempt usage and bounded retry reasons.
- Keep a separate progress deadline.
Removing context is not safe if the canary is specifically meant to test that context-dependent path.
Limitations
Token counts depend on provider, model, schemas, caching and message framing. A production-agent canary answers a different question from a dedicated probe. Compare like with like; token volume alone does not establish quality or task success.
FAQ
Does a one-line response mean the request used few tokens?
No. Output and input length are different. Measure the final request and provider usage.
Is 10,000 input tokens normal for every canary?
No universal figure is established here. Validate any run-specific claim against provider records.
Can I estimate tokens from characters?
Only roughly; estimates may miss model tokenization, framing and non-text inputs.
Should every heartbeat call an LLM?
No. Use a deterministic probe for process responsiveness unless an LLM-specific path is what you need to test.
Further reading
- Experiments
- Methodology
- AI Costs
- Local AI
- AI Context Cost Calculator — compare input-only costs using your rates.
- AI Agent Operating Rules Builder — assemble concise canary and recovery rules.
Token Optimisation Diagnostic
Teams operating a recurring AI workflow can enquire about a bounded diagnostic covering a representative workflow, token accounting boundaries and prompt/context structure. Scope and data handling are agreed before sample processing; no savings, semantic equivalence or production outcome is guaranteed. Contact Token Optimisation with a high-level description only. Do not email prompts, secrets, API keys, personal data, customer records or production exports.
The 10k+ figure is not asserted as a universal measured result. Verify against run-specific provider usage and request-boundary evidence.