Token optimisation for an AI agent is a workflow-design problem. Keep the current task’s requirements, relevant evidence and verification contract available while avoiding unnecessary repeated context. A shorter request is not an improvement if it silently drops something the job needs.
Where agent context grows
An agent workflow can accumulate much more than the latest user message. Depending on the system, the working request may include stable instructions, conversation history, retrieved evidence, tool results, schemas, task state and expected-output rules.
Instead of treating all of that as one giant prompt, inspect the sources separately:
| Context source | Question | Safe optimisation test |
|---|---|---|
| Stable instructions | Are the same rules repeated unnecessarily? | Deduplicate only if the runtime still applies the rule reliably. |
| Task constraints | Which requirements define acceptance and scope? | Keep a requirement checklist and fail the optimisation if a binding item disappears. |
| Retrieved evidence | Does the current task need every retrieved passage? | Preserve source identity and the evidence needed for the answer. |
| Tool output | Does the next step need the full raw response? | Project the minimum useful fields while retaining identifiers, errors and evidence. |
| Conversation history | Which old decisions are still active? | Carry forward current state and unresolved items rather than replaying everything. |
| Output contract | Does the downstream consumer need every field? | Reduce the schema only when the consumer and verifier accept the smaller contract. |
A repeatable optimisation loop
- Freeze a representative task. Define what counts as an acceptable result before changing the prompt.
- Capture the request boundary. Record the instructions, task state, evidence, tool definitions and output requirements that actually reach the model where possible.
- Separate stable and task-specific context. This makes duplication easier to see.
- Remove repetition cautiously. Do not remove a rule merely because it looks similar to another rule.
- Run the same task again. Keep the model, tools and acceptance test comparable.
- Verify the result. A smaller request that fails the job is not an optimisation.
- Count retries and failures. Optimising one successful attempt while ignoring extra retries can give a misleading result.
Verification belongs inside the context budget
One of the easiest ways to make an agent prompt smaller is to remove instructions that look repetitive: acceptance criteria, stop conditions, required evidence or recovery rules. That can make the request shorter while making the workflow worse.
Useful verification checks include:
- required files, fields or citations exist;
- source references resolve to the intended evidence;
- the required schema or test suite passes;
- unknown values remain explicitly unknown;
- forbidden files or systems were not changed;
- the final artifact can be read back from the external system of record.
Tool output is often an overlooked source of context
Agents frequently receive large tool responses and then pass most of that response into later steps. A better pattern is to preserve the fields the next step actually needs: object IDs, status, errors, timestamps, hashes, evidence references and the relevant content.
This is not the same as blindly summarising everything. A summary that removes the identifier needed for a later update can create more work than it saves.
Conversation history should become state
Long-running agent workflows often repeat entire conversations because the system has no durable representation of the current state. A better architecture records active decisions, ownership, unresolved blockers and evidence outside the model turn, then retrieves the relevant subset for the current task.
The goal is not to make the model remember less for its own sake. The goal is to stop paying repeatedly for irrelevant history while preserving the information that makes the job correct.
Optimise the verified job, not the prompt
The useful denominator is the complete accepted workflow. Include retries, failed attempts, verification and human recovery where those are part of producing the result.
A prompt can become dramatically shorter while the overall workflow becomes more expensive because it causes more corrections. Conversely, a slightly larger request may be worthwhile if it prevents ambiguity or expensive recovery.
Practical takeaway
Treat an agent’s context as a versioned input contract, not a bag of text to minimise. Remove duplication and irrelevant material, preserve requirements and evidence, and then measure whether the accepted outcome survives.
The target is not minimum tokens. The target is minimum unnecessary tokens per verified useful result.