Context Engineering for AI Agents: Giving Models the Right Information at the Right Time

When an AI agent makes a poor decision, the instinct is to blame the model or rewrite the prompt. More often the problem is context: the agent did not have a key fact, had too many irrelevant documents, carried a stale assumption from twenty steps earlier, or misread which tool to use. Context engineering is the discipline of getting this right.

Anthropic’s engineering team described context engineering in September 2025 as the natural progression of prompt engineering: managing the whole set of tokens a model sees at inference time, with the goal of finding the smallest possible set of high-signal tokens that make the desired outcome likely. This guide turns that idea into practical design steps.

What counts as context

Why agents need it more than chatbots

A chatbot answers one question with one context. An agent works over many steps, and each step adds tool results and reasoning to its context. Three problems follow:

• Accumulation. After 30 tool calls, most of the context is old tool output, much of it irrelevant to the current step.
• Drift. Early assumptions persist even when later evidence contradicts them.
• Cascading errors. A misunderstood fact at step three leads to wrong actions at steps four to ten.

Anthropic describes the underlying constraint as an attention budget: every token draws on the model’s finite ability to focus, and recall tends to degrade as context grows — what it calls context rot.

Techniques that work

Right-altitude instructions

Instructions should be specific enough to guide behaviour but not hard-code brittle logic. State the goal, constraints, priorities and when to stop or escalate. Replace long lists of rules with a few principles and a small set of diverse examples.

Explicit over implicit

Agents cannot infer unwritten business rules. If refunds above a limit need approval, say so — and enforce it in the tool as well. If “urgent” means a response within four hours, define it.

Just-in-time retrieval

Rather than loading every possibly relevant document upfront, give the agent lightweight references — record IDs, file paths, search tools — and let it fetch what it needs when it needs it. Combine with a small amount of preloaded context for information almost every task needs.

Compaction

When history grows long, summarise it: keep decisions, open questions and key facts; drop raw tool outputs that have served their purpose. Clearing old tool results is often the easiest win.

Structured notes outside the context

For long tasks, let the agent write notes to a file or store — a plan, progress, findings — and read them back when needed. This preserves important information without keeping everything in the active context.

Isolated sub-agents

Delegate focused subtasks to sub-agents with clean contexts. They explore, then return a short summary to the main agent. This keeps the main context focused, at the cost of more model calls.

Tool design

Tools are part of context. Keep them few, distinct and self-explanatory. Return only the fields needed, in compact form. A search tool that returns 50 full documents wastes context; one that returns ten ranked snippets with IDs lets the agent fetch details selectively.

A worked token budget

Consider a support investigation agent with these components at a typical step:

The point is not the exact numbers but the method: measure each component, change one thing at a time and confirm on an evaluation set that task success holds or improves. In many systems, success rises as noise falls.

Worked example: diagnosing a failing agent

A hypothetical agent that prepares supplier onboarding packs fails on about a third of evaluation tasks. The team logs full context for 30 failed runs and classifies the causes:

After the first three fixes, re-running the evaluation set shows which changes helped. Fixing context first is usually cheaper than switching to a larger model, and the improvements carry over when you do upgrade.

Tool descriptions are context too

A vague tool description forces the model to guess. Compare:

A per-step context checklist

• Does the agent know the current goal and what “done” means?
• Does it have the facts this step needs, and nothing it clearly does not?
• Are tool results from earlier steps still relevant, or should they be summarised or cleared?
• Is any retrieved content older than the systems of record allow?
• Is untrusted content clearly separated from instructions?
• Is the total within the budget set for this step?

Measuring context quality

• Task success rate on a fixed evaluation set — the primary measure.
• Retrieval precision and recall: did the agent see the facts it needed, and how much irrelevant material?
• Tokens per step and per task, by component.
• Steps to completion: better context often means fewer wasted actions.
• Failure analysis: for each failed task, record whether the cause was missing, wrong, stale or excessive context, or a genuine reasoning error.

Context is an attack surface

Anything that enters context can influence the agent. Documents, web pages, emails and tool results may contain instructions written by an attacker. Mitigations:

• Mark retrieved and tool content clearly as data, separate from instructions.
• Never let retrieved content add tools, change permissions or alter goals.
• Enforce permissions in tools, not only in instructions.
• Filter retrieval by the user’s access rights so the agent never sees data the user should not.
• Validate memory before writing it, and record its source.

These map to OWASP’s LLM risks, notably prompt injection (LLM01), sensitive information disclosure (LLM02) and vector and embedding weaknesses (LLM08).

Common pitfalls

• Adding every document “just in case”.
• Keeping every tool result for the whole task.
• Writing instructions that contradict the examples.
• Giving the agent many similar tools with overlapping descriptions.
• Storing memories without scope, source or expiry.
• Changing several context components at once and not knowing which change helped.

Frequently asked questions

Is context engineering just prompt engineering with a new name?

It includes prompt writing, but covers much more: retrieval, tools, history management, memory and the overall token budget across many steps.

Do larger context windows remove the need for it?

No. Larger windows allow more information, but attention is still limited and cost and latency rise with every token. Curation remains important.

How do we start improving context for an existing agent?

Log the full context for a sample of failed tasks, classify the causes, then fix the largest category first, measuring against an evaluation set.

Should every agent use sub-agents?

No. Sub-agents help with broad exploratory tasks but add cost and complexity. Use them when a single context becomes the bottleneck.