Context Engineering and Agents

From prompt to context

In a chat, you write the prompt and the model answers. An AI agent works in a loop: it reads, calls a tool, reads the result, calls another tool, and so on, sometimes for hundreds of steps. With every step, the text in front of the model changes, and most of it was not written by you. At that point the question is no longer only "how do I phrase this?" but "what should the model see right now?"

Context engineering is the practice of choosing and maintaining everything that goes into a model's context window at each step — instructions, tool definitions, retrieved documents, conversation history, tool results and memory — so that the model has what it needs and little else. Anthropic described it in September 2025 as "curating and maintaining the optimal set of tokens (information) during LLM inference" and as the natural next step after prompt engineering. Prompt engineering is part of it: the system prompt is still the most carefully written piece of the context.

Why more context is not better context

As of October 2026, leading models accept around a million tokens of context (see context window). It is tempting to paste everything. That rarely works well:

• Precision drops as the context grows. Anthropic calls this context rot and compares a model's attention to a limited budget: every extra token competes for it. The well-known "lost in the middle" finding points the same way: information buried in the middle of a long input is used less reliably (see How a Model Reads Your Prompt).
• Irrelevant text is not neutral. An old decision, a failed attempt or an outdated runbook in the context can steer the answer in the wrong direction.
• Everything is paid for, at every step. An agent resends its context with each call, so a large document pasted at step 1 is paid for again at step 40 (see tokens and cost).
• Everything is attack surface. Each document, web page or tool result in the context can carry instructions that someone planted (see Prompt Security and Responsible Use).

The goal is the smallest set of high-value information that lets the model do the next step well. For people this is familiar: a good handover note gives the next engineer what matters, not the whole shift's chat log.

Is anything wrong with the configuration of our access switch? Kestrel Energy moved the smart-meter gateways to VLAN 30 on port Gi0/3 yesterday.

Change CHG-2207 says: "Move the smart-meter gateways to port Gi0/3 and put that port in VLAN 30. Port Gi0/2 is then unused and must be shut down."

Below is the interface section of the access switch sw-sub-07 after the change (lines 41-53). Does the configuration match the change? Answer "yes" or "no", then list each mismatch with the exact line that shows it. Use only these lines; if something cannot be determined from them, say so.

<config>
41 interface GigabitEthernet0/2
42  description meter-gateways (old)
43  switchport mode access
44  switchport access vlan 30
45  no shutdown
46 !
47 interface GigabitEthernet0/3
48  description meter-gateways (new)
49  switchport mode access
50  switchport access vlan 20
51  no shutdown
52 !
53 end
</config>

Loading information when it is needed

There are two ways to get knowledge into the context. The first is to retrieve it up front: retrieval-augmented generation (RAG) searches a knowledge base before the model is called and adds the best-matching passages to the prompt. That works well when one search is enough.

Just-in-time retrieval is the second way: the context holds only lightweight references — file names, ticket numbers, the titles of runbooks — and the agent uses a tool to load the full content at the moment it decides it needs it. Coding agents work this way: they list folders, search with grep and open the few files that matter, instead of receiving the whole repository. The trade-off is speed: each lookup is an extra step, and the agent needs good tools and clear names to find the right thing.

The example compares the size of both approaches for three runbooks, using the rough rule of four characters per token (an estimate, not a real tokenizer).

# Just-in-time retrieval: the context holds short references; the full text
# is loaded only when the (scripted) model asks for it.
RUNBOOKS = {
    "runbooks/portal-503.md": "Portal returns 503\n"
    + "- Check the load balancer pool.\n" * 40,
    "runbooks/vpn-error-809.md": "VPN error 809\n"
    + "- Check that UDP 500 and 4500 are open.\n" * 60,
    "runbooks/meter-upload.md": "Meter readings not saved\n"
    + "- Check the upload queue.\n" * 50,
}


def estimate_tokens(text):
    return len(text) // 4  # rough rule of thumb, not a real tokenizer


up_front = "\n\n".join(RUNBOOKS[name] for name in sorted(RUNBOOKS))
index = "Runbooks you can open with read_file:\n" + "\n".join(sorted(RUNBOOKS))

# A scripted stand-in for the model: it reads the index and opens one file
requested = "runbooks/portal-503.md"
just_in_time = index + "\n\n" + RUNBOOKS[requested]

for label, context in [("all runbooks up front", up_front),
                       ("index only", index),
                       ("index + portal-503.md", just_in_time)]:
    print(f"{label:<22} ~{estimate_tokens(context):>5} tokens")

all runbooks up front  ~ 1260 tokens
index only             ~   27 tokens
index + portal-503.md  ~  353 tokens

With three short files the difference is small in absolute terms; with three hundred runbooks it decides whether the agent works at all. Good file names do real work here: the model chooses from the index, so portal-503.md is far more useful than notes-v2-final.md.

Tools are context too

Every tool definition is sent with every request, and the model picks tools by reading their names and descriptions (see Tool descriptions are prompts). Anthropic's guidance on context engineering and on writing tools for agents adds three points that matter once an agent has many tools:

• Keep the set small and without overlap. If an engineer could not say for certain which of two tools to use in a situation, the model cannot either.
• Return only what is useful. A tool that returns a 2,000-line log fills the context with noise; one that returns the 20 lines around the error, with a way to ask for more, keeps the agent focused.
• Make errors helpful. "Unknown service 'portal'; valid names are portal-login, portal-web" lets the model correct itself in one step.

Tools connected through the Model Context Protocol (MCP) bring their own descriptions into the context. The MCP specification warns that tool descriptions should be treated as untrusted unless they come from a trusted server, and that tools amount to running code: connect only servers you trust, and let the host ask before a tool acts.

Conversation history and compaction

Because the API is stateless, a conversation or agent run grows with every turn until it no longer fits, or long before that, becomes slow, expensive and unfocused. Simply dropping the oldest turns loses decisions that still matter. Context compaction is the technique of replacing older parts of a conversation or agent run with a short summary of what still matters — decisions, open questions, key facts — while keeping the most recent turns word for word, so that the work can continue in a much smaller context. Agent tools do this automatically when the context gets full; Anthropic's guidance also suggests starting a fresh context window for long tasks, with the state saved in files.

The example compacts a NOC conversation to a token budget. In a real system a model call writes the summary; here it is fixed text so the output stays the same.

SYSTEM = "You are the NOC assistant at Kestrel Energy. Be brief and factual."
HISTORY = [
    ("user", "fw-edge-02 dropped the VPN tunnel to 203.0.113.10 at 02:14."),
    ("assistant", "Check the IKE log on fw-edge-02 around 02:14 for rekey errors."),
    ("user", "Log shows 'peer not responding' from 02:13:58 to 02:14:40. " * 6),
    ("assistant", "The peer stopped answering; check its WAN link and uptime."),
    ("user", "Peer WAN link flapped twice. Provider confirms maintenance."),
    ("assistant", "Then the drop matches the provider's maintenance window."),
    ("user", "Draft a two-line note for the handover to Tomasz."),
]
BUDGET = 120  # tokens allowed for system prompt + history in this request


def tokens(text):
    return len(text) // 4  # rough estimate, not a real tokenizer


def size(system, turns):
    return tokens(system) + sum(tokens(text) for _, text in turns)


def compact(system, history, budget, keep_last=3):
    """Keep the newest turns verbatim; fold older ones into a summary."""
    if size(system, history) <= budget:
        return system, history
    # In a real system a model call writes this summary; here it is fixed.
    summary = ("Earlier in this conversation: the VPN tunnel from fw-edge-02 "
               "to 203.0.113.10 dropped at 02:14 because the peer stopped "
               "responding.")
    return system + "\n" + summary, history[-keep_last:]


for label, (system, turns) in [("before", (SYSTEM, HISTORY)),
                               ("after", compact(SYSTEM, HISTORY, BUDGET))]:
    print(f"{label:<6} {len(turns)} turns, ~{size(system, turns)} tokens")

system, turns = compact(SYSTEM, HISTORY, BUDGET)
print()
print("system   ", system.splitlines()[-1][:60] + " ...")
for role, text in turns:
    print(f"{role:<9} {text}")

before 7 turns, ~187 tokens
after  3 turns, ~89 tokens

system    Earlier in this conversation: the VPN tunnel from fw-edge-02 ...
user      Peer WAN link flapped twice. Provider confirms maintenance.
assistant Then the drop matches the provider's maintenance window.
user      Draft a two-line note for the handover to Tomasz.

The summary goes into the system part, so the conversation still starts with a user turn. The quality of the summary decides everything: a summary that leaves out "the provider confirmed maintenance" sends the agent back to investigating a solved question. Compaction is lossy by nature; when the details matter later, keep them in a file the agent can reopen.

Memory outside the context

A model remembers nothing between requests, and an agent's context is wiped or compacted during long work. Agents therefore keep notes. Agent memory is information an agent writes to storage outside its context window — notes files, a progress list, a database or the Git history — and reads back in a later step or session, so that it can continue work it no longer has in context. Anthropic's guidance for long tasks suggests exactly this: a progress file, a structured list of tests and their status, and Git commits as a log of what changed.

The example shows the idea: the first session ends by writing structured notes; the second starts with an empty context and builds its prompt from them.

import json
from pathlib import Path

NOTES = Path("progress.json")  # the agent's memory lives in a file, not in the model

# Session 1: the agent ends its work by writing structured notes.
NOTES.write_text(json.dumps({
    "task": "Update the portal's TLS certificate runbook",
    "done": ["Checked current steps against CHG-2207",
             "Rewrote step 3 (renewal)"],
    "next": ["Rewrite step 5 (rollback)", "Ask Ravi Kumar to review"],
    "decisions": ["Keep the old certificate for 24 hours after renewal"],
}, indent=2), encoding="utf-8")

# Session 2, with an empty context window: the notes are read back in.
notes = json.loads(NOTES.read_text(encoding="utf-8"))
lines = [f"You are continuing this task: {notes['task']}.",
         "Already done: " + "; ".join(notes["done"]) + ".",
         "Decisions made (do not reopen): " + "; ".join(notes["decisions"]) + ".",
         "Start with: " + notes["next"][0] + "."]
print("\n".join(lines))

You are continuing this task: Update the portal's TLS certificate runbook.
Already done: Checked current steps against CHG-2207; Rewrote step 3 (renewal).
Decisions made (do not reopen): Keep the old certificate for 24 hours after renewal.
Start with: Rewrite step 5 (rollback).

Structured notes (JSON for status, plain text for reasoning) are easier to read back reliably than a free-form diary. Memory is also a risk: anything written there is read as context later, so a wrong conclusion or a planted instruction can persist across sessions. Treat memory files like code — readable, reviewable and free of secrets and personal data.

Subagents: a fresh context for a side task

A subagent is a separate agent instance that a main agent starts for a focused side task, such as searching a codebase or reading a set of logs; it works in its own, clean context window and returns only a short result. The main agent's context receives a condensed summary, often 1,000 to 2,000 tokens according to Anthropic, instead of the tens of thousands the search itself consumed. This is one of the multi-agent patterns, chosen here for context reasons rather than for specialization.

Prompting a subagent is like briefing a colleague who has seen nothing: it does not share the main agent's history, so its task description must state the goal, the scope, where to look and what form the result should take ("list the files that call renew_certificate, with one line each on why").

Instructions files for coding agents

An instructions file is a text file, usually Markdown, kept in a repository or user folder, that an AI coding agent reads automatically at the start of its work as standing instructions: how to build and test the project, its conventions, and what not to touch. It is a standing instruction in file form, versioned with the code. As of October 2026 the main ones are:

A short example for Kestrel Energy's customer portal repository:

# Kestrel customer portal

## Build and test
- Python 3.12; install with: pip install -r requirements.txt
- Run all tests before you say a task is done: python -m pytest
- New API endpoints need a test in tests/api/.

## Conventions
- User-facing text in plain English, US spelling. No jargon in error messages.
- Times are stored in UTC and shown in Europe/Amsterdam.
- Test data comes from tests/fixtures/. Never use real customer records.

## Ask first
- Changes under migrations/ (they run against production data).
- Adding a new dependency.

## Where things are
- Login and sessions: portal/auth/
- Meter-reading upload: portal/meters/ (see docs/meter-upload.md)

Writing a good instructions file

• Write what the agent cannot discover. Build commands, test commands, conventions and "ask first" areas help; a description of what Python is does not.
• Keep it short. The whole file is loaded into every session and competes for attention with the actual task. Anthropic notes that longer files reduce how well the instructions are followed. Point to other documents instead of copying them in.
• Be specific and explain why. "Ask before changing migrations/ — they run against production data" is followed better, and applied more sensibly, than "NEVER touch migrations" (see Clear and Specific Instructions).
• Prune it. OpenAI's guidance for its newest models suggests going through each instruction and asking whether it is still needed: rules written to correct an older model can make a newer one worse.
• Never put secrets in it. The file is in Git and in every agent's context. Point to where credentials come from (environment variables, a secrets manager), never to their values.

A context checklist for agent work

• Does the system prompt or instructions file state the goal, the boundaries and how success is checked, at a level that guides without scripting every step?
• Is every tool needed, distinct from the others, and does it return only what is useful?
• Is reference material loaded when needed, rather than all up front?
• Is there a plan for long runs: compaction, notes files, or a fresh context with saved state?
• Is outside content (documents, web pages, tool results, cloned instructions files) treated as data that may contain planted instructions?
• Are the rules that must never be broken enforced outside the model, with a human in the loop for actions that are hard to undo?