Context Engineering for AI Agents: The Complete Developer Guide

Why Context Engineering Is Replacing Prompt Engineering

For the first two or three years of applied AI development, prompt engineering was the central skill. Write a better system prompt. Tweak the instruction phrasing. Get more consistent outputs. That was the job.

It still matters. But something more important has emerged as agents have gotten more capable and tasks have gotten longer: context engineering.

The difference is scope. Prompt engineering asks: "How do I write better instructions?" Context engineering asks: "What is the complete state of the model's context window at each step of this task, and how do I make it as useful as possible?"

As Anthropic's Applied AI team put it in their definitive guide on the subject: "Building with language models is becoming less about finding the right words and phrases for your prompts, and more about answering the broader question of 'what configuration of context is most likely to generate our model's desired behavior?'"

If you're building anything that runs more than a few turns -- a coding agent, a research assistant, an automated workflow -- context engineering is the lever that separates agents that work from agents that drift, hallucinate, or stall.

The Attention Budget: Why Context Is a Finite Resource

Modern language models support context windows of 128K, 200K, even 1M tokens. This sounds like it should make context scarcity obsolete. It doesn't.

The reason is the transformer architecture. Every token in the context attends to every other token -- an O(n2) relationship. As the context window grows, the model's attention budget gets stretched across more and more pairwise relationships. Research from Chroma has documented the result: context rot.

Context rot is the empirically observed degradation in a model's ability to recall information as the context window fills. The model doesn't fail catastrophically -- it continues to operate -- but its precision for specific retrieval and long-range reasoning decreases. A model that can perfectly recall a fact from a 10-page context may miss it from a 200-page one.

This means treating the context window as a simple container -- "just add more" -- is the wrong mental model. The right mental model is a budget. Every token you add depletes the attention budget by some amount. Context engineering is the discipline of spending that budget well.

The Four Components of Context to Manage

Everything that enters the model's context window when it processes a request falls into one of these categories:

• System prompt -- Your instructions, agent role, behavioral guidelines
• Tools -- Function definitions the agent can call, plus their results
• Message history -- The conversation so far, including all prior tool calls and outputs
• Retrieved data -- Documents, database results, file contents loaded into context

Each of these requires different engineering decisions. Let's work through them.

System Prompt: The Right Altitude

The most common system prompt failure is getting the altitude wrong. Anthropic describes two failure modes:

• Too specific: Hardcoded if-then logic for every edge case. Brittle, hard to maintain, and doesn't generalize. The prompt becomes a 3,000-token specification that the model follows rigidly until it encounters something you didn't specify.
• Too vague: High-level guidance that assumes shared context the model doesn't have. "Be helpful and accurate." The model interprets this differently on every run.

The right altitude is specific enough to guide behavior and flexible enough to let the model use judgment in novel situations. You're giving the model strong heuristics, not a flowchart.

Practical format: organize prompts into clearly delimited sections. Anthropic recommends using XML tags like <background_information>, <instructions>, and <output_description>. The exact format matters less than the principle: separate what the agent is from what it should do from how it should format output.

Start minimal. Test a short prompt with the most capable model available. Find where it fails. Add targeted guidance based on actual failure modes -- not anticipated ones. Every sentence you add to the system prompt is a tax on the attention budget.

Tools: Minimal Viable Set

Tools create two context costs: the function definition (always in context) and the tool result (added each time the tool is called). Both should be treated as precious.

The most common tool design mistake is giving agents too many tools with overlapping functionality. If a human engineer can't definitively say which tool to use in a specific situation, the agent can't either -- and it will waste tokens exploring both options before committing.

Design tools like well-scoped functions in a codebase:

• One clear purpose per tool
• No overlap in what tools cover
• Descriptive parameter names that guide correct usage
• Errors that are informative rather than cryptic
• Results that are token-efficient -- return structured summaries, not raw dumps

The token-efficiency point is underrated. A tool that returns a full database record when you only needed three fields is burning context budget on noise. Design tools to return what the agent needs, not what was convenient to extract.

Message History: Compaction

Message history is the fastest-growing component of context in multi-turn agents. Every tool call adds the function invocation and its result. Every model response adds its output. Over a long task, the history becomes the largest context component -- and most of it is outdated.

The solution is compaction: periodically summarize the conversation history and reinitiate with a compressed version.

Claude Code does this automatically. When the context window approaches its limit, it passes the message history to the model to summarize. The model preserves what matters -- architectural decisions, unresolved bugs, key implementation choices, files that were changed -- and discards what doesn't: redundant tool outputs, intermediate reasoning that led to dead ends, repeated error messages.

The compressed summary, plus a handful of recently accessed files, becomes the new context. The agent continues with full access to what was important from the previous context, without carrying all the noise.

For engineers implementing their own compaction:

1. Start by maximizing recall -- make sure your compaction prompt captures everything that could possibly matter
2. Iterate to improve precision -- remove superfluous content that never actually influenced later behavior
3. The safest first step: clear tool results from history once the action is complete. The raw output of a tool call is rarely needed again; only the outcome matters

# Example: Simple tool result clearing in a LangChain-style agent
def compact_history(messages: list[dict]) -> list[dict]:
    compacted = []
    for msg in messages:
        if msg["role"] == "tool":
            # Replace verbose tool output with a brief acknowledgment
            compacted.append({
                "role": "tool",
                "content": f"[Tool {msg.get('name', 'call')} completed -- result summarized]",
                "tool_call_id": msg.get("tool_call_id")
            })
        else:
            compacted.append(msg)
    return compacted

For deeper compaction, pass history to the model for summarization

def summarize_history(client, messages: list[dict], model: str = "claude-haiku-4-5") -> str: summary_prompt = """Summarize the conversation history below. Preserve:

• All architectural decisions made
• Unresolved errors or blockers
• Current state of files or data being worked on
• What was attempted and why it did/didn't work

Discard: verbose tool outputs, intermediate reasoning that was abandoned, redundant error messages.

Conversation: """ + "\n".join([f"{m['role']}: {str(m['content'])[:500]}" for m in messages])

response = client.messages.create(
    model=model,
    max_tokens=2048,
    messages=[{"role": "user", "content": summary_prompt}]
)
return response.content[0].text</code></pre>

Retrieved Data: Just-in-Time over Pre-Loading

Many agents pre-load a large corpus of potentially relevant data at the start of every run. This is the context engineering equivalent of copying an entire library into your working memory before starting research.

The more effective pattern: just-in-time retrieval. The agent maintains lightweight references -- file paths, query templates, document IDs, URLs -- and retrieves specific information on demand using tools.

Claude Code implements this precisely. Instead of loading the entire codebase into context, it uses grep and glob to find relevant files. It reads only the files it needs, when it needs them. The context window contains the agent's active working set, not a speculative pre-fetch of everything that might be relevant.

# Pre-loading pattern (avoid for large corpora)
def build_agent_context_naive(document_store: list[str]) -> str:
    # Loads everything -- expensive, pollutes context with irrelevant material
    return "\n\n".join(document_store)

# Just-in-time pattern (preferred)
def search_documents(query: str, document_store: dict[str, str]) -> str:
    """Tool the agent calls when it needs specific information."""
    # Return only the most relevant document, not the full store
    results = []
    for doc_id, content in document_store.items():
        if query.lower() in content.lower():
            # Return a snippet, not the full document
            idx = content.lower().find(query.lower())
            snippet = content[max(0, idx-200):idx+500]
            results.append(f"[{doc_id}]: ...{snippet}...")
    return "\n\n".join(results[:3]) if results else "No relevant documents found."

The key insight is that agent navigation itself is informative. File names, directory structures, timestamps -- these metadata signals help the agent understand context before it reads a single byte of content. Progressive disclosure lets the agent assemble understanding layer by layer, keeping its active context focused on what it's actually working with.

Structured Note-Taking: Persistence Across Resets

Compaction handles the context window limit, but there's a related problem: agents working on multi-session tasks lose continuity when their context resets between sessions.

The solution that works in practice is structured note-taking -- maintaining a persistent file (NOTES.md, TODO.md, or whatever your agent's convention is) that survives across context resets.

Claude Code does this automatically when you work on extended tasks. The agent maintains a to-do list and writes notes about decisions it has made, blockers it has found, and the current state of work in progress. When a new session starts, it reads its own notes and continues without requiring you to re-explain the context.

Anthropic's research on Claude playing Pokemon shows how far this pattern scales. Across thousands of game steps and multiple context resets, the agent maintained precise progress tracking: "For the last 1,234 steps I've been training in Route 1, Pikachu has gained 8 levels toward a target of 10." It developed maps of explored regions and strategic notes on combat. This coherence across compaction steps enabled multi-hour goal-directed behavior that would be impossible without structured external memory.

# Simple structured note-taking tool for long-horizon agents
import json
from pathlib import Path
from datetime import datetime

class AgentMemory:
    def __init__(self, notes_path: str = "AGENT_NOTES.md"):
        self.path = Path(notes_path)

    def read(self) -> str:
        if not self.path.exists():
            return "No notes yet."
        return self.path.read_text()

    def append(self, section: str, content: str) -> None:
        timestamp = datetime.now().strftime("%Y-%m-%d %H:%M")
        entry = f"\n### [{timestamp}] {section}\n{content}\n"
        with open(self.path, "a") as f:
            f.write(entry)

    def update_status(self, task: str, status: str) -> None:
        # Read current notes, update or add task status
        current = self.read()
        status_line = f"- [{status.upper()}] {task}"
        # In practice, use a proper parser; this is illustrative
        self.append("Status Update", status_line)

# Usage: give the agent these as tools
memory = AgentMemory()

def write_note(section: str, content: str) -> str:
    """Save a note for future sessions or context resets."""
    memory.append(section, content)
    return f"Note saved to {section}."

def read_notes() -> str:
    """Read all notes from previous sessions."""
    return memory.read()

Sub-Agent Architectures: Isolation as a Feature

When a single agent can't complete a task within its context budget -- either because the task is too complex or the required exploration would consume too many tokens -- the answer is sub-agent architectures.

The pattern: a coordinating agent maintains a high-level plan. It delegates focused sub-tasks to specialized sub-agents, each of which starts with a clean context window. The sub-agent explores its task deeply -- potentially using tens of thousands of tokens in tool calls and reasoning -- then returns a condensed summary (typically 1,000-2,000 tokens) to the coordinator.

The coordinator never sees the sub-agent's working context. It receives only the distilled output. This isolation means the main agent's context stays manageable regardless of how deeply the sub-agents explore.

Anthropic's multi-agent research system showed this pattern delivers substantial improvements over single-agent systems on complex tasks: in their internal research eval, a multi-agent setup (Claude Opus 4 lead, Claude Sonnet 4 sub-agents) outperformed single-agent Claude Opus 4 by 90.2%. The reasons are clear: parallel exploration, isolated context windows, and the ability to have sub-agents with specialized system prompts suited to their specific task.

import anthropic

client = anthropic.Anthropic()

def run_sub_agent(task: str, context: str, model: str = "claude-sonnet-4-5") -> str:
    """Run a focused sub-agent and return only its condensed output."""
    response = client.messages.create(
        model=model,
        max_tokens=2048,
        system="""You are a focused research sub-agent. Complete the assigned task thoroughly,
then provide a concise summary of your findings (under 1,500 words).
Do not include your reasoning process -- only the conclusions and key facts.""",
        messages=[
            {"role": "user", "content": f"Task: {task}\n\nContext: {context}"}
        ]
    )
    return response.content[0].text

def orchestrate_complex_task(main_task: str, subtasks: list[str]) -> str:
    """Coordinate multiple sub-agents and synthesize their outputs."""
    sub_results = []
    for subtask in subtasks:
        result = run_sub_agent(subtask, context=main_task)
        sub_results.append(f"Sub-task '{subtask}':\n{result}")

    # Coordinator synthesizes -- only sees summaries, not full sub-agent contexts
    synthesis_prompt = f"""Main task: {main_task}

Sub-agent findings:
{chr(10).join(sub_results)}

Synthesize these findings into a coherent final output."""

    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=4096,
        messages=[{"role": "user", "content": synthesis_prompt}]
    )
    return response.content[0].text

Choosing the Right Technique for Your Task

These techniques are not mutually exclusive. The best agents use combinations, and the right mix depends on task characteristics:

• Compaction works best for conversational tasks with extensive back-and-forth. It maintains the flow of a single agent across long interactions. Use it when the task has natural checkpoints where history can be safely summarized.
• Structured note-taking excels for iterative development and multi-session work. Use it when the agent needs to remember decisions across resets or when you need an audit trail of what the agent tried and why.
• Just-in-time retrieval is the right default for agents working with large corpora. Never pre-load -- always retrieve on demand. Use it whenever the agent's information needs are unknowable upfront.
• Sub-agent architectures handle complex research and analysis where parallel exploration pays off. Use them when a single agent's context budget can't accommodate the necessary depth of exploration, or when parallel subtasks would benefit from simultaneous execution.

A practical hybrid for a coding agent: start with just-in-time retrieval (the agent navigates the codebase on demand), apply compaction at context window limits, use structured note-taking to persist decisions across sessions, and spin up sub-agents for isolated research tasks like "analyze all usages of this function across the codebase."

The Guiding Principle

All of context engineering reduces to one idea: find the smallest set of high-signal tokens that maximizes the likelihood of your desired outcome.

Not the largest. Not the most comprehensive. The smallest set that works. Every token you add is a tax on the attention budget. Every noisy or redundant token you remove is a direct performance improvement.

This principle has practical consequences that run counter to the instinct to give agents more:

• Shorter, tighter system prompts often outperform longer, more exhaustive ones
• Fewer, well-designed tools beat comprehensive tool libraries
• Compacted history outperforms full history for retrieval accuracy
• Just-in-time retrieval beats pre-loaded context for large corpora

As models get smarter, the required engineering gets simpler. But even as capabilities scale, the context window will remain finite and attention will remain the scarce resource. Treating context as something to be carefully curated -- rather than something to be maximized -- will continue to be the distinguishing characteristic of agents that work reliably in production.

If you want to go deeper on building production-grade AI agents, join AI Builder Club. We share what's actually working in real codebases -- not just demos.