"Prompt engineering" always sounded like a bag of tricks: magic phrases, capital letters, "think step by step". In the last two weeks a better name has taken over. On June 18, Shopify's CEO Tobi Lutke wrote: "I really like the term 'context engineering' over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM." On June 25, Andrej Karpathy agreed, describing it as "the delicate art and science of filling the context window with just the right information for the next step."

The shift matters because it moves attention from the wording of one instruction to the whole input: system prompt, examples, retrieved documents, tool definitions, tool results, memory, conversation history and state. In an agent that runs for fifty steps, the hand-written prompt is a small fraction of what the model reads. Everything else is assembled by your code, and that code decides whether the agent works.

This cheatsheet collects what is known as of the end of June 2025: what goes in the window, the four strategies to manage it, how long contexts fail, how to budget tokens, how to order things for caching, and a small Python builder that puts it together.

1. Definitions

Term Meaning
Context window Everything the model sees for one call, measured in tokens: input plus the output it generates
Context engineering Designing the system that decides what goes into the window, in what form and order, at each step. LangChain's definition: "building dynamic systems to provide the right information and tools in the right format such that the LLM can plausibly accomplish the task"
Prompt engineering Writing the instructions. Now one part of context engineering
Static context Parts that rarely change: instructions, tool definitions, examples
Dynamic context Parts that change per step: retrieved documents, tool results, history, state
Token budget The maximum tokens allotted to each part of the context

Why "dynamic systems": in a chat app or agent, the context is not written once. It is rebuilt on every call, from many sources, by code. Most agent failures are not model failures; the model did not have the information it needed, or had too much of the wrong information.

2. What goes into the context window

Component What it is Typical problems
Instructions (system prompt) Role, goals, rules, output format, constraints Too vague, or so long and contradictory the model ignores parts
Examples (few-shot) Input/output pairs showing the expected behavior Overfitting to the examples; stale examples after the task changes
Retrieved documents RAG results, files, search hits Irrelevant chunks, duplicates, outdated versions
Tool definitions Names, descriptions and schemas of available tools Too many tools, overlapping descriptions
Tool results Outputs of calls: API responses, file contents, logs, search results Huge raw payloads (a 20K-token JSON for one field)
Memory Facts persisted across sessions: user preferences, past decisions Wrong or stale memories reused forever
History Previous turns of this conversation or agent run Grows without bound; early mistakes keep being referenced
State Plan, TODO list, scratchpad, current step, variables Lost when history is truncated if it only lived in the transcript
User request The actual task for this step Buried in the middle of everything else

A useful exercise: log the full context of a failing call and read it as the model would. Most of the time the cause is obvious within a minute: the needed fact is missing, a stale fact contradicts it, or it is buried under 80K tokens of tool output.

3. The four strategies: write, select, compress, isolate

Lance Martin (LangChain) grouped the common techniques into four buckets. They are a good map:

Strategy Idea Techniques
Write Save information outside the window so it does not have to stay inside Scratchpads, plan files, TODO lists, long-term memory stores, writing large tool outputs to files
Select Pull only what is relevant into the window for this step RAG over docs and memories, selecting relevant tools ("tool loadout"), choosing relevant examples
Compress Keep the signal with fewer tokens Summarizing history, trimming old turns, extracting fields from tool outputs, pruning
Isolate Split context across separate windows Sub-agents with their own context, sandboxed environments that hold large objects, state fields not shown to the model

Drew Breunig's "How to Fix Your Context" (June 26) gives a similar list with names that are handy in design reviews: RAG, tool loadout, context quarantine, context pruning, context summarization and context offloading.

Isolation and multi-agent systems: two views

  • Cognition (June 12), "Don't Build Multi-Agents": sub-agents that see only fragments of the context make conflicting decisions. Their principles: "Share context, and share full agent traces, not just individual messages" and "Actions carry implicit decisions, and conflicting decisions carry bad results". Default to a single-threaded agent, and compress history when it gets too long.
  • Anthropic (June 13), multi-agent research system: for broad research tasks that parallelize well, a lead agent with sub-agents outperformed a single agent by 90.2% on their internal research eval, at roughly 15 times the tokens of a chat interaction.

Both are consistent with one rule: isolate when sub-tasks are truly independent (parallel reading, searching), keep one context when decisions depend on each other (writing code, editing one document).

4. How long contexts fail

Drew Breunig's "How Long Contexts Fail" (June 22) names four failure modes. They are the clearest vocabulary available right now:

Failure Definition (Breunig) Example Fix
Context poisoning "When a hallucination or other error makes it into the context, where it is repeatedly referenced" The Gemini 2.5 Pokemon agent hallucinated game state into its goals and pursued impossible objectives Validate before writing to memory or goals; quarantine; restart from a clean summary
Context distraction "When a context grows so long that the model over-focuses on the context, neglecting what it learned during training" The same agent, beyond about 100K tokens, repeated past actions instead of planning new ones Summarize and prune; keep the working context short
Context confusion "When superfluous content in the context is used by the model to generate a low-quality response" Irrelevant tool definitions cause wrong tool calls, especially in smaller models Tool loadout: give only relevant tools; drop irrelevant docs
Context clash "When you accrue new information and tools in your context that conflicts with other information in the context" Information revealed over several turns: Laban et al. (May 2025) measured an average 39% drop vs. giving it all at once Consolidate into a single updated statement of the task; remove superseded content

Rules of thumb that follow from these:

  • Fewer tools is better. Breunig cites results where models started confusing tools at around 30 definitions and a small model failed with 46 tools but succeeded with 19. Select tools per step.
  • Do not let errors persist. Once a wrong fact is in history or memory, the model will keep citing it. Correct it explicitly or rebuild the context.
  • Restate the task. In long runs, keep a current, consolidated task statement and plan near the end of the context, rather than relying on turn 3 of 60.

5. Long context limits: lost in the middle

A 1M-token window does not mean 1M tokens of equal attention.

  • Lost in the Middle (Liu et al., 2023): on multi-document QA and key-value retrieval, accuracy was highest when the relevant information was at the beginning or end of the context and dropped when it was in the middle, even for long-context models.
  • Effective length is shorter than advertised. Breunig cites a Databricks study where Llama 3.1 405B accuracy started to drop around 32K tokens, earlier for smaller models.
  • Multi-turn is harder than single-turn. Laban et al. found models "make assumptions in early turns and prematurely attempt to generate final solutions, on which they overly rely".

Practical placement:

  1. Instructions and stable reference material at the start.
  2. The current task, latest state and the most relevant evidence at the end, closest to where the model generates.
  3. Less important or older material in the middle, where losing some attention costs least.

6. A token budget table

Budgets make trade-offs explicit and stop one component (usually tool results or history) from eating the rest. An example for an agent on a model with a 200K-token window that you want to keep well below the limit for quality, cost and latency:

Component Budget Policy when over budget
Instructions 2-4K Hard limit. If it does not fit, the prompt needs editing, not truncating
Tool definitions 2-6K Select relevant tools per step (tool loadout)
Examples 1-3K Pick the 2-5 most similar examples to the current task
Memory 1-2K Retrieve top-k relevant memories, never dump all
Retrieved documents 10-30K Rerank, keep top chunks, deduplicate
State (plan, TODO, scratchpad) 1-2K Keep current; rewrite rather than append
History 10-30K Keep recent turns verbatim, summarize older ones
Tool results (current step) 5-20K Extract needed fields; write full output to a file and reference it
User request as needed Always included
Reserved for output and reasoning 8-32K Never borrow from this
Target total 50-120K Treat the window maximum as a ceiling, not a goal

The exact numbers depend on the task. The habit that matters is measuring the actual tokens per component on real traces, and alerting when one component grows out of line.

7. Ordering for prompt caching

Prompt caching makes repeated prefixes cheaper and faster, and it rewards a specific ordering: stable first, volatile last.

  • Anthropic (August 2024): the cache prefix is built in the order tools, system, messages; a change at one level invalidates that level and everything after it. Cache reads cost 10% of the base input price, writes 25% more; the default lifetime is 5 minutes. Anthropic reported up to 90% lower cost and 85% lower latency on long prompts.
  • OpenAI (October 2024): caching is automatic for prompts longer than 1,024 tokens, with a 50% discount on cached input tokens. Put static content at the beginning.

Rules that follow:

Do Do not
Put instructions, tool definitions and fixed examples first, byte-identical across calls Put a timestamp, request ID or user name at the top of the system prompt
Append to history; keep earlier turns unchanged Rewrite or reorder earlier turns on every step
Keep the tool list stable within a session where possible Add and remove tools mid-session without need (it also breaks the prefix)
Put dynamic content (retrieved docs, current task) after the stable prefix Serialize JSON with non-deterministic key order

There is a real tension with section 3: selecting tools per step and summarizing history change the prefix and lose cache hits. Resolve it by compressing in large, infrequent steps (summarize once at a threshold, then append again) rather than rewriting the context on every call.

8. A context builder with budgets

A minimal, framework-free sketch: each block has a budget, a priority (what to drop first) and an order (where it goes in the prompt, stable blocks first for caching).

from dataclasses import dataclass, field

MARKER = "[...older content truncated...]\n"


def count_tokens(text: str) -> int:
    """Rough estimate: about 4 characters per token in English. Use the model's tokenizer in production."""
    return max(1, len(text) // 4)


@dataclass
class Block:
    name: str
    text: str
    budget: int      # max tokens for this block
    priority: int    # lower = more important; low-priority blocks are dropped first
    order: int       # position in the prompt; stable blocks first so the prefix can be cached


@dataclass
class ContextBuilder:
    window: int = 200_000
    reserve_output: int = 8_000
    blocks: list[Block] = field(default_factory=list)

    def add(self, block: Block) -> "ContextBuilder":
        self.blocks.append(block)
        return self

    @staticmethod
    def fit(text: str, budget: int) -> str:
        """Trim a block to its budget, keeping the most recent part."""
        if count_tokens(text) <= budget:
            return text
        return MARKER + text[-(budget * 4 - len(MARKER)):]

    def build(self) -> tuple[str, dict[str, int]]:
        available = self.window - self.reserve_output
        used: dict[str, int] = {}
        kept: list[tuple[Block, str]] = []
        for b in sorted(self.blocks, key=lambda b: b.priority):
            text = self.fit(b.text, b.budget)
            cost = count_tokens(text)
            if sum(used.values()) + cost > available:
                continue  # does not fit: drop it (or summarize it)
            used[b.name] = cost
            kept.append((b, text))
        kept.sort(key=lambda bt: bt[0].order)
        prompt = "\n\n".join(f"<{b.name}>\n{t}\n</{b.name}>" for b, t in kept)
        return prompt, used


ctx = ContextBuilder(window=32_000, reserve_output=4_000)
ctx.add(Block("instructions", "You are a support agent for Acme. " * 50, budget=2_000, priority=0, order=0))
ctx.add(Block("tools", "search_orders(order_id): ... " * 40, budget=3_000, priority=1, order=1))
ctx.add(Block("examples", "Q: ... A: ... " * 200, budget=2_000, priority=5, order=2))
ctx.add(Block("memory", "User prefers email. Plan: Pro. " * 20, budget=1_000, priority=2, order=3))
ctx.add(Block("retrieved_docs", "Refund policy: ... " * 3_000, budget=12_000, priority=3, order=4))
ctx.add(Block("history", "user: ... assistant: ... " * 4_000, budget=8_000, priority=4, order=5))
ctx.add(Block("task", "Where is my refund for order 1234?", budget=500, priority=0, order=6))

prompt, used = ctx.build()
print(used)
# {'instructions': 425, 'task': 8, 'tools': 290, 'memory': 155,
#  'retrieved_docs': 12000, 'history': 8000, 'examples': 700}

What to add for production:

  • Summarize instead of truncate for history: replace the oldest turns with an LLM-written summary when the block exceeds its budget, and keep decisions and open questions explicitly.
  • Select instead of truncate for documents and tools: rerank and keep the top items rather than cutting text mid-chunk.
  • Log used per call. Token usage per component is the most useful context-engineering metric you can chart.
  • Keep state outside history (a plan or TODO block rebuilt each step), so truncating history never loses the plan.

9. Checklist

  • [ ] Can you print the exact full context of any production call?
  • [ ] Does each component have a budget, and is usage logged?
  • [ ] Are only relevant tools exposed for the current step?
  • [ ] Are large tool outputs reduced to what is needed, with the full output stored elsewhere?
  • [ ] Is the current task restated near the end?
  • [ ] Are memories and summaries validated before they are written?
  • [ ] Is the stable prefix byte-identical across calls?
  • [ ] Do sub-agents get enough shared context to avoid conflicting decisions?

Why this matters

  • The model can only use what it sees. Most agent failures are context failures: missing, wrong, stale or buried information.
  • More context is not better context. Poisoning, distraction, confusion and clash all get worse as the window fills.
  • Write, select, compress, isolate is a complete enough toolbox to reason about any design.
  • Budgets and ordering are engineering, not prompting. They are measurable, testable and cacheable.
  • The term is new; the discipline is not. It is information retrieval, state management and caching, applied to a model's input.

Final Thoughts

"Context engineering" caught on because it names what people building agents were already spending their time on. The instructions are a page; the context is everything your system decided to show the model at step 37.

Treat that context as a data structure you own. Give it a schema, budgets, an order, tests and logs. When an agent misbehaves, read its context before you touch the prompt. Most of the time the fix is there.

Sources