The beginner thinks the 200K or 1M-token window is a bucket: the more you pour in, the more the agent knows. The reality is the opposite. Context is a finite, non-uniform, degradable resource you have to curate on every single turn. Every irrelevant token you add doesn't just cost money and latency. It costs accuracy. This module is about treating context the way an operating system treats RAM, and about having the precise vocabulary to diagnose why your agent "gets dumber" as the conversation runs.
Context is curated every turn
The line between prompt engineering and context engineering is temporal. Prompt engineering optimizes one static thing you write once: the system prompt. Context engineering manages the entire token stream (history, tool results, memory, files) that changes at every step of the agent loop. The system prompt still matters, but treated as an isolated artifact it's a subset of the problem, not the problem.
The governing principle fits in one Anthropic sentence: "find the smallest set of high-signal tokens that maximize the likelihood of your desired outcome." That turns context design into a constrained optimization, not an accumulation. The physical reason: transformer attention is pairwise, so it's O(n²), and models have a limited number of parameters trained for long-range dependencies, so recall reliability decays as length grows.
The most useful frame comes from MemGPT: treat the context window as limited working memory (RAM) and external storage as long-term memory (disk), with the LLM itself doing the paging, deciding what to load and what to evict, the way an OS moves pages between RAM and swap. The pedagogical extension is mine: thinking of context bloat as thrashing (when an OS spends more time paging than computing) explains why an agent with a window full of junk reasons worse even though "it all fits." [D]
Context rot: the finding that breaks the intuition
The central empirical evidence is Chroma's Context Rot study, which evaluated 18 frontier models (Claude Opus 4, GPT-4.1, Gemini 2.5, Qwen3 among them) and showed that models do not use their context uniformly: performance grows increasingly unreliable as input length grows.
This is why the classic lexical needle-in-a-haystack (NIAH) is misleading as a long-context capability test: it returns ~100% and hides the real degradation, because finding an exact string is trivial next to reasoning semantically over the content. To measure anything real you need multi-step benchmarks like LongMemEval or LOCOMO.
The most counterintuitive Chroma result almost nobody applies: shuffling the haystack (breaking the logical and structural coherence of the filler) consistently improved performance across all 18 models tested. The logical structure of the filler can confuse attention, which means "arranging the context nicely" can actively hurt. Worth noting what the study did not find: on the standard needle-in-a-haystack retrieval task there was no notable "lost in the middle" position effect (no significant variation across the 11 needle positions). A start-of-sequence advantage showed up only in a separate generation task (the Repeated Words experiment), so the familiar "put critical stuff at the front" lever is on weaker footing than the folklore suggests.
The four pathologies: a diagnostic vocabulary
"Context rot" describes that long context fails. Drew Breunig gives the taxonomy of why, which is what hands you a debugging vocabulary: when your agent goes stupid, is it poisoning or distraction?
The canonical poisoning case is documented by Google DeepMind: in their agent playing Pokémon with Gemini 2.5, a hallucination entered context and reinforced itself by being cited in later turns, derailing the agent's strategy. This connects straight to security: poisoning is a cousin of prompt injection, because a hostile fact injected through conversation can contaminate every future turn. [D]
Just-in-time retrieval: the agent explores, you don't pre-load
Instead of pre-loading all the data into the window (embedding-RAG style that injects a blob), the agent keeps lightweight identifiers (paths, queries, links) and loads data on demand with tools, discovering relevance progressively: a file's size hints at complexity, names hint at purpose, timestamps hint at recency. The center of gravity moves from "retrieval pipeline" to "agent that explores."
You index the whole corpus, retrieve top-k by similarity, inject the blob into the window. You pay for the pipeline, the indexing, and the drift, and the agent gets whatever the retriever decided. Still the right call for huge, heterogeneous corpora.
The agent uses grep/glob/read over files and decides what to read turn by turn. No embeddings, no indexing. For bounded corpora it matches or beats RAG, and it sidesteps the index-drift problem entirely. The retriever's judgment is replaced by the agent's. [C]
The trade is real: just-in-time means slower runtime exploration and more turns spent navigating. For a small, fast-changing set of documents that's a good deal. For a million heterogeneous PDFs it isn't, and a hybrid (pre-index the obvious, explore the rest) often wins.
Compaction and memory: discarding without losing what matters
Compaction is not "summarizing." It's a lossy operation where you choose what to throw away, at the risk of tossing context that's subtle but critical (an architecture decision, an unresolved bug). Anthropic's recommendation is to maximize recall first, over-preserve, then iterate on precision by removing the superfluous.
But compaction alone is not enough over very long horizons. Anthropic says it plainly: even a frontier coding model like Opus 4.5 falls short on very-long-horizon tasks with compaction alone. You complement it with filesystem memory (files like claude-progress.txt, feature_list.json, git) plus verification loops. That's structured note-taking: write-through long-term memory that survives context resets.
The key architectural distinction is short-term vs long-term. Short-term memory is the turn or session (the session transcript). Long-term crosses sessions (a per-user notes file). Frameworks like Mem0 go past "flat" memory: they add extraction and consolidation for selective retrieval, instead of reloading the whole file every time.
Profundizar: how a real harness approximates compaction
A harness doesn't need an explicit LLM-compaction step to get most of the benefit. A common pattern: rotate to a fresh session after N turns, persist only a pointer plus turn count, and resume with a recap fallback if the SDK's resume fails (distinguishing a timeout abort from a real error). Long-term memory is then a write-through save: the database is the source of truth, the local notes file is the turn's cache. The session transcript is short-term memory; the notes file is long-term. None of this is "true" compaction, but the combination of rotation plus externalized notes covers the same failure it's meant to prevent: the window filling with stale turns.
The ignored tension: pruning vs prompt caching
Here's the production knowledge almost everyone misses: compressing or pruning history invalidates cached prefixes. Prefix caching (the KV cache) requires stable prefixes. If you rotate or trim the start of the prompt, the cache breaks and you pay for the whole prefix again.
The remedy is cache-aware layout (dynamic-content-at-the-end): put the stable prefixes (system prompt, tool definitions, persona) at the front to maximize the cache, and move what changes every request (timestamps, session ids, user input) to the end.
[ system prompt ] stable prefix, cacheable (~0.1x on hits)
[ tool defs ] stable prefix
[ persona ] stable prefix
[ history ] semi-stable
[ timestamp, session_id, user input ] volatile, goes LASTThe conflict is real and it's an architecture decision, not a detail: aggressive context management (pruning, compaction) versus cache hit rate. The expert picks the cost-accuracy-latency trade-off deliberately. One caveat on durability: this caching knowledge is short-lived (APIs change their caching mechanisms), while the attention budget is timeless. [D]
Isolation via sub-agents, and its counter-thesis
A sub-agent with a clean window can explore with tens of thousands of tokens and return only 1-2K distilled tokens to the coordinator, keeping it uncontaminated. That's pure context isolation. But there's a genuine debate here that a serious engineer has to be able to arbitrate.
For read-heavy, parallel research with self-contained tasks, sub-agents work: each brings back its condensed finding and they don't coordinate mid-task. The regime where multi-agent earns its keep is parallel, read-heavy investigation.
Actions carry implicit decisions; parallel sub-agents make conflicting choices that can't be reconciled. The example: you ask for a Flappy Bird and get something half Super Mario. The argument is for single-threaded plus a dedicated compression LLM.
What the consensus overrates and what it ignores
Overrated: the obsession with bigger windows (1M tokens) as the solution. Context rot shows filling the window degrades accuracy; what matters is signal density, not capacity. Prompt engineering as a standalone discipline. It is a subset of context engineering. Lexical NIAH as a capability test. Multi-agent "by default because it scales." Embedding RAG as the automatic answer to "it doesn't fit."
Ignored or underused: there's a modest start-of-sequence advantage (it showed up in Chroma's generation task, though not in plain retrieval), so putting instructions and HITL rules at the front is a cheap lever almost nobody applies on purpose — just don't expect front-loading to rescue a needle buried in the middle. Tools are context too. Bloated or ambiguous tool definitions cause confusion, so shrinking and disambiguating the tool set is context engineering, not just API design. Memory without verification is debt: a write-through memory with no quality gate can persist poisoned data that contaminates every future turn. And structure helps. Markdown or XML headers and sections improve instruction-following over "everything in one blob."
Common mistakes
The through-line is one reflex. When the agent degrades over a long run, don't reach for a bigger window. Name the pathology, find the stale tokens, and decide what to evict.