Context Engineering for AI Agents
Here’s a counterintuitive fact: give a model a 1-million-token context window, and it will often start losing the plot well before 300,000 token.This is context rot — a quiet, gradual decline in reasoning quality as the window fills, even when you’re nowhere near the technical ceiling.For most…
Here’s a counterintuitive fact: give a model a 1-million-token context window, and it will often start losing the plot well before 300,000 token.This is context rot — a quiet, gradual decline in reasoning quality as the window fills, even when you’re nowhere near the technical ceiling.For most current models, the effective context window — the zone where the model actually reasons well — tops out under 256k tokens, no matter what the spec sheet promises.This is the problem Context Engineering exists to solve: designing a system that feeds an LLM exactly the right information and tools, in exactly the right shape, to get a task done.It breaks down into four moves — offloading information out of the context entirely, reducing what’s already in there, retrieving new information on demand, and isolating context so unrelated work doesn’t bleed together.None of this lives inside the model itself. It’s handled by the agent harness — the software wrapped around the LLM that executes tool calls, manages the running message history, and applies all of this Context Engineering logic.The model reasons and proposes a tool call; the harness is what actually runs it and decides what the model gets to see next.Two failure modes tend to sneak in when the harness gets this wrong:Context pollution — the context fills with irrelevant, repeated, or contradictory information that actively distracts the model.Context confusion — the model can no longer tell instructions apart from data, or gets handed genuinely contradictory rules (often because system instructions collide with each other, or with the user’s).What follows are five patterns — pulled largely from how Manus has iterated on its own harness — for keeping an agent sharp as a task grows long and complicated.1. Compaction before summarization — reversibility beats compressionReducing context isn’t a single move — it’s a ladder, and the rule of thumb is: prefer whichever rung preserves the most information, and only climb down when you have to.Compaction is reversible. It strips out information that’s redundant because it already lives somewhere else — like the environment or filesystem. If an agent writes a 500-line file, the conversation shouldn’t carry all 500 lines forward. It just needs the path — Output saved to /src/main.ts — and the agent can go read the file again later if it actually needs to.Summarization is lossy, and it’s the fallback once compaction stops buying enough room.An LLM condenses the message history — tool calls included — usually once a token threshold is crossed (Manus uses roughly 128k). But there’s a subtlety worth stealing: Manus keeps its most recent tool calls in full, raw detail, and only summarizes the older ones.Compress everything uniformly and the model’s tone and formatting start to drift; keep the last few turns untouched, and it holds its rhythm.The order matters: raw context first, compaction second, summarization only when compaction alone can’t free enough space.2. Share context by communicating — not by communicating through shared contextMulti-agent setups tend to break for a specific, avoidable reason: every subagent dumping into the same shared context. Do that, and you’re paying a heavy KV-cache penalty on every call, while burying the model in details that have nothing to do with the subtask in front of it.Manus borrows a rule straight out of Go’s concurrency model here: share memory by communicating, don’t communicate by sharing memory. In practice, that splits into two patterns:Discrete tasks — anything with a clean input and output (”search this documentation for X”) gets a brand-new subagent with a fresh, isolated context and one specific instruction. Nothing more.Complex reasoning — only pass the full history when a subagent genuinely needs the whole trajectory to function, like a debugging agent that has to see every prior failed attempt so it doesn’t repeat one.Treat shared context as an expensive dependency, not a default. Forking context breaks your cache — so isolate unless you have a specific reason not to.3. Keep the model’s toolset smallHand a model 100+ tools and you’ll start seeing context confusion in a very specific form: hallucinated parameters, or the wrong tool called entirely. Manus’s fix is a hierarchical action space — three tiers, not one flat list:Level 1 — atomic tools: roughly 20 core actions the model actually sees — file_write, browser_navigate, bash, search, and the like. Stable, and cache-friendly because they barely change turn to turn.Level 2 — sandbox utilities: rather than giving the model a dedicated tool for every utility (a separate grep tool, say), it reaches those through Level 1 — calling bash, which calls ffmpeg or grep or Manus’s own mcp-cli via the command line. The tool definitions for all of that never touch the context window.Level 3 — code and packages: for a chained sequence like “fetch the city → get its ID → get the weather,” don’t force three separate LLM round-trips. Give the agent a library that handles the chain, and let it write one script that runs the whole thing deterministically.4. Treat “agent as tool” — not agent as coworkerIt’s tempting to build a whole org chart of agents — a Manager, a Designer, a Coder — chatting back and forth. Resist it. The cleaner pattern is to treat an agent exactly like you’d treat any other tool.From the main agent’s point of view, “run deep research” or “make a plan” is just a function call: call_planner(goal="..."). The harness spins up a temporary subagent loop behind the scenes and hands back a single structured result — no visible back-and-forth, no conversation to parse.This is effectively a MapReduce pattern for agents: the main loop defines the goal, the available tools, and the exact output schema it expects back — a specific JSON shape, say — so the result is immediately usable without any further parsing or interpretation.5. Five things worth internalizing before you build this yourselfSkip RAG for tool definitions. Dynamically fetching which tools are “relevant” per step based on semantic similarity sounds efficient, but it tends to backfire — the tool list shifts turn to turn, which breaks your KV cache and leaves the model confused about a tool that existed a moment ago and now doesn’t.Don’t train your own model yet. This is the Bitter Lesson playing out in real time: the custom harness you spend weeks fine-tuning today is likely obsolete the moment the next frontier model ships. Context Engineering is the flexible layer that adapts as models improve — locking yourself into a narrow, hand-trained action space just traps you in a local optimum.Set a pre-rot threshold, and act before you hit it. If a model’s advertised limit is 1M tokens but real degradation shows up under 256k, don’t wait for an API error to tell you you’ve gone too far. Monitor token count yourself and trigger compaction or summarization before you enter the rot zone.Use agent-as-tool for planning, specifically. Manus’s early versions leaned on a constantly-rewritten todo.md — which alone burned around 30% of tokens in earlier builds. A dedicated planner subagent that returns a structured plan object, injected only when it’s actually needed, is far cheaper than paying that tax every single turn.Sandbox isn’t optional, and neither is a human in the loop. The moment an agent has browser or shell access, isolation alone doesn’t fully cover you — enforce rules so tokens can’t leave the sandbox, and require manual confirmation before anything consequential actually executes.Judge agents with the “intern test.” Static benchmarks like GAIA saturate fast and stop tracking real user satisfaction. Better signal: binary, computationally verifiable checks. Did the code compile? Does the file exist after the command ran? Can a subagent’s output be verified by its parent? Prefer pass/fail on real environments over subjective LLM-as-a-judge scoring.Expect to rewrite it — repeatedly. Manus has been rebuilt five times in six months. LangChain re-architected Open Deep Research four times. That’s not a sign of instability — it’s what keeping pace with better models actually looks like.If your harness keeps growing in complexity while the underlying models keep improving, that’s usually a sign you’re over-engineering.The real lesson: less scaffolding, not moreThe most telling detail in all of this: Manus’s biggest performance gains over the last six months didn’t come from adding anything — no fancier RAG pipeline, no cleverer routing logic. They came from removing things.Dedicated tool definitions gave way to general shell execution. A layer of “management agents” gave way to simple, structured handoffs through agent-as-tool.As the underlying models keep getting stronger, the job isn’t to build more scaffolding around them — it’s to get out of their way. Context Engineering was never about maximizing how much context you can cram in. It’s about finding the smallest, most relevant context the model actually needs for its next step.This story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories.Subscribe to our newsletter and YouTube channel to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!Context Engineering for AI Agents was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.Source: Generative AI Pub — Published — Category: Image AI