Context engineering: making production agents more reliable

Prod, not IDE session

In production, context engineering goes far beyond compact and clear in a Claude Code session. It means designing everything that reaches the model when a business agent runs: the system prompt, the tools, the documents, the history and the policies. IDE session hygiene has its own article.

Incomplete, contradictory or excessive context can make a task fail even with a capable model. Inspect the information supplied alongside the model choice, then measure the effect on a test set.

The bottleneck therefore moves from model choice to the context sent on every request, in a replayable way. And without the third brick, the control one (allowlist, logs, human review), you just have a POC or a demo.

Four strategies: Write, Select, Compress, Isolate

A widely reused frame, formalized by LangChain for agents, organizes the work in four moves:

• Write: persist context outside the window (state machine, ticket store, file handoff, project memory). The window is finite, so what must survive between calls gets written elsewhere.
• Select: retrieve only what is relevant, with a bounded RAG, filters, a top-k and inclusion rules. A whole Drive is not context.
• Compress: summarize, truncate tool outputs, keep the last N raw turns plus a summary of the rest (hybrid sliding window). On long tasks, compress first.
• Isolate: separate contexts per agent or sub-task, through subagents. The orchestrator does not swallow the full research history.

What sits in a context window

Four practical pillars recur in agent guides:

• Instructions: the system prompt and the framing
• Retrieval: the RAG and the documents
• Memory: the short-term conversation, the long-term project
• Tools: function calling, increasingly via MCP

Noisy tools, such as huge JSON or raw logs, rot the context as fast as an overly long chat. In practice, we filter, summarize and cap the size of tool responses.

If you wire several MCP servers, count the tool-schema tokens before any user request. That entry cost is often underestimated.

Context rot and quality

Retrieval quality can vary with information position and context length. The effect depends on the model and task: evaluate your own documents rather than assuming a universal context-fill threshold.

On a business agent, that means an invented price, a forgotten policy or an obsolete doc being cited. The countermeasure combines a versioned corpus, mandatory citations, a document cap, a state reset between tickets and isolation for heavy research.

Also version the prompts, the templates, the retrieval pipelines and the tool definitions like code. An undated "small change" makes debugging impossible.

Skills, routing, evaluation

Versioned skills and playbooks make the procedure explicit. A programmed trigger can make workflow selection predictable; it does not make model responses deterministic.

If the agent serves several domains, add routing, even simple keyword rules at first, to cut the bloat before investing in an LLM classifier.

Finally, evaluate with a set of 30-50 gold cases that map an input to the expected output or action class. Measure tokens per ticket, failure rate and human edit rate. Without a gold set, every "improvement" stays an opinion.

Prod checklist

Before calling it production, check these eight points:

1. The injected sources are listed
2. Write, Select, Compress and Isolate are applied
3. Tool response size is capped
4. MCP schema tokens are counted
5. Prompts, templates and pipelines are versioned
6. Human review covers high-impact outputs
7. A monthly budget is set
8. A gold case set exists

If any point is missing, you still optimize by feel. That is fine for a spike, not for customer production.

If you want a context audit on a business agent, we can scope in 20-40 minutes.

Frequently asked questions