Context Engineering for AI Systems: Architecture, Security, and Evals

Prompt engineering asks how to express an instruction. Context engineering asks a larger production question: what information, authority, state, and capabilities should the model receive at this moment—and what must remain outside?

The distinction matters because a modern AI application rarely sends one prompt. It assembles system policy, user intent, conversation state, retrieved documents, memory, tool definitions, tool results, examples, output schemas, and safety metadata. If these pieces are stale, contradictory, overbroad, or indistinguishable from untrusted text, a more capable model may simply fail with greater confidence.

Anthropic’s 2025 guide to effective context engineering describes context as a finite resource whose tokens should be selected for utility. Research gives a stronger warning: the Lost in the Middle study found that long-context models can perform worse when relevant information sits in the middle of a long input. A large window is capacity, not a guarantee that every token receives equal use.

This guide turns context engineering into a concrete architecture: a typed context manifest, a trust-aware assembly pipeline, token and latency budgets, security boundaries, and evaluations that isolate context failures from model failures.

Start With a Typed Context Manifest

Do not construct production prompts by concatenating strings. Represent each context item as a typed record before rendering it for the model.

A useful item envelope might contain:

id, type, content, source, owner, trust_level, authority,
created_at, effective_at, expires_at, sensitivity,
allowed_uses, transformation_history, token_estimate

The model does not need to see every field verbatim. The context builder does. These fields make it possible to filter, order, explain, invalidate, and audit inputs rather than treating the final prompt as an opaque blob.

Authority and Trust Are Different

An item can be trustworthy as data but lack authority to change the task. A webpage may accurately describe a product while having no authority to override the user’s instruction. A prior model summary may be useful state but should not outrank a current primary record. A user may be authoritative about their goal while still providing a mistaken factual claim.

Model these dimensions separately:

• Source authenticity: do we know where this item came from?
• Integrity: was it altered after retrieval?
• Factual authority: is this the system of record for the claim?
• Instruction authority: may it set or change behavior?
• Actor permission: is the requesting identity allowed to use it?
• Freshness: is it valid for the relevant time?
• Confidence: how reliable is extraction or inference?

Then define precedence in code. For example, application policy can constrain behavior; the current user request defines the goal inside those constraints; retrieved pages provide data, not instructions; tool output updates state but cannot silently broaden scope.

This is the key defense against “flat context,” where every natural-language token appears equally directive.

Build Context Through a Deterministic Pipeline

A robust context builder behaves more like a compiler than a scrapbook.

1. Normalize the request

Resolve active identity, tenant, locale, task type, time reference, expected output, and risk tier. Preserve the original user instruction alongside any structured interpretation so the system can detect semantic drift.

2. Load applicable policy

Select policy by product, jurisdiction, organization, role, task, and effective date. Do not send every policy document. Compile only applicable rules and preserve policy IDs for later explanation.

3. Restore scoped state

Load only the state required to continue the current workflow. Distinguish confirmed external state—such as a record returned by an API—from a model’s previous plan. Revalidate state when it may have changed since the last turn.

4. Retrieve evidence

Filter by permissions and source authority before semantic ranking. Return exact supporting passages, stable identifiers, dates, and access labels. If sources conflict, include the conflict or escalate; do not let a similarity score choose the truth silently.

Read RAG and enterprise knowledge quality for the retrieval layer and evidence-first automation for claim-to-source traceability.

5. Select memory

Apply identity, project, sensitivity, status, and temporal rules before relevance. A recalled preference is not a policy. A stale project decision should be invalidated, not summarized more elegantly. The control-first AI memory guide explains this lifecycle.

6. Register tools

Expose only capabilities required for the task and current actor. Tool descriptions should state side effects, required approval, idempotency expectations, and failure behavior—not only a friendly name and parameter list.

7. Allocate a token budget

Assign budgets by type and risk. Preserve non-negotiable policy, immediate task, critical evidence, identifiers, and unresolved conflicts. Compress verbose history and low-value examples first.

8. Render and validate

Render clear structural boundaries, then run machine checks: required sections present, no item above the actor’s permissions, token limit respected, schema valid, source IDs resolvable, and tool set consistent with policy.

Select Before You Compress

Summarizing a noisy pile produces a smaller noisy pile. Use four operations in order:

1. Exclude: remove irrelevant, unauthorized, expired, duplicated, or low-authority content.
2. Extract: keep exact spans and structured fields that support the task.
3. Compress: summarize remaining long material while preserving source IDs, disagreements, numbers, and qualifications.
4. Position: place high-priority information where the model is most likely to use it and where it cannot be confused with untrusted data.

Compression is lossy. Store both the compressed representation and its source references. For consequential facts, require the model or verifier to consult the original supporting span before action.

The “lost in the middle” result should not be turned into a universal trick such as repeating instructions at both ends. The durable response is evaluation: shuffle evidence position, add realistic distractors, and measure whether the system still identifies the right source. If moving one paragraph changes a decision, the context pipeline is brittle.

Context Has a Token, Latency, and Money Budget

Every additional item has at least four costs:

• Input tokens and provider charges.
• Prefill latency before generation starts.
• Attention competition with more relevant material.
• Disclosure risk if the content is sensitive or untrusted.

Maintain an item-level estimate before rendering. A practical budget might reserve fixed capacity for policy and task state, dynamically allocate evidence based on risk, and leave room for tool results that arrive during execution.

Do not optimize only for the smallest prompt. The objective is the smallest context that achieves the required quality and safety. A regulatory comparison may require multiple authoritative sources; a simple formatting request may need no retrieval or personal memory at all.

Prompt caching can reduce repeated prefill cost for stable prefixes, but caching is a storage decision. Provider documentation may impose different retention or eligibility rules. OpenAI’s current API data-controls documentation, for example, states that extended prompt caching has storage implications and is not compatible with its Zero Data Retention option. Treat such details as provider-specific and verify them before deployment.

Tools and MCP Expand the Context Boundary

Tool integration adds both capabilities and new context sources. The official Model Context Protocol architecture distinguishes tools, resources, and prompts:

• Tools execute functions.
• Resources provide context data.
• Prompts provide reusable interaction templates.

That separation is useful, but a protocol does not decide which server to trust, which result is authoritative, or whether an action matches user intent. Those remain application responsibilities.

The July 28, 2026 MCP release candidate introduced a stateless core, extensions, tasks, apps, and authorization hardening. Fast-moving protocol changes make version pinning and capability negotiation part of context engineering. Record server identity, protocol version, declared capabilities, tool-schema version, authorization scope, and response provenance in each run.

Never treat a tool description or response as policy merely because it arrived through an integration. A compromised server, document, email, webpage, or database field can return text that looks like an instruction.

Prompt Injection Is a Context-Architecture Problem

The OWASP 2025 prompt-injection guidance notes that indirect injections can arrive through external websites and files, and that RAG or fine-tuning does not fully solve the problem. The essential controls are architectural:

• Label and isolate external content as untrusted data.
• Prevent retrieved text from granting permissions or adding tools.
• Enforce authorization and business rules outside the model.
• Give each task the least-privilege tool set.
• Screen proposed actions against the original user intent, not the latest retrieved text.
• Require explicit confirmation for high-impact or irreversible actions.
• Keep secrets and credentials outside model-visible context.
• Sanitize and validate model output before using it in code, SQL, URLs, or commands.
• Log the context item that influenced each tool action.
• Red-team hidden instructions in HTML, PDFs, images, metadata, tool output, and remembered summaries.

The OWASP prompt-injection prevention cheat sheet describes a useful pattern: the component with privileged tools should not directly consume uncontrolled content. Even then, a model-based guard is one layer, not proof of safety.

For implementation detail, see browser-agent security and tool permission design.

Evaluate the Context Builder Separately From the Model

If model, retrieval, memory, policy, and tools all change together, a failed answer is hard to diagnose. Freeze the model and evaluate context variants; then freeze the context fixture and compare models.

Build a test matrix covering:

Measure:

• Task success and factual correctness.
• Context precision: fraction of included items that were useful.
• Context recall: fraction of required items included.
• Source attribution and citation support.
• Stale-context and conflict-detection rates.
• Unauthorized disclosure and prompt-injection success rates.
• Tool-selection and argument correctness.
• Tokens, prefill latency, total latency, and cost by context type.
• Sensitivity to ordering, paraphrase, noise, and context length.
• Human correction and escalation quality.

A “good prompt” that works only with one document order is not production-ready context engineering.

A Reference Implementation Plan

1. Define the typed item schema and source registry.
2. Write precedence rules for policy, user intent, state, evidence, and memory.
3. Create permission and freshness filters before retrieval ranking.
4. Implement token accounting per item and per context class.
5. Add a renderer that creates stable, testable structural boundaries.
6. Validate the final manifest before every model call.
7. Store item IDs, not only the rendered prompt, in traces.
8. Create golden fixtures from real tasks, conflicts, incidents, and adversarial documents.
9. Compare no-retrieval, no-memory, and reduced-context baselines.
10. Monitor production drift by model, context-builder version, source, language, and risk tier.

Frequently Asked Questions

Is context engineering just RAG?

No. RAG is one source-selection mechanism. Context engineering also covers policy, user intent, workflow state, memory, tool affordances, output contracts, permissions, ordering, compression, and evaluation.

Should the full conversation always be included?

No. Preserve the current goal, decisions, unresolved questions, and necessary evidence. Expire or summarize low-value turns, and never assume old model output is authoritative merely because it is recent.

Does a larger model reduce the need for context design?

No. A stronger model cannot retrieve a document it was not given, repair a wrong permission boundary, know that a policy is superseded, or safely distinguish untrusted instructions without an application architecture that represents those facts.

Where should non-negotiable instructions go?

Place them in a structurally distinct, high-authority policy layer and enforce critical rules outside the model. Do not rely on ordering alone for permissions, financial limits, data access, or destructive-action controls.

Sources and Review Date

This article was substantially reviewed on July 30, 2026 using:

• Lost in the Middle
• Anthropic: Effective context engineering for AI agents
• Model Context Protocol architecture
• MCP 2026-07-28 release candidate
• OWASP LLM01:2025 Prompt Injection
• OWASP Prompt Injection Prevention Cheat Sheet
• OpenAI API data controls

The model is only the execution engine. The context builder decides what world the model is allowed to see—and therefore deserves the same rigor as any other production decision system.