Context Management System Architecture

You're building an agentic system. It works in demos. Single-agent, single-session, carefully curated prompts. Then you scale it. More agents. Longer sessions. Multiple projects running in parallel. And everything degrades.

The agents start repeating themselves. They forget constraints they acknowledged three turns ago. They make decisions that contradict decisions made in parallel sessions. Context gets bloated, responses slow down, and the quality that impressed you in the demo evaporates under production load.

This is the context management problem, and it's the central architectural challenge of any serious agentic system. The models themselves are capable enough. The bottleneck is getting the right information to the right agent at the right time without drowning it in everything else.

The Multi-Tier Context Architecture

Context in an agentic system isn't a single thing. It exists at multiple levels, each with different lifetimes, scopes, and update frequencies. Treating it as one undifferentiated blob leads to either bloated sessions (everything loaded everywhere) or amnesia (nothing available when needed).

The architecture that works at scale uses four tiers:

Tier 1: Global Context

Global context is information that applies to every agent across every project. Your organizational coding standards. Your security policies. Your preferred technology stack. Your communication conventions. These are the rules of the road - they change rarely, apply everywhere, and should be present in every agent's working context.

In practice, global context maps to files like CLAUDE.md at the user level, organization-wide skill definitions, and shared reference documents. The key property is stability - global context gets loaded once and rarely updated within a session.

The constraint on global context is size. Because it's loaded into every agent session, every token of global context multiplies by the number of concurrent agents. A 2,000-token global context across 100 parallel agents costs 200,000 tokens of aggregate budget. This creates a natural pressure toward conciseness: global context should contain only what genuinely applies everywhere.

Tier 2: Project Context

Project context is information specific to one project but shared across all agents working on that project. The project's architecture. Its file structure. Its dependency versions. Its domain terminology. Its pending decisions and unresolved questions.

This is where .planning/ directories, CLAUDE.md files at the project level, and codebase documentation live. Project context changes more frequently than global context - typically updated at phase boundaries or after significant architectural decisions.

The architectural decision here is inheritance: project context should extend global context, not replace it. An agent working on a project loads both the global rules and the project-specific rules. Where they conflict, project context wins. This layering prevents repetition while allowing customization.

Tier 3: Agent Context

Agent context is information specific to a particular agent's role and current task. The Planner agent needs different context than the Executor agent, even when they're working on the same project. The Planner needs architectural documentation and research findings. The Executor needs conventions and file structure. Loading both into both agents wastes tokens and dilutes attention.

Agent context is assembled dynamically based on the agent's role and the task at hand. A well-designed system maps agent types to relevant context categories:

Agent Role	Context Loaded
Planner	Architecture docs, research findings, requirements
Executor	Conventions, file structure, plan files
Verifier	Success criteria, test patterns, architectural constraints
Debugger	Error logs, recent changes, system dependencies

This selective loading is one of the biggest performance wins in context management. An agent that receives only the context it needs performs dramatically better than one drowning in everything the project knows.

Tier 4: Session Context

Session context is the accumulated state within a single agent interaction. The conversation history. Tool call results. Intermediate reasoning. Partial outputs. This is the most volatile tier - it grows continuously during a session and resets when the session ends.

Session context is where most systems fail. The naive approach appends every message and tool result to a growing transcript, which works for short sessions but degrades rapidly. After 20-30 minutes of active work, the transcript is so large that the model starts exhibiting recency bias - paying attention to whatever appeared most recently rather than what's actually important.

The solution is treating session context not as an append log but as a computed view. Instead of carrying every historical message, the system maintains a structured event log and compiles a relevant subset into working context for each inference call. Full tool results are stored by reference, not inlined. Previous conversation turns are summarized or pruned based on relevance to the current task.

Context Preparation and Initialization

When a new project starts or a new agent session begins, context doesn't assemble itself. Something needs to determine what information is relevant and prepare it for loading.

Automatic context preparation follows a pattern: analyze the task, identify relevant context categories, retrieve the corresponding artifacts, and compile them into a working context that fits within token budgets.

For new projects, this means:

Template-based initialization. Different project types need different context foundations. A frontend project needs component conventions and routing patterns. A backend API needs data access patterns and error handling conventions. A machine learning project needs data pipeline patterns and evaluation frameworks. Templates pre-select the relevant context categories for each project type.

Cross-project seeding. If you've done similar work before, the context from that work is valuable. Patterns that worked in Project A are likely relevant to Project B if they share similar characteristics. The system identifies related projects and seeds relevant patterns - not copying context wholesale, but extracting reusable elements.

Predictive pre-loading. Based on the task description and project type, the system predicts which context artifacts will be needed and loads them proactively. This reduces latency during the session - the agent doesn't need to search for context, it's already there.

Compression and Optimization

Context windows are large but not infinite, and every token has a cost - both in latency and in attention competition. Compression strategies let you fit more relevant information into the same budget.

The most effective approaches:

Semantic compression preserves meaning while reducing token count. Instead of carrying the full output of a tool call, carry a structured summary that captures the key findings. Instead of including an entire document, include the sections relevant to the current task. Research demonstrates 59% token reduction while maintaining task accuracy - a dramatic efficiency gain.

Progressive summarization compresses older context more aggressively than recent context. The full conversation history from five minutes ago is relevant. The full history from two hours ago is not - but its key decisions and conclusions are. Each older segment gets summarized to its essential content, freeing budget for recent high-value information.

Reference-based storage avoids tokenizing large artifacts entirely. Instead of including a full file's content in context, include a reference (file path, artifact ID) that the agent can fetch on demand. The agent sees what information is available without paying the token cost unless it actually needs the content.

Deduplication catches the same information stated multiple ways. If three documents all explain the same architectural decision, the system consolidates them into one representation. Redundant tokens consume budget without adding signal.

Cross-Project Context Sharing

In a multi-project environment, knowledge generated in one project is often valuable in others. A pattern for handling rate limiting discovered in Project A is relevant when Project B encounters the same API. An architectural decision about database indexing in one project informs similar decisions elsewhere.

Cross-project sharing requires three mechanisms:

Knowledge extraction identifies reusable insights from project-specific context. Not everything is shareable - implementation details are project-specific, but patterns, decisions, and lessons learned generalize.

Shared context pools store extracted knowledge in a searchable repository. When a new project initializes or an agent encounters a relevant situation, the system queries the pool for applicable knowledge.

Context federation synchronizes shared knowledge across projects without creating tight coupling. Each project maintains its own context hierarchy but can pull from the shared pool when relevant. Updates to shared knowledge propagate without disrupting active sessions.

Versioning and Rollback

Context evolves as projects progress. Requirements change. Architectural decisions get revised. New information invalidates old assumptions. The system needs to track these changes and support rolling back when an evolution proves wrong.

Context versioning works like Git for your project knowledge:

Snapshots capture the full context state at significant moments - phase completions, major decisions, milestone achievements. These snapshots provide restoration points when something goes wrong.

Diffs track what changed between versions. When a context update causes agent behavior to degrade, comparing the current context against the previous version reveals what changed and what might have caused the problem.

Branching supports experimental context changes without risking the stable version. You can test a new set of conventions or an updated architecture description on a branch and merge it back only if the results are satisfactory.

Rollback restores a previous context version when the current one is causing problems. This is the safety net that makes it safe to evolve context aggressively - you can always go back if an update was harmful.

Performance Metrics

Context management isn't something you build once and forget. It's an ongoing optimization problem that needs measurement.

The metrics that matter:

Token utilization rate - what percentage of the context budget is actually consumed? If you're consistently using only 30% of the available window, you're leaving value on the table. If you're consistently at 95%, you're one unexpected tool result away from truncation.

Context retrieval latency - how long does it take to assemble working context for a new agent call? Context that takes seconds to prepare introduces perceptible lag in interactive workflows.

Relevance score - when the agent receives context, how much of it does it actually reference in its response? Low relevance indicates wasted tokens. High relevance indicates effective context selection.

Cache hit rate - when the system assembles context, how often can it reuse previously computed context versus assembling from scratch? Higher hit rates mean faster context preparation and lower compute costs.

Quality correlation - how does context configuration affect output quality? This is the metric that connects context management to business value. Better context should produce measurably better agent outputs.

The Architecture in Practice

The multi-tier context architecture isn't theoretical. It's the pattern that emerges when you build agentic systems that actually work at scale.

At the simplest level, it's CLAUDE.md files (global context), project documentation (project context), agent-specific prompts (agent context), and conversation state (session context). The compression is progressive summarization. The versioning is git commits. The cross-project sharing is copying useful patterns between projects.

At the most sophisticated level, it's a managed platform with automated context compilation, semantic compression pipelines, intelligent retrieval, and continuous optimization based on performance metrics.

Most systems fall somewhere between these extremes. Start simple. Add sophistication when the simple version creates measurable problems. The architecture provides the framework for growth without requiring premature complexity.

Related

For practical implementation patterns and token management strategies, see Context Management Best Practices.

For the comprehensive framework that context management enables, see ACE Comprehensive Reference Specification.