祝融说。

The context window has an upper bound, but a session can grow indefinitely. Compaction is not an option — it is a survival requirement.

Pattern Layer

13.1 Compaction is a Survival Requirement

Context window sizes have been growing — from 4K tokens a few years ago to 1M tokens today. But no matter how large the window is, a continuously running agent session can fill it.

A typical scenario: an agent executes 100 tool calls in 8 hours. Each tool call (including input and output) averages around 2,000 tokens — that is 200K tokens of context a day. Add the system prompt and user messages, and it easily exceeds 100K tokens within a single day. A session spanning a week can reach 1M tokens.

Without compaction, the consequence is passive eviction: the earliest messages are pushed out of the window by new messages, with no prioritization. The earliest and most important information (the user's first requirement constraint) and the earliest and least important information (200 file names returned by the first directory listing) are treated equally — pushed out together.

The difference between active compaction and passive eviction is that active compaction has a prioritization strategy — it knows what to drop first, what to drop later, and what must never be dropped. Passive eviction has no strategy — first come, first gone.

Active compaction is not "deleting" information — it is degrading information from "complete context" to "summary," freeing up token budget for new information. The compacted content remains in the persisted Session file and can be restored at any time.

13.2 The Invariant: Derived vs Overwritten

Compaction and persistence are two different operations that must be clearly distinguished.

Persistence is "overwritten" — the original record is saved in its entirety to the Session file on disk. The Session file is append-only; nothing is deleted or truncated. Compaction does not affect persistence — the Session file always contains the complete operation history.

Compaction is "derived" — a summary is generated from the complete context and replaces the position of the original content. The compacted content is removed from the context window, but its original content remains in the Session file.

This invariant means: the compacted context window is not a complete session record — it is a working set. The working set contains the information needed at the moment, but not the complete operation history. If the complete history is needed, the Session file's original content can be read.

13.3 Two-Tier Compaction Strategy

CodeCoder's compaction strategy is divided into two tiers:

tier-1: Discard + Placeholder.

tier-1 triggers when the context reaches a threshold. It does two things:

1. Discard Reasoning tokens: The LLM's thinking process. In CodeCoder's usage observations these typically occupy a substantial fraction of the context (roughly thirty to fifty percent, varying by task type) and have the lowest information density. If the agent needs to reproduce the reasoning later, it should explicitly save the reasoning conclusion in Memory

2. Placeholder-ize old ToolResult bodies: Replace the verbose body of older ToolResults with a summary plus file path. Preserves "what was done" but discards "what details were returned"

tier-1 modifications are reversible — the placeholders' original ToolResult bodies are fully preserved in the Session file. If the agent needs to look back, it can re-read the Session file.

tier-2: Structured Summary.

When the context still exceeds the threshold after tier-1, tier-2 is triggered. It produces a structured summary of the earliest conversation segment:

[Summary]
Goal: Refactor module A's interface
Constraints: Maintain backward compatibility
Progress: Interface extraction complete, tests passing
Key Decisions: Abandoned generics approach; using trait objects instead
Next Steps: Implement module B's interface in new token

The five fields of the summary template:

• Goal: What was being done at the time
• Constraints: Explicit user requirements
• Progress: What has been accomplished
• Key Decisions: What design choices were made
• Next Steps: What should be done next

tier-2 summaries are iteratively merged — each time only the incremental portion is summarized, and accumulated file-tracking information is appended at the end of the summary:

[File Tracking]
Read: src/mod.rs, src/interface.rs
Modified: src/interface.rs, tests/interface_test.rs

Case Layer

13.4 tier-1: Discard Reasoning + Placeholder-ize ToolResult

The concrete implementation of tier-1 compaction:

fn compact_tier1(messages: &mut Vec<Message>) -> usize {
    let mut freed_tokens = 0;

    for message in messages.iter_mut() {
        // 1. Discard Reasoning tokens
        let reasoning_count = message.items.iter()
            .filter(|item| matches!(item, MessageItem::Reasoning(_)))
            .count();
        message.items.retain(|item| !matches!(item, MessageItem::Reasoning(_)));
        freed_tokens += reasoning_count * AVG_TOKEN_PER_REASONING;

        // 2. Placeholder-ize old ToolResult bodies
        // Only keep the most recent N ToolResults in full
        let recent_count = 10;  // Keep the 10 most recent
        let tool_results: Vec<_> = message.items.iter_mut()
            .filter_map(|item| {
                if let MessageItem::ToolResult(tr) = item {
                    Some(tr)
                } else {
                    None
                }
            })
            .collect();

        let total = tool_results.len();
        for (i, tr) in tool_results.iter_mut().enumerate() {
            if i < total.saturating_sub(recent_count) {
                // Placeholder-ize: replace content with summary
                let summary = tr.content.iter()
                    .map(|c| match c {
                        ContentItem::Text(s) => s.chars().take(200).collect::<String>(),
                        _ => "[binary]".to_string(),
                    })
                    .collect::<Vec<_>>()
                    .join(" ");
                tr.content = vec![ContentItem::Text(format!(
                    "[TRUNCATED: {} bytes, {} chars]",
                    tr.original_size, summary.len()
                ))];
                freed_tokens += tr.original_size / AVG_BYTES_PER_TOKEN;
            }
        }
    }

    freed_tokens
}

Key design points of tier-1:

• Anchor protection: recent_count = 10 ensures the most recent N ToolResults are not placeholder-ized — even if they also belong to "earlier" messages. Anchors represent "the minimum context needed for current reasoning"
• All Reasoning discarded: No selection — all Reasoning tokens are dropped
• Placeholders retain a small amount of information: Placeholder-ized ToolResults still keep the "file path / result summary" (the first 200 characters), not completely blank

13.5 tier-2: Structured Summary

tier-2 compaction is triggered when the context still exceeds the threshold after tier-1:

fn compact_tier2(messages: &mut Vec<Message>, context: &Context) -> Result<usize> {
    // 1. Find the earliest segment (contiguous message block eligible for compaction)
    let span = find_compressible_span(messages)?;

    // 2. Call the LLM to generate a structured summary
    let summary = summarize_span(&span, context)?;

    // 3. Replace the segment with the summary
    let summary_tokens = estimate_tokens(&summary);
    let original_tokens = span.iter().map(|m| m.tokens).sum::<usize>();

    replace_span_with_summary(messages, &span, &summary);

    Ok(original_tokens - summary_tokens)
}

fn summarize_span(span: &[Message], context: &Context) -> Result<String> {
    let prompt = format!(
        "Please summarize the following conversation segment into a structured format.\
         \nFields: Goal, Constraints, Progress, Key Decisions, Next Steps.\
         \n\nConversation content:\n{}",
        format_messages(span)
    );
    context.llm_complete(&prompt)
}

Summary structure:

[Summary]
Goal: Refactor module A's interface, extract into an independent trait
Constraints: Maintain backward compatibility, do not modify module B
Progress: Interface extraction complete, 3 test cases written
Key Decisions: Using trait objects rather than generics (to avoid impacting module B's compile time)
Next Steps: Begin module B's interface adaptation

[File Tracking]
Read: src/mod.rs, src/interface.rs, tests/interface_test.rs
Modified: src/interface.rs, tests/interface_test.rs

Iterative merging: tier-2 does not regenerate summaries from all history on every invocation. It only summarizes the incremental portion — from the point of the last summary to the present time. Then it merges into the previous version of the summary:

fn merge_summaries(old: &Summary, new: &Summary) -> Summary {
    Summary {
        goal: if new.goal.is_empty() { old.goal.clone() } else { new.goal.clone() },
        constraints: merge_items(&old.constraints, &new.constraints),
        progress: merge_items(&old.progress, &new.progress),
        key_decisions: merge_items(&old.key_decisions, &new.key_decisions),
        next_steps: new.next_steps.clone(), // Keep only the latest next steps
        file_tracking: merge_file_tracking(&old.file_tracking, &new.file_tracking),
    }
}

13.6 Session Persistence Format and Migration Chain

The JSON format of the Session file:

{
  "schema_version": 5,
  "session_id": "cc-session-20260715-a3b2c1",
  "created_at": "2026-07-15T10:00:00Z",
  "messages": [
    {
      "role": "user",
      "items": [{"type": "text", "text": "Refactor module A for me"}],
      "id": 1
    },
    {
      "role": "assistant",
      "items": [
        {"type": "text", "text": "Sure, let me analyze it"},
        {"type": "tool_call", "id": "call_1", "name": "read_file", "args": {"path": "src/mod.rs"}}
      ],
      "id": 2
    },
    {
      "role": "tool",
      "tool_call_id": "call_1",
      "items": [{"type": "tool_result", "content": "// module A ..."}],
      "id": 3
    }
  ]
}

The schema_version field is used for migration. When the Session format changes, the system detects the version number at load time and automatically migrates to the new version:

fn migrate_session(session: &mut Session) -> Result<()> {
    match session.schema_version {
        1 => migrate_v1_to_v2(session),
        2 => migrate_v2_to_v3(session),
        3 => migrate_v3_to_v4(session),
        4 => migrate_v4_to_v5(session),
        5 => Ok(()),  // Current version
        _ => Err("unknown schema version"),
    }
}

The migration chain ensures backward compatibility: older Session files are automatically updated to the current version when loaded. Migration is idempotent — if the migration fails, the session file is not corrupted (the system creates a backup before migrating).

13.7 Automatic Decompression

When a session is restored to a larger model window (e.g., switching from a 32K to a 128K window), the system can automatically decompress — restoring placeholder-ized ToolResults from tier-1 to their full content.

fn decompress(compacted: &mut Session, full_records: &Session) -> Result<()> {
    for message in compacted.messages.iter_mut() {
        for item in message.items.iter_mut() {
            if let MessageItem::ToolResult(tr) = item {
                if tr.content.iter().any(|c| matches!(c, ContentItem::Text(t) if t.starts_with("[TRUNCATED:"))) {
                    // Restore from the full record
                    if let Some(original) = find_original(tr.tool_call_id, full_records) {
                        *tr = original.clone();
                    }
                }
            }
        }
    }
    Ok(())
}

Automatic decompression requires that full_records is available — meaning the Session file retains the complete original records. If the Session file itself has also been compacted (extended running causing the Session file to exceed storage limits), decompression may be incomplete.

ADR Deep Reading

From No Compaction to Two-Tier Compaction (ADR 0023)

CodeCoder initially had no compaction mechanism. When the context window filled up, it filled up — behavioral degradation was considered "acceptable."

The first turning point came in headless mode (the following is a pedagogically reorganized account, not a verbatim record): after a headless session ran for a long stretch, its context kept growing. The agent began exhibiting "repeated decision-making" behavior — decisions made a while earlier were being reconsidered. Analysis revealed that the early messages in the context had already fallen outside the model's effective utilization range — the agent was not repeating itself because it "forgot," but because "the earliest decisions were no longer visible." This phenomenon is directionally consistent with published research showing that models utilize information in the middle of long contexts markedly worse than information at the beginning or end (e.g., Lost in the Middle: How Language Models Use Long Contexts, Liu et al., TACL 2023, positional-bias experiments on multi-document question answering and key-value retrieval); this book's case is a pedagogical reorganization and does not claim to share that study's sample.

The first compaction scheme was simple: when the window filled up, discard the earliest messages. But after discarding, the agent lost the context of "why was this decision made" — it knew "what it was doing right now" but not "why it started doing this."

The second scheme (the one ultimately adopted) was two-tier compaction: tier-1 discards low-value content (Reasoning), and tier-2 replaces the earliest messages with structured summaries. The structured summary preserves the "why" information while significantly reducing token usage.

ADR 0023 also recorded the fallback strategy for summary compaction failures: if the tier-2 LLM summary call fails (API timeout, network error, model unavailable), the system falls back to tier-1 compaction and does not attempt tier-2. Falling back is not a failure — per CodeCoder's internal usage observations, tier-1 can still free thirty to fifty percent of tokens, while tier-2 achieves a higher release rate (roughly seventy to eighty percent); tier-1, though less effective than tier-2, is enough to keep the session running.

End of Part 4. The next part enters engineering practice — the daemon-client architecture, observability, and testing strategies.