"Done" is not for the agent to declare — it is for the system to confirm after a three-tier gate chain.
Pattern Layer
11.1 The Three-Tier Acceptance Gate
An acceptance gate is the mandatory checkpoint a milestone must pass to transition from "in_progress" to "done." CodeCoder defines three tiers of gates, executed in sequence:
Tier 1: Command Gate.
The simplest form of acceptance: the agent calls milestone done with a self-assessment. The system records "the agent says it is done," then proceeds to the next gate.
The trust model of the command gate is "there is a human in the loop." In interactive mode, the user can see the agent's output and judge whether it is actually complete. If the user disagrees, they can send the milestone back to needs_fix.
But the command gate itself performs no verification — it only receives the agent's declaration. Its purpose is to give the acceptance pipeline a "starting point": once the agent declares completion, the system knows "acceptance has begun."
Tier 2: Check Gate.
The check gate introduces deterministic checks independent of the agent. CodeCoder defines four kinds of CheckSpec:
- BuildExitZero: Whether the compile/test command's exit code is 0
- NoTemplateContent: Generated files do not contain template residue ("TODO", "placeholder", "your code here")
- FileCountMin: A sufficient number of files were created
- MinLinesPerFile: Generated code is not a stub
The defining characteristic of the check gate is determinism. The exit code either is 0 or is not; a file either exists or does not; template residue either exists or does not. No LLM judgment, no gray area of "looks reasonable."
Tier 3: Review Gate.
The review gate introduces an independent read-only sub-agent that evaluates output quality using a structured rubric. The sub-agent reads files, reads diffs, reads code structure, then returns a structured Verdict.
The review gate handles what the check gate cannot: architectural soundness, over-engineering, terminology consistency. These questions have no binary "right/wrong" answers, but they can be transformed into comparable signals through a structured rubric.
The three-tier gates execute in sequence:
milestone done (command gate)
|
CheckSpec checks (check gate) — fail -> needs_fix
|
Review sub-agent evaluation (review gate) — fail -> needs_fix
|
Milestone status: done
If any gate fails, the milestone enters the needs_fix state and does not automatically revert to pending. needs_fix is an explicit "failed + needs modification" state — not "back to square one."
11.2 Objective vs Subjective
The boundary between the check gate and the review gate is "objective check vs subjective judgment."
The check gate handles objective checks. These checks are characterized by: unambiguous results. Did compilation pass? Is the exit code 0 or non-zero? Is there template residue? Did grep "TODO" return results? Are there enough files? Is count >= threshold?
The benefit of objective checks is that they are automatable and reproducible. The same check produces identical results whether run in CI or locally. If the agent claims "compilation passed," the check gate can verify by re-running the compile command — no need to trust the agent's output.
The review gate handles subjective judgment. These judgments are characterized by: no single correct answer, but structure reduces arbitrariness. The four-signal rubric (foundation, over_engineering, volume, terminology) was defined in Chapter 5, Section 5.5. This chapter only expands on the scoring criteria and merge rules.
- "Is this module's interface design over-abstracted?" — different people may judge differently
- "Is the new code's terminology consistent with the project?" — requires understanding project context to judge
The review gate uses a rubric to transform subjective judgments into comparable signals. Four signals (foundation, over_engineering, volume, terminology) each have scoring criteria, and the LLM scores according to those criteria. The four signals are merged into a single Verdict. This process still involves LLM judgment (non-deterministic), but it is far more controlled than an open-ended question like "what do you think of this code?"
11.3 Drift Signals
Acceptance gates address quality at the point "when something is done." But architectural drift — code gradually deviating from the original design — is often cumulative rather than the result of a single change.
Among the four signals in the rubric, the terminology signal is specifically designed to detect terminology drift: does the new code use terms prohibited by CONTEXT.md? The over_engineering signal detects design drift: were unnecessary abstractions introduced?
Acceptance gates are not the only tool for drift detection, but they are the last line of defense. If earlier code reviews and design discussions missed the issue, acceptance gates can at least catch architectural drift at the point of completion.
Case Layer
11.4 The Three-Tier Acceptance Pipeline
The complete execution flow of the three-tier acceptance gates is as follows:
graph LR
subgraph "Milestone State: in_progress"
AGENT[Agent executing task]
end
subgraph "Level 1: Command Gate"
REPORT[agent calls<br/>milestone done]
NOTE[attach self-assessment]
end
subgraph "Level 2: Check Gate"
BUILD[BuildExitZero<br/>build exit code = 0]
TEMPL[NoTemplateContent<br/>no template residue]
FILE[FileCountMin<br/>file count meets threshold]
LINES[MinLinesPerFile<br/>lines of code meets threshold]
end
subgraph "Level 3: Review Gate"
REVIEW[read-only sub-agent<br/>architecture review]
SIGNAL[four-signal rubric<br/>foundation<br/>over_engineering<br/>volume<br/>terminology]
end
subgraph "Acceptance Result"
PASS[Verdict::Pass<br/>→ done]
NFIX[Verdict::NeedsFix<br/>→ fix]
RB[Verdict::Rebuild<br/>→ rebuild]
end
AGENT --> REPORT
REPORT --> BUILD
BUILD --> TEMPL
TEMPL --> FILE
FILE --> LINES
LINES -->|all checks pass| REVIEW
LINES -->|any check fails| NFIX
REVIEW --> SIGNAL
SIGNAL -->|all normal| PASS
SIGNAL -->|foundation < 0.3| RB
SIGNAL -->|other signals abnormal| NFIX
PASS -->|state: done| FINAL([Milestone Complete])
NFIX -->|self-recovery loop| AGENT
RB -->|human intervention| FINAL
style PASS fill:#e8f5e9
style NFIX fill:#fff3e0
style RB fill:#fce4ec
The complete implementation of the acceptance pipeline:
fn verify_milestone(milestone: &Milestone, context: &Context) -> Result<Verdict> {
// 1. Command gate — agent self-reports completion
// milestone.done() was already called externally, no additional handling here
// 2. Check gate — deterministic checks
if let Some(specs) = &milestone.check_specs {
for spec in specs {
let result = match spec {
CheckSpec::BuildExitZero { command } => {
let output = context.run_command(command)?;
if !output.status.success() {
return Ok(Verdict::NeedsFix {
reason: format!("build failed: {}", output.stderr),
});
}
}
CheckSpec::NoTemplateContent { patterns } => {
for pattern in patterns {
let matches = context.grep(pattern)?;
if !matches.is_empty() {
return Ok(Verdict::NeedsFix {
reason: format!("template content found: {:?}", matches),
});
}
}
}
CheckSpec::FileCountMin { min } => {
let files = context.list_created_files()?;
if files.len() < *min {
return Ok(Verdict::NeedsFix {
reason: format!("expected {} files, got {}", min, files.len()),
});
}
}
CheckSpec::MinLinesPerFile { min } => {
for file in context.list_created_files()? {
let lines = context.count_lines(&file)?;
if lines < *min {
return Ok(Verdict::NeedsFix {
reason: format!("{}: {} lines, expected {}", file, lines, min),
});
}
}
}
};
}
}
// 3. Review gate — architecture-level review
if let Some(review_config) = &milestone.review_config {
let verdict = context.run_review_agent(review_config)?;
if !matches!(verdict, Verdict::Pass) {
return Ok(verdict);
}
}
Ok(Verdict::Pass)
}
Key design points of the pipeline:
- Both the check gate and review gate are optional (controlled by the milestone's
check_specsandreview_configfields). Not every milestone needs all three tiers — most milestones only require the command gate plus the check gate - The four check types in the check gate are parallel (all checks must pass), not sequential (one must pass before the next runs)
- The review gate only executes when the milestone has a
review_configconfigured — it is off by default
11.5 CheckSpec: Scoring Criteria for the Four Checks
BuildExitZero:
fn check_build_exit_zero(command: &str) -> CheckResult {
let output = run_command(command);
if output.status.success() {
CheckResult::Pass
} else {
CheckResult::Fail(format!(
"exit code: {}, stderr: {}",
output.status.code().unwrap_or(-1),
output.stderr
))
}
}
BuildExitZero is the most commonly used check — almost every milestone should have it configured. It verifies that the agent's output is at least "compilable."
NoTemplateContent:
fn check_no_template_content(patterns: &[&str]) -> CheckResult {
let files = list_created_files();
for file in files {
for pattern in patterns {
if grep_file(&file, pattern).is_some() {
return CheckResult::Fail(format!("{} contains '{}'", file, pattern));
}
}
}
CheckResult::Pass
}
The default patterns are ["TODO", "FIXME", "placeholder", "your code here", "implement me"]. These are the most common template residues left behind in agent-generated code.
FileCountMin / MinLinesPerFile:
These two checks prevent the agent from generating stub code (files that exist but have empty or near-empty content). FileCountMin ensures the agent created a sufficient number of files, while MinLinesPerFile ensures each file is not a stub.
fn check_min_lines_per_file(min_lines: usize) -> CheckResult {
let files = list_created_files();
for file in files {
let count = count_lines(&file);
if count < min_lines {
return CheckResult::Fail(format!(
"{} has {} lines, minimum {}",
file, count, min_lines
));
}
}
CheckResult::Pass
}
11.6 Review Verdict: Four-Signal Rubric
The review gate's scoring criteria (scoring each signal from 0.0 to 1.0). The ReviewSignals struct (with four fields: foundation, over_engineering, volume, terminology) was defined in Chapter 5, Section 5.5. This section provides the detailed scoring criteria for each signal:
foundation (Structural Integrity):
- 1.0: Module structure is complete, type declarations are comprehensive, public interfaces are clear
- 0.7: Structure is mostly complete with minor omissions that can be repaired
- 0.3: Structure is missing critical modules or types
- 0.0: Nearly unusable
over_engineering:
- 1.0: Completely unnecessary abstraction, or generality implemented that was not requested
- 0.7: Abstraction level is slightly higher than appropriate but still acceptable
- 0.3: Design is reasonable, no obvious over-engineering
- 0.0: Design is exactly right
volume (Scope of Change):
- 1.0: Single change far exceeds the task description (may have done things that were not requested)
- 0.7: Scope is slightly larger than needed, but mostly relevant
- 0.3: Scope is reasonable
- 0.0: Scope is too small, task not completed
terminology (Terminology Consistency):
- 1.0: Used terms prohibited by CONTEXT.md, or created new terms that conflict with existing ones
- 0.7: Minor inconsistencies in terminology usage
- 0.3: Terminology usage is consistent
- 0.0: Terminology usage is flawless
Verdict merge rules:
fn merge_signals(signals: &ReviewSignals) -> Verdict {
if signals.foundation < 0.3 {
return Verdict::Rebuild;
}
if signals.foundation < 0.5 || signals.over_engineering > 0.7
|| signals.volume > 0.7 || signals.terminology > 0.7 {
return Verdict::NeedsFix;
}
Verdict::Pass
}
Rebuild is more severe than NeedsFix — it indicates that "the foundation is flawed, and fixing costs nearly as much as rewriting."
11.7 needs_fix -> Self-Recovery Loop
When an acceptance gate fails, the milestone enters the needs_fix state. If this happens in headless mode (no user present), the system initiates a self-recovery loop (the design evolution of this loop is documented in Chapter 12's ADR Deep Reading):
fn needs_fix_recovery(milestone: &mut Milestone, context: &Context) -> Result<()> {
let max_attempts = context.config.bg_max_fix_attempts; // Default 3
let mut attempts = 0;
while attempts < max_attempts {
// 1. Inject the failure reason into the fix prompt
let fix_prompt = format!(
"Milestone '{}' acceptance failed. Reason: {}. Please fix these issues.",
milestone.name,
milestone.fix_reason
);
// 2. Re-execute the milestone
context.execute_fix(fix_prompt)?;
// 3. Re-accept
let verdict = verify_milestone(milestone, context)?;
if matches!(verdict, Verdict::Pass) {
milestone.status = MilestoneStatus::Done;
return Ok(());
}
// 4. Update the failure reason
milestone.fix_reason = format!("{}. Retry {} failed: {}",
milestone.fix_reason, attempts + 1, verdict.reason());
attempts += 1;
}
// 5. Budget exhausted, report stuck
Err(Error::StuckNeedsFix {
milestone: milestone.name.clone(),
reason: milestone.fix_reason.clone(),
attempts,
})
}
Key design points of the self-recovery loop:
- Bounded retries: Default 3 attempts (
bg_max_fix_attemptsis configurable; 0 disables auto-retry). Milestones that still fail after exhausting the budget reportStuckNeedsFixand wait for human intervention - Cumulative failure reasons: The failure reason from each retry is appended to
fix_reason— the agent can read the complete failure history when attempting the next fix - Retries do not count against max_auto:
max_autotracks "milestones autonomously completed by the agent"; retries in the self-recovery loop do not consume this budget
11.8 The Evolution from Self-Report to Objective Gate Coverage
In early versions of CodeCoder, milestone completion relied entirely on the agent's self-report (the command gate). The agent said "done" and it was done.
The problem surfaced in headless mode: the agent reported completion, but compilation failed. The agent had reported done without actually running the compile command — because its acceptance criterion was "file was modified," not "file was modified and compilation passed."
The solution was the introduction of the check gate (BuildExitZero check). The check gate executes after the command gate, overriding the agent's self-report. If the agent says "done" but compilation fails, the check gate sends the milestone back to needs_fix.
This is a microcosm of CodeCoder's acceptance system evolution: from trusting the agent's self-report, to covering it with deterministic checks, to covering those checks with structured review. Each step was taken because the previous gate exposed a blind spot during actual execution.
ADR Deep Reading
The Introduction of Review Gates and Self-Recovery (ADR 0039)
ADR 0039 documents the simultaneous introduction of the review gate and the self-recovery loop.
Before their introduction, headless mode acceptance relied entirely on the command gate plus the check gate. The agent self-reported completion, the check gate verified compilation and file counts, and if those passed, the milestone was done. The problem: the agent could pass the check gate but have architectural issues. Compilation passed but module dependency direction was reversed. File count was sufficient but all were stubs. Terminology was consistent but logic was wrong.
The review gate introduced an independent LLM call to evaluate architecture quality. But the review agent is also LLM-driven, and its judgment can also be wrong. To prevent the review agent's misjudgment from blocking the workflow, the review gate's Verdict was designed to be overridable by the check gate — if the review agent says "needs_fix" but the check gate says "all passed," the system trusts the check gate (because it is deterministic).
The self-recovery loop was introduced for another problem: the agent did not know what to do next in the needs_fix state. In early versions, once a milestone entered needs_fix, the agent needed the user to manually set it back to pending or in_progress. In headless mode, there is no user to perform this operation. The self-recovery loop solves this by injecting the failure reason into a fix prompt, allowing the agent to automatically re-execute.
Next chapter enters headless autonomous operation — complete engineering practices for the no-user mode.