The Adjudicator
A read-only referee for AI code review. It reads other agents' critiques and marks each one keep, drop, or recalibrate, judged against this project's own standards, not generic best practice. It reads every file and changes none.
Still a work in progress. The line we're least sure of is how hard to default to a drop: too eager and a real concern slips through, too soft and the noise wins. Tell us where that line sits. Reply on LinkedIn.
How to read this map
Pick a stage to walk, or start at the top
The walkthrough, as text
Every stage of the interactive walkthrough above, as plain readable text. Each stage has its own link.
Stage 1Read the inputsPlan, findings, project rules
Read · inputs
Before judging anything, the referee reads three things: the plan being critiqued, the critics' findings, and the project's own rule files, because it judges against this project's standards, not generic best practice.
Read the locked plan, the critic findings, and the project rule files. The rubric is the project's own docs: CLAUDE.md, the style guide, STACK.
What happens
The Adjudicator is handed three inputs. The locked plan: to know the declared scope. The critic findings: structured markdown, one finding per line: a file:line citation, a severity, the suggestion, and its rationale. And the project rule files it can find in the working directory: CLAUDE.md, docs/STYLE_GUIDE.md, STACK.md, plus any doc a critic explicitly cites.
Those rule files are the authoritative rubric for this project: a generic best practice that contradicts a project standard loses. In the pipeline the findings arrive in a fixed shape, a PLAN: path and a CRITICS: block, which the referee concatenates and numbers 1..N in the order received.
Who acts
- Read, tool: Opens the locked plan and the project rule files: CLAUDE.md, the style guide, STACK.
- critic agents, agent: Upstream: their structured findings are the input the referee judges. Named for context; not run here.
The rules
- Project docs are the rubric.
CLAUDE.md,docs/STYLE_GUIDE.md,STACK.md: a generic best practice that contradicts them loses. - One finding per line. Each critic finding carries a
file:line, a severity, the suggestion, and a rationale. - Number them 1..N. Findings across critics are concatenated and numbered in the order received, one verdict block each.
- Read-only from the start. The referee's tools are
Read,Grep,Glob. It can open any file and change none.
In: The locked plan, the critics' findings, and the project rule files. Out: A numbered list of findings, each ready to judge against the project's rubric.
Stage 2The signal gateDoes it carry any signal?
Gate · parse
Each finding clears one bar before it's judged: does it carry any real signal at all? No file reference, no concrete failure, no actionable fix: that's noise, and it's dropped.
A finding needs at least one of three signals: a file reference, a concrete failure mode, an actionable suggestion. Zero signals is unparseable.
What happens
The gate runs first, as a filter. Three signals matter: a file reference, a concrete failure mode, and an actionable suggestion. A finding missing all three is unparseable: DITCHed with a reason that names what was missing. Anything with even one signal proceeds to the real rubric.
There's a batch case. If every finding misses all three signals, the referee doesn't grind out one DITCH per line. It returns a single MALFORMED INPUT verdict, names what was missing across the batch, and recommends re-running the critic. That block fires only when 100% of findings fail the gate; one good signal anywhere and the standard rubric applies to all.
Who acts
- Read, tool: Scans each finding for the three signals before any substantive judgment.
The rules
- One signal is enough. A file reference OR a concrete failure OR an actionable fix, any one passes the gate into the rubric.
- Zero signals is unparseable. Missing all three → DITCH, with a reason naming what was absent.
- All-malformed is one verdict. If 100% of findings miss every signal → a single
MALFORMED INPUTblock, not one DITCH each. - Not the same as zero findings.
MALFORMED INPUTmeans the critic returned unparseable content; a clean plan with no findings isNO CRITIQUES TO ADJUDICATEinstead.
In: The numbered findings from Stage 1. Out: Findings with at least one signal, passed to the rubric, or a malformed-input halt.
Stage 3The rubric: first match winsKEEP, DITCH, or MODIFY
Judge · rubric
The core: each finding runs down a fixed table of tests, top to bottom, and the first one it matches decides the verdict: keep it, drop it, or recalibrate its weight.
Run each finding down the rubric top-to-bottom; the first row it matches sets the verdict: KEEP, DITCH, or MODIFY.
What happens
The rubric is a decision table, applied top to bottom, first match wins. A finding that cites file:line and names a concrete failure mode is a KEEP, as is a style critique that cites the project's actual style guide. Vague 'could be cleaner' with no citation is a DITCH, and so is a finding outside the plan's declared scope that isn't a security issue or a blocker.
Several DITCH rows encode the project's own discipline: demanding an abstraction at fewer than two callers (YAGNI), treating within-file repetition as a cross-file extraction, a defensive check for an impossible condition, a comment that just restates code, or a style note appealing to generic 'best practice' over the project's docs. When the claimed severity is simply out of proportion to the failure mode, the verdict is MODIFY: agree with the finding, recalibrate the weight.
Who acts
- the-adjudicator, agent: The read-only referee running the rubric, the same agent serves Stage 3 and Stage 7 of /change-heavy.
The rules
- Cite + concrete = KEEP. A
file:linecitation plus a concrete failure mode passes. Evidence is the price of a KEEP. - Vibes = DITCH. "Unclear" or "could be cleaner" with no citation fails the evidence test.
- Out of scope = DITCH. Outside the plan's declared scope, not a security issue, not a blocker → a follow-up, not this diff.
- Project standards beat generic advice. A style note citing the project's style guide is a KEEP; one appealing to generic 'best practice' is a DITCH.
In: The findings that carried a signal. Out: A first-pass verdict per finding: KEEP, DITCH, or MODIFY.
Stage 4The citation-existence checkDoes the cited line exist?
Verify · citation
A KEEP isn't final yet. A citation proves a finding is well-shaped, not that it's real, so before trusting one, the referee opens the cited line and checks the thing it describes is actually there.
Before any KEEP or upgrade resolves, open the cited file:line and confirm the named construct exists. If it doesn't, flag it, never silently drop it.
What happens
A file:line proves the critic's shape, not that the finding is real. So before any verdict resolves to KEEP or a severity upgrade, the referee reads, or greps, the cited location and confirms the named construct is there: the function, the query, the selector the critique describes. It runs lazily, only on findings already headed for KEEP or upgrade; a DITCH needs no verification.
If the citation doesn't verify, line absent, or present but missing the described construct, the referee keeps the substantive verdict and appends a CITATION-UNVERIFIED flag. It does not drop the finding. That's the safe asymmetry: a real-but-miscited security bug is worse to lose than a phantom is to surface, so the human decides. (An earlier version dropped unverifiable citations; the current one flags and surfaces.)
Who acts
- Read, tool: Opens the cited file:line to confirm the described construct is actually there.
- Grep, tool: Finds the named construct when the exact line has drifted.
The rules
- A citation is shape, not proof. It shows the finding is well-formed, not that the code it describes exists.
- Verify lazily. Only findings headed for KEEP or an upgrade get checked; a DITCH is already settled.
- Unverified is flagged, never dropped. Line absent or construct missing → keep the verdict, append
CITATION-UNVERIFIEDfor the human to confirm. - Existence, not correctness. The check is that the construct is present, not a re-review of whether it's buggy, and not SHA-anchoring.
In: The KEEP and upgrade candidates from the rubric. Out: Verified verdicts, plus any surfaced CITATION-UNVERIFIED flags.
Stage 5Severity recalibrationAsymmetric: one way up
Weigh · severity
The referee can adjust how heavy a finding is rated, but not symmetrically. It dials down anything over-claimed, and dials up only one thing: a real security or correctness bug.
Downgrade any over-claimed severity; upgrade only security or correctness findings. Any change makes the verdict MODIFY.
What happens
Severity moves are deliberately asymmetric. Downgrades are fair game for any finding whose claimed weight exceeds its actual failure mode: a cosmetic rename billed 'must-fix' drops to 'consider'. Upgrades are allowed in exactly one case: a security or correctness finding the critic under-claimed: an SQL injection filed as 'minor' becomes 'must-fix'. That's the only direction severity ever increases.
The encoding is strict: any time the adjudicated severity differs from the original, the verdict is MODIFY, never a KEEP with a quiet severity change. KEEP means 'I agree with the finding and its weight'; MODIFY means 'I agree with the finding, I recalibrated the weight.' Downstream tooling reads the verdict label, so the distinction has to hold.
Who acts
- the-adjudicator, agent: Applies the asymmetric severity floor: down for over-claims, up only for security or correctness.
The rules
- Downgrades: any over-claim. Claimed severity above the real failure mode → dial it down.
- Upgrades: security or correctness only. The one direction severity rises. A real security bug under-claimed is floored upward.
- A severity change is always MODIFY. Never a KEEP with a silent re-weight: the verdict label is load-bearing for tooling.
- Never drop a real bug. A finding naming a real security or correctness failure is never DITCHed: KEEP or upgrade, even if written badly.
In: The verified verdicts. Out: Verdicts with severity floored: down for over-claims, up for under-claimed security.
Stage 6Anti-yes-man & the steelman gateArgue the other side, once
Guard · steelman
Two guards against being either a pushover or a hangman. Default to dropping vague non-security findings, but before dropping one on a judgment call, argue the strongest case for keeping it, in a single sentence.
Default DITCH for ambiguous non-security findings, but steelman the two highest-risk drops in one sentence each before committing.
What happens
The anti-yes-man rule sets the default: for an ambiguous non-security finding with no concrete failure mode, the default is DITCH. A false KEEP wastes attention; a false DITCH of a real bug misses it, but the referee's whole value is filtering, not validating, so it doesn't preserve critiques out of politeness.
The steelman gate is the counterweight, applied in exactly two spots: a DITCH on scope, YAGNI, or generic best practice, where the critique is concrete but rejected on other grounds, and every missed concern the referee raises. In each, it writes one sentence for the strongest opposing reading first; if that steelman names a real failure mode, the DITCH escalates to KEEP or an upgrade. One sentence, no hedging, and forbidden on clear-cut rows, so the output stays lean.
Who acts
- the-adjudicator, agent: Runs the anti-yes-man default and the selective steelman before committing each high-risk verdict.
The rules
- Default DITCH the ambiguous. No concrete failure mode and non-security → drop it. Filtering is the job, not validating.
- Steelman the risky drops. One sentence for the strongest opposing case on any DITCH-for-scope / YAGNI / generic-best-practice.
- A strong steelman flips it. If the opposing case names a real failure mode, the DITCH becomes a KEEP or an upgrade.
- One sentence, or none. Steelman is forbidden on clear-cut rows: no arguing both sides indefinitely.
In: The severity-floored verdicts. Out: Committed verdicts, each risky drop tested against its strongest counter-argument.
Stage 7Render verdicts & summaryThe verdict block, no essay
Render · verdicts
Finally the referee prints the result in one fixed format: a short block per finding, then a summary of whether the critics ran hot or cold and what they missed. No preamble, no conclusion essay.
Emit one terse block per finding: verdict, severity, reasoning, then a summary: critic skew, up to three missed concerns, and the net actionable count.
What happens
The output is fixed and terse. One block per finding: the VERDICT (KEEP / DITCH / MODIFY), the SEVERITY as original → adjudicated, a one-sentence REASONING naming the rubric row that applied, and, only where the rule requires it, a one-sentence STEELMAN. No prose preamble, no conclusion.
Then a SUMMARY: whether the critics were balanced, over-critical, or under-critical, with one sentence of evidence; up to three missed concerns as one-liners, each with its own steelman-omission line; and the net actionable count. A critic that returned nothing is NO CRITIQUES TO ADJUDICATE; one that returned unparseable noise is MALFORMED INPUT: two different empty states, never conflated.
Who acts
- the-adjudicator, agent: Emits the fixed verdict-block-plus-summary format the orchestrator presents verbatim.
The rules
- One block per finding.
VERDICT · SEVERITY · REASONING, plusSTEELMANonly where mandated. No essay. - Skew, in one line. Balanced / over-critical / under-critical, with a sentence of evidence for the label.
- Missed concerns, max three. One-liners, observations not recommendations. Wanting more than three means the critic was under-critical. Flag that instead.
- Net actionable count. The bottom line: how many critiques survived the filter and are worth acting on.
In: The committed verdicts. Out: A verdict block per finding plus a skew-and-missed summary, presented verbatim.
Stage ✓Signal, filtered from noiseJudged, nothing written
Outcome
The run ends with every critique marked and weighed, the noise cut, and not one line of code or plan changed. The referee decided what's worth your attention. It never touched the work itself.
Every finding marked KEEP / DITCH / MODIFY against the project's own standards, read-only, terse, and biased toward cutting noise.
What happens
What you're left with is a filtered review: each finding marked KEEP, DITCH, or MODIFY against this project's actual standards, real security and correctness findings protected by a one-way floor, and a summary of what the critics missed. The forbidden actions keep it honest: it generates no new critiques beyond three missed one-liners, rewrites no plans, suggests no alternatives, and doesn't re-litigate a design decision already locked in the plan.
The whole run is read-only: Read, Grep, Glob, nothing that writes. Its bias is deliberate: default to DITCH when a non-security finding is ambiguous, because the referee's value is in what it removes. Signal out of noise, and the human still makes the call.
Who acts
- the-adjudicator, agent: A read-only judge: its only product is the verdict, never an edit.
The rules
- Read-only, always.
Read,Grep,Glob: no write path exists. The verdict is text, not a change. - It filters, it doesn't author. No new critiques past three missed one-liners, no plan rewrites, no alternative approaches.
- Locked decisions stay locked. A design the user already chose in the plan is not re-argued.
- Biased toward cutting. In doubt on a non-security finding, DITCH: the value is in the noise removed.
In: The critics' raw findings. Out: A filtered, weighted verdict set: signal the human can act on, zero files touched.