Four reviews of one release

A recent Flight Deck release added a fifth skill, for repository, branch and worktree safety, a fourth optional hook, and new rules across three existing skills. The package's house rule is that every rule in it has to trace to a real miss that actually happened; generic best-practice advice gets deleted on sight. That rule applies to the package itself, and this time it got applied hard: before the release shipped, the work went through four adversarial reviews.

This is not a promotional recap. It is a report on what each of those four reads caught that the read before it structurally could not, and what running four of them actually cost.

The plan, and what the design review cut

The first review ran on the design, before a line of code existed. Its verdict: do not build this as designed.

Nine rules were proposed for the release. The review cut four of them. Two had no incident behind them, generic advice dressed up as doctrine, which is exactly what the package's own rule exists to catch. The other two restated rules that already lived elsewhere in the package, adding a second copy instead of a second idea.

It also killed two entire work items outright, not just individual rules within them. The first was a plan to consolidate five scattered fragments of one doctrine into a single owner. Its premise was wrong: three of the five fragments were not stating the same doctrine at all, and one of them said the opposite thing outright. Consolidating them under one owner would not have unified anything.

The second work item was a section written into a skill whose trigger fires only on code changes. The artifact the section was meant to govern was a written document, not code, so it would never have fired in the one place it needed to.

Both cuts landed before any implementation cost was paid. That is the specific value of a review that runs on the design rather than the result.

What one lens sees that the other cannot

The second and third reviews ran in parallel on the built result, with deliberately different lenses: one reading for doctrine, one running the new hook mechanically.

The doctrine lens found that one piece of doctrine was now stated in three separate files with no cross-reference between them. That is the exact defect a previous release of the same package had been spent undoing, reintroduced by this one.

The mechanical lens ran 44 probes against the new hook and found a real false negative: a flag abutting a closing quote, the kind that shows up in a command wrapped as bash -c "...", never fired the reminder it was supposed to. The script's own comment documented a false-positive risk that did not reproduce in the probes, and said nothing about the false negative that did.

Neither reviewer could have found the other's catch. The doctrine reviewer was not running probes against a hook script. The mechanical reviewer was not reading a design document for anything resembling confidentiality. Running the two in parallel, with different questions asked of the same result, is what caught both.

The finding nobody was looking for

The doctrine lens's other catch is the one I keep coming back to, because of how it happened.

A design document written during the work quoted identifiers from private, unrelated work, in a repository whose remote is public. The leak came from doing something correct. The package's own rule says every rule has to trace to a real miss that actually happened, so past incidents get mined as evidence when a new rule is drafted, and the identifiers rode along with the incident they were describing.

An adversarial design review had already read that document in full, and an editing pass had rewritten parts of it afterwards. Neither surfaced the identifiers, because neither was looking for them. The pass reading specifically for confidentiality is the one that caught it. Nothing had been committed, and the repository's history was verified clean before anything else happened.

When the higher-tier reviewer was wrong

The fourth review, a final top-level pass, flagged a version claim in the release notes as uncorroborated, because the upstream changelog did not record it. That observation was accurate, as far as it went.

But the official documentation stated the claim verbatim. Checking further, the changelog turned out not to mention the feature anywhere at all, not even the part of the claim the reviewer had already accepted as fine. It could not arbitrate either half of the claim, because it was silent on the whole thing.

The finding was rejected, with the source cited.

What it costs, honestly

Four reviews, roughly a dozen findings acted on across all of them. The reviews cost more than the implementation did.

There is a diminishing return visible in the sequence: the first review rewrote the plan, and the fourth found two counting errors in the release notes. One of those errors described a test suite as having 30 cases when it had 37, which is the exact defect class the release's own new rule describes, and it was about to ship in the same commit as the rule that names it.

Three of the defects across the whole process came from how the work was split, not from the work itself. It was divided among subagents by file, so no two agents ever wrote the same file, which is the correct way to split it. Each time, a single concern crossed the split anyway:

• a count stated in one file and owned by another
• an enumeration repeated in four places, where a corrective pass fixed three of them
• a dependency one agent added to a script, while another agent wrote a comment asserting the reduced environment that script runs in was sufficient

I misread a command's exit code three times during the session, and once accused a piece of work that was actually correct, on the basis of a test fixture I had built wrong myself.

The rule that came out of all of this now ships in the package: when work is split by file, the orchestrator names the cross-cutting invariants up front and assigns each one to a specific agent's done criteria, or checks them itself at the end, because no agent can see them by construction. A self-contained brief is exactly what hides the other agents from each other.

And a brief that lists the known occurrences of a problem teaches the agent to stop at that list. It has to be told to find every occurrence itself.