Skip to content

A gate ladder for auto-merging generated changes

Problem

A pipeline opens merge requests automatically. Left for humans, they queue and go stale; the pipeline that was supposed to save effort now generates a backlog. Merged unconditionally, an unreviewed generation reaches the default branch.

The question is not whether to auto-merge but what must be true first — and a single confidence score does not answer it, because the ways a generated change can be unsafe are not one dimension.

Technique

Four sequential gates, cheapest first, each disqualifying:

  1. Size cap. Above a diff-line threshold, stop and leave it for a human. A large generated diff is not more valuable; it is harder to review and more likely to contain something unintended.
  2. CI. Poll until the pipeline passes, fails, or times out. Distinguish all three — a timeout is not a pass.
  3. Reviewer. A read-only agent evaluating the diff against named, enumerated criteria, returning a structured verdict with a reason.
  4. Merge or comment. Approved, squash-merge. Rejected, post the reason on the MR and leave it. Never close it silently.

Two properties matter more than the ladder's shape.

The reviewer is read-only, with no tools. It cannot fix what it finds. A reviewer that can edit will edit rather than reject, and the gate becomes a second author.

"When in doubt, reject" is explicit in its instructions. Without it, an agent asked to judge borderline output approves, because approving reads as helpful.

When it applies

High-volume, low-individual-risk generated changes with a cheap revert path — documentation, generated reference, changelogs. It earns its keep where the review burden is the actual bottleneck.

When it does NOT apply

Anything where a bad merge is expensive or hard to reverse: application code, schema migrations, security configuration, infrastructure. The size cap does not make those safe; it makes small unsafe changes eligible.

It also assumes CI means something. Where the pipeline does not meaningfully exercise the change, gate 2 is theatre — and worse than absent, because it looks like coverage.

Evidence

A documentation pipeline ran this ladder over auto-generated updates. It works, and its most instructive failure is worth recording.

The reviewer's criteria included "content is not truncated mid-sentence". A document was merged that ended mid-sentence. The criterion was correct, present, and did not fire.

Two consequences. The document was also orphaned from the site navigation, so the strict site build never rendered it and never failed — the mechanical check that would have caught it was not looking. And a judgement-based gate needs its own evidence: this one had been trusted for months on the strength of its criteria list rather than on any measurement of whether it applied them.

Mechanical checks belong at the write path where they are deterministic. The reviewer should be the last gate, not the only one.

See also