LibreMail/docs/ci/mergify-integration-spec.md

Mergify integration spec (proposal for issue #407)

Status: PROPOSAL — NOTHING IS ACTIVE. This document and its companion docs/ci/mergify.yml.proposed are a review artifact. No Mergify app is installed and no live .mergify.yml exists in the repo, so Mergify does nothing until a maintainer explicitly adopts it. This spec exists to be reviewed before adoption. It must not weaken the require branches to be up to date invariant — see The require-up-to-date trilemma, which is the central decision this proposal surfaces.

TL;DR / recommendation

• Mergify is free for open source (public repos, unlimited contributors) and the free "Open Source" plan includes the full Merge Queue + batching + speculative (parallel) checks.
• Adopting it lets us retire the entire hand-rolled traffic-controller (autoupdate.yml
  • ci-trigger.yml + the mothballed traffic-control.yml) and the manual serial-bump grind, replacing a "poor-man's merge queue" with a real one.
• Recommendation: adopt Mergify, in two phases.
  • Phase 1 (adopt now, zero impact on the hard rule): a serial queue (batch_size: 1, merge_method: merge). Mergify respects branch protection, keeps require branches up to date literally ON, does the up-to-date-ing + merge itself. This alone kills the manual grind. No throughput multiplier yet.
  • Phase 2 (the batching throughput win — needs one conscious maintainer decision): batch N test-only PRs into one CI run. Batching is incompatible with GitHub's literal require branches up to date checkbox; to get it, the guarantee must move into Mergify's queue (checkbox off, invariant preserved/strengthened by speculative testing) — or history must go linear via fast-forward (which breaks the merge-commit policy). This is a trilemma: require-up-to-date literal × merge-commits × batching — you can keep at most two. Phase 2 should not ship until the maintainer picks which constraint to relax.
• Do I recommend batching? Yes, if the maintainer accepts moving the up-to-date guarantee from GitHub's checkbox into Mergify (Phase 2, merge-batch). That preserves the merge-commit policy and the actual invariant (nothing merges without being tested on top of the latest main), and it is the only path to the #373-style throughput win now that the native merge queue is off the table. If the literal checkbox is non-negotiable and linear history is unacceptable, batching is unreachable and we stop at Phase 1.

---

1. Problem recap

The merge cascade (see the ci-merge-cascade note + issue #349) comes from three things multiplying together:

1. Branch protection require branches to be up to date before merging — a hard maintainer rule that stays. Every PR must be tested against the current tip of main before it merges.
2. Serial merges — merging PR A advances main, which makes every other open PR out-of-date, so each must be re-bumped and re-run before it can merge.
3. A slow, wedge-prone E2E matrix — each PR fans out to API 29–36 + the API 37 preview (~15 min, occasionally wedging on an emulator boot race).

Draining a multi-PR batch is therefore painful: the #373 Robolectric epic was ~6 PRs, each a full E2E run, hand-bumped one at a time. GitHub's native merge queue solves exactly this, but it is org/enterprise-only and JMR-dev is a user account, so it is unavailable.

The repo's current mitigation is a hand-built "poor-man's merge queue":

Piece	File	Role	Status
Auto-update	.github/workflows/autoupdate.yml	rebases behind PRs with GITHUB_TOKEN (no CI re-trigger, by anti-recursion)	active, being mothballed
CI trigger / scheduler	.github/workflows/ci-trigger.yml	re-triggers CI for the top-priority PRs, MAX_INFLIGHT_RUNS: 2	active, being mothballed
Runner priority	.github/workflows/traffic-control.yml	in-run P0-preempt / P1–P9 hold-back ordering	mothballed/disabled
Decision core	.github/scripts/traffic_control.py	pure, unit-tested priority logic	kept (still unit-tested by traffic-control-tests)

Mergify replaces the first three outright. The single required gate — the ci-passed job, status name CI passed — stays exactly as is.

---

2. What Mergify is + free-for-OSS eligibility

Mergify is a third-party GitHub App providing a real merge queue with batching, speculative/parallel checks, auto-update/rebase, and label-driven priority.

• Free for open source. The "Open Source" plan is $0 with unlimited users for public repositories (private teams are free up to 5 active contributors). LibreMail is a public GPL-3.0 repo, so it qualifies. (pricing, marketplace)
• Full product on the free tier. "Every plan includes … Merge Queue, Merge Protections …" — batching and speculative checks are not gated behind a paid tier. (pricing)

---

3. How Mergify's merge queue works

1. A PR is queued (by a pull_request_rules queue action when it is green + approved, or manually via a @mergifyio queue comment).
2. Mergify builds a temporary branch = latest main + the queued PR(s), and runs your CI on that temporary branch — i.e. it tests against an up-to-date base speculatively, without touching the PR's own branch. (merge queue, parallel checks)
3. When the temporary branch's checks pass, Mergify merges the original PR(s).
4. Speculative / parallel checks: Mergify validates several positions of the queue at once — (A), (A+B), (A+B+C) — so it does not pay one-CI-run-per-PR serially. merge_queue.max_parallel_checks bounds the concurrency (our analogue of the old MAX_INFLIGHT_RUNS: 2). (parallel checks)
5. Batching (batch_size > 1): Mergify combines the next N PRs into one temporary batch branch and validates the whole batch with a single CI run — the key win for test-only batches. It groups by priority + similarity (touching-similar-files) so batches are cohesive. (batches)
6. Batch failure → automatic bisection: a red batch is not wholesale dequeued; Mergify splits it (down toward single PRs) to isolate the culprit, dequeues only that PR, and lets the rest proceed — bounded by batch_max_failure_resolution_attempts. (batches)

---

The require-up-to-date trilemma

This is the crux of the whole proposal. Mergify's own docs are explicit:

"Batches require the branch protection setting Require branches to be up to date before merging to be disabled." — Mergify: Merge Queue Batches

The reason: with batching, Mergify tests a temporary batch branch that is up-to-date, then merges the original PRs — whose own branches are not up-to-date per GitHub — so the literal checkbox would block the merge. The documented ways out each cost something:

You want to keep…	…and also…	…then batching is	Cost
require-up-to-date checkbox literally on	merge commits (merge_method: merge)	serial only (no batching)	no throughput win
require-up-to-date checkbox literally on	batching	via queue_branch_merge_method: fast-forward	linear history — breaks the merge-commit policy (discussion #5138)
merge commits and batching	(the throughput win)	yes, merge_method: merge-batch	checkbox must be off; guarantee moves into Mergify

So { require-up-to-date literal · merge-commits · batching } — pick two.

The key reframing: the maintainer rule is really about the invariant — never merge code that wasn't tested against the latest main. Mergify's speculative queue enforces that invariant by construction (it literally builds latest-main + PRs and tests that before merging). Whether the invariant is enforced by GitHub's checkbox or by Mergify's queue, it holds either way. Turning the checkbox off in Phase 2 does not weaken the invariant — arguably it strengthens it (GitHub's checkbox only guarantees the branch was up-to-date at some point and then requires a fresh run; Mergify guarantees the exact merged commit set passed on top of the exact tip). But it does flip a literal setting the rule names, so it is a conscious call for the maintainer, not something this proposal does silently.

Recommended resolution: Phase 1 keeps the checkbox literally on (serial). Phase 2 uses merge_method: merge-batch (one merge commit per batch — preserves the merge-commit policy), turns the checkbox off, and closes the bypass with "block any merge outside Mergify" (below). The fast-forward row is documented for completeness but not recommended — it would silently drop merge commits.

Open item to verify in a trial (do not take on faith from docs): confirm on a throwaway branch that merge-batch lands a real merge commit and that, with the checkbox off + "merge only via Mergify" set, no path exists to merge a stale PR outside the queue.

---

4. Coexistence with the single CI passed gate

• Branch protection keeps requiring the one context CI passed. No change to ci.yml's ci-passed fan-in.
• Mergify injects the repo's required checks as queue conditions automatically (branch_protection_injection_mode, default queue), and the proposed config also names check-success = CI passed in merge_conditions explicitly, so a PR merges only when the same gate branch protection requires is green. (queue action)
• During Phase-1 serial operation Mergify "respects your GitHub branch protections and rulesets" out of the box — including require-up-to-date, which it satisfies by updating the branch itself before merging. (setup)

Gotcha: the condition string must match the check name exactly — CI passed (the name: of the ci-passed job), not ci-passed. A wrong name means PRs queue but never merge (or never queue).

---

5. Priority: mapping the P0–P9 labels

Mergify priority_rules assign a priority from PR conditions; higher merges first (keywords low=1000 / medium=2000 / high=3000, or numeric 1–10000; unmatched → medium). (priority) The proposed config maps P0 → 10000 … P9 → 1000 linearly, so:

• P0 (emergency-only) outranks everything and is picked first for the next batch.
• Unlabelled PRs fall to Mergify's default medium (2000) — the same "no label ⇒ P5-ish default" behaviour traffic_control.py uses today.
• broken/draft PRs are excluded from queueing (they simply shouldn't merge), rather than run at the bottom as the old traffic-controller did.

allow_checks_interruption (default true) reproduces the old P0-preempts behaviour: a higher-priority arrival can interrupt in-flight speculative checks of lower-priority PRs.

---

6. What it replaces

Old piece	Fate under Mergify
autoupdate.yml (branch bumping)	Disable — the queue updates branches itself.
ci-trigger.yml (priority scheduler, MAX_INFLIGHT_RUNS)	Disable — replaced by the queue + max_parallel_checks.
traffic-control.yml (already mothballed)	Delete (or leave disabled).
traffic_control.py + traffic-control-tests job	Keep — harmless, still unit-tested; can be removed later. Note: ci-passed currently lists traffic-control-tests in its needs; leave it or drop it, either is fine.
AUTOUPDATE_TOKEN secret / PAT plumbing	Becomes unnecessary — Mergify acts via its App installation, not a PAT.

Net: a large amount of bespoke CI-orchestration YAML + Python is superseded by one .mergify.yml and the App.

---

7. Interaction with the other CI workstreams

• E2E path-filter (#399 / #402): synergistic. Mergify runs CI on the batch branch, so the path filter evaluates the batch's combined diff: a docs-only batch still skips E2E and merges fast; a test-only batch still triggers the E2E it needs. The one requirement — that a path-skipped run still reports a green CI passed — is already how branch protection is satisfied, and Mergify keys on the same context, so no extra work. (Verify the skip path produces CI passed, not an absent check, or the queue will wait forever.)
• E2E sharding (#372, and the API 37 N=2 shards): orthogonal and compounding — faster CI ⇒ faster batch validation ⇒ shorter queue latency. No config interaction.
• Wedge diagnostics (#404 / #406): still run inside the CI Mergify triggers. Better, checks_timeout on the queue bounds a wedged leg: instead of a hung run blocking the queue, the batch times out and Mergify bisects/re-runs — turning "hand-babysit a wedged emulator" into an automatic dequeue of the offending PR.

---

8. Proposed .mergify.yml

The full proposed config is in docs/ci/mergify.yml.proposed (kept as *.proposed, not a live .mergify.yml, so nothing activates). Outline below.

# NOT ACTIVE — see docs/ci/mergify.yml.proposed for the annotated version.
queue_rules:
  - name: default
    merge_conditions:
      - check-success = CI passed          # the single required gate
      - "#approved-reviews-by >= 1"
      - -draft
      - label != broken
    # Phase 1 (keeps require-up-to-date literally ON): batch_size: 1 + merge_method: merge
    # Phase 2 (batching — checkbox off, guarantee moved into the queue):
    batch_size: { min: 1, max: 5 }         # dynamic; one CI run per batch
    batch_max_wait_time: 5 min
    merge_method: merge-batch              # one MERGE COMMIT per batch (keeps merge policy)
    checks_timeout: 45 min                 # slow/wedge-prone E2E: time out -> bisect

merge_queue:
  max_parallel_checks: 2                   # == old MAX_INFLIGHT_RUNS

priority_rules:                            # P0 -> 10000 ... P9 -> 1000 (higher merges first)
  - { name: p0-emergency, conditions: [label = P0], priority: 10000 }
  # … P1..P9 …

pull_request_rules:
  - name: Queue green, approved, non-draft PRs targeting main
    conditions:
      - base = main
      - -draft
      - label != broken
      - "#approved-reviews-by >= 1"
      - check-success = CI passed
    actions:
      queue: { name: default }

---

9. Setup steps (free-OSS)

1. Install the Mergify GitHub App on JMR-dev/LibreMail from the marketplace (Open Source plan, $0). Requires GitHub admin on the repo. The App requests read/write on pull requests, checks, and contents (it must be able to create temporary branches and merge).
2. Commit .mergify.yml at the repo root — start from docs/ci/mergify.yml.proposed, Phase-1 variant (batch_size: 1, merge_method: merge). Mergify validates the config on push.
3. Keep branch protection require branches up to date + the required CI passed context unchanged (Phase 1). Confirm PRs queue and merge.
4. Phase 2 (only after the maintainer signs off on the trilemma): switch to merge_method: merge-batch + batch_size, turn off GitHub's literal require branches up to date checkbox, and block merges outside Mergify (restrict who can push/merge to main so the queue is the only merge path — Mergify's recommended companion to disabling the checkbox (discussion #5138)).
5. Disable autoupdate.yml and ci-trigger.yml; delete/retire traffic-control.yml.

---

10. Risks & tradeoffs

• Third-party in the merge path. Mergify gets write/merge authority on main. It is an OSS-standard, widely used app, but it is a new external dependency and a supply-chain / availability surface. If Mergify is down, merges pause (fail-safe, not fail-open).
• The require-up-to-date decision (Phase 2). Turning off the literal checkbox is the price of batching. Mitigated by moving the guarantee into the queue and blocking out-of-queue merges — but it is a real, conscious change to a rule the maintainer has pinned, and must be signed off, not assumed.
• Speculative-check CI cost. Parallel checks and batch bisection run CI more: a failed batch can re-run the ~15-min E2E matrix several times while splitting to find the culprit. On a slow/wedge-prone matrix that is a genuine runner-minute cost. Mitigate with a small batch_size cap, batch_max_wait_time, checks_timeout, and modest max_parallel_checks.
• Batching hides which PR broke it (until bisection finishes). A red batch doesn't immediately name the culprit; Mergify bisects automatically, but that costs runs + latency. Best for test-only / low-risk batches (the #373 case); route risky feature PRs through a serial path (batch_size 1) or lower priority so they validate alone.
• Config complexity + a new failure mode. .mergify.yml is another surface. The classic footgun is a condition that doesn't match the real check name (CI passed) — PRs then queue but never merge. Needs a short trial to shake out.
• Loss of bespoke control. The traffic-controller's exact P0-preempt / hold-back semantics are approximated (not byte-identical) by Mergify priorities + allow_checks_interruption. Close enough in practice, but not the same code.

---

11. Recommendation

Adopt Mergify, phased.

• Phase 1 now: serial queue. Immediate elimination of the manual serial-bump grind and the whole traffic-controller apparatus, with zero change to the require-up-to-date rule or the merge-commit policy. Low risk, high daily-quality-of-life win.
• Phase 2 when the maintainer signs off: batching via merge-batch, moving the up-to-date guarantee into the queue (checkbox off, invariant preserved, merges locked to Mergify). This is the only remaining path to the #373-scale throughput win (native queue is unavailable) and it keeps the merge-commit policy. Gate it on the explicit trilemma decision and a throwaway-branch trial that verifies (a) merge-batch really lands merge commits and (b) no stale PR can merge outside the queue.

If the literal checkbox is truly immovable and linear history is unacceptable, stop at Phase 1 — that is still a large win over today.

---

Sources

• Mergify — Merge Queue
• Mergify — Merge Queue Batches
• Mergify — Parallel Checks
• Mergify — Using Queue Rules
• Mergify — Using Priorities
• Mergify — Merge Strategies
• Mergify — Configuration File Format
• Mergify — queue action
• Mergify — Merge Queue setup
• Mergify changelog — new merge-batch merge method
• Mergify discussion #5138 — Merge Queue and branch up-to-date protection
• Mergify pricing · Mergify on GitHub Marketplace
• Repo: .github/workflows/ci.yml, autoupdate.yml, ci-trigger.yml, traffic-control.yml, .github/scripts/traffic_control.py; issue #349 + the ci-merge-cascade note.