test(coverage): lane 6 — coverage ratchet gate (enforce >=95% in CI) #251

Closed
opened 2026-07-03 17:58:01 +00:00 by JMR-dev · 2 comments
JMR-dev commented 2026-07-03 17:58:01 +00:00 (Migrated from github.com)

Strangler capstone (lane 6 of 6) — the ratchet that makes the whole push safe. Depends on the JaCoCo report (#192 / PR #241); its final threshold lands as lanes 1–5 complete.

Scope: build + CI. Add jacocoTestCoverageVerification with a minimum-coverage rule, wired into the CI coverage job so a PR fails when it drops below the floor.

Strangler mechanism: start the floor at the current baseline (~38% line per PR #241) and ratchet it upward as each lane lands — prefer per-package minimums so covered packages can't regress while others catch up — until the overall floor is ≥95%. Document the ratchet in CLAUDE.md.

Target: enforced ≥95% overall in CI, with no regression permitted below each package's floor.

**Strangler capstone (lane 6 of 6)** — the ratchet that makes the whole push safe. Depends on the JaCoCo report (#192 / PR #241); its final threshold lands as lanes 1–5 complete. **Scope:** build + CI. Add `jacocoTestCoverageVerification` with a minimum-coverage rule, wired into the CI coverage job so a PR **fails** when it drops below the floor. **Strangler mechanism:** start the floor at the current baseline (~38% line per PR #241) and **ratchet it upward** as each lane lands — prefer per-package minimums so covered packages can't regress while others catch up — until the overall floor is **≥95%**. Document the ratchet in CLAUDE.md. **Target:** enforced ≥95% overall in CI, with no regression permitted below each package's floor.
JMR-dev commented 2026-07-03 19:01:15 +00:00 (Migrated from github.com)

Heads-up from lane 1 (#246 / PR #254) for the ratchet design:

Instruction coverage of data/repository can't reach 95% by tests alone — ~82.8% of that package's missed instructions are Kotlin-coroutine synthetics: Flow.map collector continuations (…$$inlined$map$1$1) and suspend-lambda state machines (…$openMessage$2), whose suspend/resume dispatch branches don't execute under synchronous test mocks (known JaCoCo × coroutines limitation). Only ~4 lines in MailRepositoryImpl are genuinely unhit; line coverage is already ≥98%.

For the capstone, the ratchet should either (a) exclude these synthetic classes from JaCoCo counting, or (b) set per-package instruction floors to the achievable level while holding line coverage ≥95%. Expect the same pattern in the other coroutine-heavy packages (sync, viewmodels).

Heads-up from **lane 1 (#246 / PR #254)** for the ratchet design: Instruction coverage of `data/repository` can't reach 95% by tests alone — ~82.8% of that package's *missed instructions* are Kotlin-coroutine synthetics: `Flow.map` collector continuations (`…$$inlined$map$1$1`) and suspend-lambda state machines (`…$openMessage$2`), whose suspend/resume dispatch branches don't execute under synchronous test mocks (known JaCoCo × coroutines limitation). Only ~4 lines in `MailRepositoryImpl` are genuinely unhit; **line** coverage is already ≥98%. For the capstone, the ratchet should either (a) **exclude these synthetic classes** from JaCoCo counting, or (b) set per-package **instruction** floors to the achievable level while holding **line** coverage ≥95%. Expect the same pattern in the other coroutine-heavy packages (sync, viewmodels).
JMR-dev commented 2026-07-03 19:32:35 +00:00 (Migrated from github.com)

Update from lane 4 (#249 / PR #256) — reinforces that a strict per-package 95% instruction gate is not achievable; recommend gating on line ≥95% (and/or instruction with explicit exclusions). Two structural reasons:

  1. JVM-untestable Android classes (repo convention: JVM-test the extracted logic, cover the Android seam via E2E):
    • push/IdleService — a foreground Service, can't be instantiated off-device; it is the entire push shortfall (23% instr / 19% line).
    • reporting/ReportUploadScheduler — WorkManager.getInstance() is a static on an abstract class MockK can't stub (AbstractMethodError). SyncScheduler is testable only because it injects Provider<WorkManager>.
    • reporting/ReportUploadWorker HTTP path — unreachable while BuildConfig.DEBUG_REPORT_ENDPOINT is empty (default, inlined constant).
    • CrashReporter.terminate — Process.killProcess/exitProcess would kill the test JVM.
  2. JaCoCo coroutine/Flow synthetic deflation (same as lanes 1–2): invokeSuspend label-dispatch + .map{}/.combine{} operator synthetics count as "missed" even when fully exercised. E.g. ui/mailbox = 81.7% instruction but 98.9% line; ui (AppViewModel) = 51% instruction but 100% line (its .map{}.take(1) synthetics).

Lane-4 line coverage is ≥95% for most in-scope packages (drafts/outbox/contacts/ui-reporting 100% line; compose 99.4; settings 99.4; mailbox 98.9). Recommended ratchet: per-package LINE floors ≥95%, plus either exclude the untestable classes above or accept a lower instruction floor for push/reporting. The testability seams for those classes are captured in a separate follow-up (see below). Lane 3 (#248, instrumented) may also surface that instrumented coverage isn't wired into the JVM report — fold that into the ratchet too.

Update from **lane 4 (#249 / PR #256)** — reinforces that a strict per-package **95% instruction** gate is not achievable; recommend gating on **line ≥95%** (and/or instruction with explicit exclusions). Two structural reasons: 1. **JVM-untestable Android classes** (repo convention: JVM-test the extracted logic, cover the Android seam via E2E): - `push/IdleService` — a foreground `Service`, can't be instantiated off-device; it is the *entire* `push` shortfall (23% instr / 19% line). - `reporting/ReportUploadScheduler` — `WorkManager.getInstance()` is a static on an abstract class MockK can't stub (`AbstractMethodError`). `SyncScheduler` is testable only because it injects `Provider<WorkManager>`. - `reporting/ReportUploadWorker` HTTP path — unreachable while `BuildConfig.DEBUG_REPORT_ENDPOINT` is empty (default, inlined constant). - `CrashReporter.terminate` — `Process.killProcess`/`exitProcess` would kill the test JVM. 2. **JaCoCo coroutine/Flow synthetic deflation** (same as lanes 1–2): `invokeSuspend` label-dispatch + `.map{}/.combine{}` operator synthetics count as "missed" even when fully exercised. E.g. `ui/mailbox` = **81.7% instruction but 98.9% line**; `ui` (AppViewModel) = **51% instruction but 100% line** (its `.map{}.take(1)` synthetics). Lane-4 line coverage is ≥95% for most in-scope packages (drafts/outbox/contacts/ui-reporting 100% line; compose 99.4; settings 99.4; mailbox 98.9). **Recommended ratchet: per-package LINE floors ≥95%**, plus either exclude the untestable classes above or accept a lower instruction floor for `push`/`reporting`. The testability seams for those classes are captured in a separate follow-up (see below). Lane 3 (#248, instrumented) may also surface that instrumented coverage isn't wired into the JVM report — fold that into the ratchet too.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: JMR-dev/LibreMail#251