Investigate whether sharding the JVM unit-test job (:app:testDebugUnitTest) across parallel CI runners would meaningfully cut CI wall-clock time — now that the coverage tranche has grown the suite to ~800+ tests — and if the analysis says it's worth it, implement it. A well-evidenced "not worth it" is an acceptable outcome.
Investigation
Measure the current testDebugUnitTest wall-clock on CI and its contribution to total pipeline time. First check whether the unit-test job is even on the critical path — the multi-API E2E matrix likely dominates total CI time, in which case sharding unit tests saves little.
Evaluate approaches, cheapest first:
maxParallelForks / forkEvery (in-JVM parallelism, no CI change) — try this before any workflow change.
A CI matrix that splits test classes across N shards (--tests filters or a test-partitioning plugin).
Account for JaCoCo aggregation: coverage exec/XML must still merge across shards so the #251 ratchet sees total coverage. This is a hard constraint — a sharding scheme that breaks the coverage report is not acceptable.
Decision + implementation
If worthwhile: implement in .github/workflows/ci.yml (+ Gradle config); keep the Static-analysis and coverage gates working (merge JaCoCo across shards); keep the managed-device E2E matrix in lockstep with app/build.gradle.kts per CLAUDE.md.
If not: document the timing evidence and the rationale on the issue/PR and close — no-op with evidence.
Sequencing (per repo owner)
Do NOT start until (1) the coverage test tranche is fully merged AND (2) the first feature PR clears (merges) after the tranche. The coordinator dispatches the agent at that trigger.
Definition of done
If implemented: validated on a real CI run — sharded unit tests green, coverage report still aggregates correctly, all gates green. (This is CI-infra, so the "test" is a green pipeline demonstrating both, rather than an app-level E2E.)
If not implemented: a written analysis with the CI-timing evidence attached to the issue.
## Goal
Investigate whether **sharding the JVM unit-test job** (`:app:testDebugUnitTest`) across parallel CI runners would meaningfully cut CI wall-clock time — now that the coverage tranche has grown the suite to ~800+ tests — and **if the analysis says it's worth it, implement it**. A well-evidenced "not worth it" is an acceptable outcome.
## Investigation
- Measure the current `testDebugUnitTest` wall-clock on CI and its contribution to total pipeline time. **First check whether the unit-test job is even on the critical path** — the multi-API E2E matrix likely dominates total CI time, in which case sharding unit tests saves little.
- Evaluate approaches, cheapest first:
- `maxParallelForks` / `forkEvery` (in-JVM parallelism, **no CI change**) — try this before any workflow change.
- A CI matrix that splits test classes across N shards (`--tests` filters or a test-partitioning plugin).
- Account for **JaCoCo aggregation**: coverage exec/XML must still merge across shards so the #251 ratchet sees total coverage. This is a hard constraint — a sharding scheme that breaks the coverage report is not acceptable.
## Decision + implementation
- **If worthwhile:** implement in `.github/workflows/ci.yml` (+ Gradle config); keep the Static-analysis and coverage gates working (merge JaCoCo across shards); keep the managed-device E2E matrix in lockstep with `app/build.gradle.kts` per CLAUDE.md.
- **If not:** document the timing evidence and the rationale on the issue/PR and close — no-op with evidence.
## Sequencing (per repo owner)
**Do NOT start until (1) the coverage test tranche is fully merged AND (2) the first feature PR clears (merges) after the tranche.** The coordinator dispatches the agent at that trigger.
## Definition of done
- **If implemented:** validated on a real CI run — sharded unit tests green, coverage report still aggregates correctly, all gates green. (This is CI-infra, so the "test" is a green pipeline demonstrating both, rather than an app-level E2E.)
- **If not implemented:** a written analysis with the CI-timing evidence attached to the issue.
Investigation result: sharding the unit-test job is not worthwhile — no-op with evidence
Recommendation: do not shard, do not add maxParallelForks. The unit-test job is ~8.5 minutes off the critical path, so any speedup to it saves zero total CI wall-clock; the only in-JVM option (maxParallelForks) would actively risk flakiness in this suite for no benefit. Closing as "investigated, not implemented" per the DoD.
1. The unit-test job is not on the critical path
Job-level timings from two recent green main-targeting runs:
Job
Run [28693380547] (15m08s total)
Run [28692202079] (18m06s total)
Unit tests (whole job)
6m35s
6m37s
— setup (checkout → Set up Gradle)
21s
~20s
— Run unit tests step
6m04s
~6m
— report + JaCoCo + upload
~7s
~7s
Critical path = E2E (API 37 preview)
14m51s
15m21s
E2E (API 36), next-slowest
10m49s
10m22s
Unit-test slack before CI passed fired
8m22s
8m49s
All jobs start together (they only needs: traffic-control) and fan into CI passed. The gate is bounded by the E2E matrix — API 37 preview at ~15 min, with API 29–36 each ~9–11 min — every one of which runs in parallel with the unit-test job. The unit-test job (~6.5 min for 943 @Tests across 113 files) finishes ~8.5 min before the gate.
Consequence: even reducing testDebugUnitTest to 0 s would leave total CI at ~15 min. Wall-clock saved by sharding unit tests = 0. This confirms the hypothesis in the ticket ("the multi-API E2E matrix likely dominates total CI time, in which case sharding unit tests saves little"). If total CI time is ever the target, the lever is the E2E matrix / API 37 preview job, not unit tests — out of scope here.
2. maxParallelForks — the cheapest option — is a net negative in this suite
org.gradle.parallel=true is already set, but it only parallelizes across Gradle modules and :app is single-module, so testDebugUnitTest currently runs one fork. Raising maxParallelForks distributes test classes across concurrent JVM forks — and this suite is not fork-safe:
All 5 GreenMail classes (ImapClientTest, ImapClientBackfillTest, ImapFolderOpenLatencyTest, SmtpSenderTest, MailBackfillerTest) construct GreenMail(ServerSetupTest.SMTP_IMAP) / ServerSetupTest.SMTP — GreenMail's fixed-port preset (SMTP 3025, IMAP 3143, …) — and start() them in @Before.
With >1 fork, two of these classes can run concurrently and both bind the same fixed ports → java.net.BindException: Address already in use → flaky CI.
Making the suite fork-safe would first require migrating those classes off ServerSetupTest onto dynamically-allocated ports (real work, real risk) — all to speed up a job that is already ~8.5 min off the critical path. Pure downside.
3. Matrix-sharding across runners is even worse here
Splitting classes across N runner jobs would: (a) pay the ~1–2 min setup (checkout + JDK + Android SDK install + Gradle config) N times; (b) recompile the main + ~113 test source files N times (the 6m04s Run unit tests step includes that compile); (c) consume 2–4× the runner-minutes; and (d) require merging JaCoCo exec/XML across shards so the #251 coverage ratchet still sees total coverage — added complexity against a hard ≥95% constraint. High risk to a hard constraint, N× runner cost, and still zero wall-clock benefit because it's off the critical path.
Bottom line
Option
Wall-clock saved
Cost / risk
Verdict
maxParallelForks
0 (off critical path)
GreenMail fixed-port flakiness
✗
Matrix shard + JaCoCo merge
0 (off critical path)
N× runners, N× compile, coverage-merge risk to #251
✗
Do nothing
—
—
✓
Re-open trigger: revisit only if the unit-test job's wall-clock ever approaches the E2E critical path — i.e. testDebugUnitTest climbs past ~13–14 min (roughly a 2× suite growth from today's ~6.5 min) or the E2E critical path drops below it. Until then, sharding is complexity and flake-surface for no gain.
## Investigation result: sharding the unit-test job is **not worthwhile** — no-op with evidence
Recommendation: **do not shard, do not add `maxParallelForks`.** The unit-test job is ~8.5 minutes off the critical path, so any speedup to it saves **zero** total CI wall-clock; the only in-JVM option (`maxParallelForks`) would actively risk flakiness in this suite for no benefit. Closing as "investigated, not implemented" per the DoD.
### 1. The unit-test job is not on the critical path
Job-level timings from two recent green `main`-targeting runs:
| Job | Run [28693380547] (15m08s total) | Run [28692202079] (18m06s total) |
|---|---|---|
| **Unit tests** (whole job) | **6m35s** | **6m37s** |
| — setup (checkout → Set up Gradle) | 21s | ~20s |
| — `Run unit tests` step | 6m04s | ~6m |
| — report + JaCoCo + upload | ~7s | ~7s |
| **Critical path** = E2E (API 37 preview) | **14m51s** | **15m21s** |
| E2E (API 36), next-slowest | 10m49s | 10m22s |
| Unit-test **slack** before `CI passed` fired | **8m22s** | **8m49s** |
All jobs start together (they only `needs: traffic-control`) and fan into `CI passed`. The gate is bounded by the E2E matrix — API 37 preview at ~15 min, with API 29–36 each ~9–11 min — every one of which runs **in parallel** with the unit-test job. The unit-test job (~6.5 min for 943 `@Test`s across 113 files) finishes ~8.5 min before the gate.
**Consequence:** even reducing `testDebugUnitTest` to 0 s would leave total CI at ~15 min. Wall-clock saved by sharding unit tests = **0**. This confirms the hypothesis in the ticket ("the multi-API E2E matrix likely dominates total CI time, in which case sharding unit tests saves little"). If total CI time is ever the target, the lever is the E2E matrix / API 37 preview job, not unit tests — out of scope here.
### 2. `maxParallelForks` — the cheapest option — is a net negative in this suite
`org.gradle.parallel=true` is already set, but it only parallelizes across Gradle modules and `:app` is single-module, so `testDebugUnitTest` currently runs one fork. Raising `maxParallelForks` distributes **test classes across concurrent JVM forks** — and this suite is not fork-safe:
- All 5 GreenMail classes (`ImapClientTest`, `ImapClientBackfillTest`, `ImapFolderOpenLatencyTest`, `SmtpSenderTest`, `MailBackfillerTest`) construct `GreenMail(ServerSetupTest.SMTP_IMAP)` / `ServerSetupTest.SMTP` — GreenMail's **fixed-port** preset (SMTP 3025, IMAP 3143, …) — and `start()` them in `@Before`.
- With >1 fork, two of these classes can run concurrently and both bind the same fixed ports → `java.net.BindException: Address already in use` → **flaky CI**.
Making the suite fork-safe would first require migrating those classes off `ServerSetupTest` onto dynamically-allocated ports (real work, real risk) — all to speed up a job that is already ~8.5 min off the critical path. Pure downside.
### 3. Matrix-sharding across runners is even worse here
Splitting classes across N runner jobs would: (a) pay the ~1–2 min setup (checkout + JDK + **Android SDK install** + Gradle config) **N times**; (b) recompile the main + ~113 test source files **N times** (the 6m04s `Run unit tests` step includes that compile); (c) consume 2–4× the runner-minutes; and (d) require **merging JaCoCo exec/XML across shards** so the #251 coverage ratchet still sees total coverage — added complexity against a **hard** ≥95% constraint. High risk to a hard constraint, N× runner cost, and still **zero** wall-clock benefit because it's off the critical path.
### Bottom line
| Option | Wall-clock saved | Cost / risk | Verdict |
|---|---|---|---|
| `maxParallelForks` | 0 (off critical path) | GreenMail fixed-port flakiness | ✗ |
| Matrix shard + JaCoCo merge | 0 (off critical path) | N× runners, N× compile, coverage-merge risk to #251 | ✗ |
| **Do nothing** | — | — | ✓ |
**Re-open trigger:** revisit only if the unit-test job's wall-clock ever approaches the E2E critical path — i.e. `testDebugUnitTest` climbs past ~13–14 min (roughly a 2× suite growth from today's ~6.5 min) **or** the E2E critical path drops below it. Until then, sharding is complexity and flake-surface for no gain.
Closing as investigated → not worthwhile (details in the prior comment): unit tests (~6.5m) finish ~8.5m before the E2E-bounded CI gate, so sharding saves ~0 wall-clock; maxParallelForks would collide on GreenMail fixed ports. Re-open if the unit-test job grows past ~13-14m or the E2E critical path drops below it.
Closing as investigated → not worthwhile (details in the prior comment): unit tests (~6.5m) finish ~8.5m before the E2E-bounded CI gate, so sharding saves ~0 wall-clock; maxParallelForks would collide on GreenMail fixed ports. Re-open if the unit-test job grows past ~13-14m or the E2E critical path drops below it.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Goal
Investigate whether sharding the JVM unit-test job (
:app:testDebugUnitTest) across parallel CI runners would meaningfully cut CI wall-clock time — now that the coverage tranche has grown the suite to ~800+ tests — and if the analysis says it's worth it, implement it. A well-evidenced "not worth it" is an acceptable outcome.Investigation
testDebugUnitTestwall-clock on CI and its contribution to total pipeline time. First check whether the unit-test job is even on the critical path — the multi-API E2E matrix likely dominates total CI time, in which case sharding unit tests saves little.maxParallelForks/forkEvery(in-JVM parallelism, no CI change) — try this before any workflow change.--testsfilters or a test-partitioning plugin).Decision + implementation
.github/workflows/ci.yml(+ Gradle config); keep the Static-analysis and coverage gates working (merge JaCoCo across shards); keep the managed-device E2E matrix in lockstep withapp/build.gradle.ktsper CLAUDE.md.Sequencing (per repo owner)
Do NOT start until (1) the coverage test tranche is fully merged AND (2) the first feature PR clears (merges) after the tranche. The coordinator dispatches the agent at that trigger.
Definition of done
Investigation result: sharding the unit-test job is not worthwhile — no-op with evidence
Recommendation: do not shard, do not add
maxParallelForks. The unit-test job is ~8.5 minutes off the critical path, so any speedup to it saves zero total CI wall-clock; the only in-JVM option (maxParallelForks) would actively risk flakiness in this suite for no benefit. Closing as "investigated, not implemented" per the DoD.1. The unit-test job is not on the critical path
Job-level timings from two recent green
main-targeting runs:Run unit testsstepCI passedfiredAll jobs start together (they only
needs: traffic-control) and fan intoCI passed. The gate is bounded by the E2E matrix — API 37 preview at ~15 min, with API 29–36 each ~9–11 min — every one of which runs in parallel with the unit-test job. The unit-test job (~6.5 min for 943@Tests across 113 files) finishes ~8.5 min before the gate.Consequence: even reducing
testDebugUnitTestto 0 s would leave total CI at ~15 min. Wall-clock saved by sharding unit tests = 0. This confirms the hypothesis in the ticket ("the multi-API E2E matrix likely dominates total CI time, in which case sharding unit tests saves little"). If total CI time is ever the target, the lever is the E2E matrix / API 37 preview job, not unit tests — out of scope here.2.
maxParallelForks— the cheapest option — is a net negative in this suiteorg.gradle.parallel=trueis already set, but it only parallelizes across Gradle modules and:appis single-module, sotestDebugUnitTestcurrently runs one fork. RaisingmaxParallelForksdistributes test classes across concurrent JVM forks — and this suite is not fork-safe:ImapClientTest,ImapClientBackfillTest,ImapFolderOpenLatencyTest,SmtpSenderTest,MailBackfillerTest) constructGreenMail(ServerSetupTest.SMTP_IMAP)/ServerSetupTest.SMTP— GreenMail's fixed-port preset (SMTP 3025, IMAP 3143, …) — andstart()them in@Before.java.net.BindException: Address already in use→ flaky CI.Making the suite fork-safe would first require migrating those classes off
ServerSetupTestonto dynamically-allocated ports (real work, real risk) — all to speed up a job that is already ~8.5 min off the critical path. Pure downside.3. Matrix-sharding across runners is even worse here
Splitting classes across N runner jobs would: (a) pay the ~1–2 min setup (checkout + JDK + Android SDK install + Gradle config) N times; (b) recompile the main + ~113 test source files N times (the 6m04s
Run unit testsstep includes that compile); (c) consume 2–4× the runner-minutes; and (d) require merging JaCoCo exec/XML across shards so the #251 coverage ratchet still sees total coverage — added complexity against a hard ≥95% constraint. High risk to a hard constraint, N× runner cost, and still zero wall-clock benefit because it's off the critical path.Bottom line
maxParallelForksRe-open trigger: revisit only if the unit-test job's wall-clock ever approaches the E2E critical path — i.e.
testDebugUnitTestclimbs past ~13–14 min (roughly a 2× suite growth from today's ~6.5 min) or the E2E critical path drops below it. Until then, sharding is complexity and flake-surface for no gain.Closing as investigated → not worthwhile (details in the prior comment): unit tests (~6.5m) finish ~8.5m before the E2E-bounded CI gate, so sharding saves ~0 wall-clock; maxParallelForks would collide on GreenMail fixed ports. Re-open if the unit-test job grows past ~13-14m or the E2E critical path drops below it.