Write down that testable code is not done until it is tested #51

Merged
JMR-dev merged 1 commits from docs/definition-of-done into main 2026-08-23 13:52:01 +00:00
JMR-dev commented 2026-08-23 13:50:33 +00:00 (Migrated from github.com)

Adds one convention to CLAUDE.md: if a piece is unit testable it gets unit tests, and if it is e2e testable it gets e2e tests, before it counts as done. Both clauses — a change that is both needs both.

Why record it rather than leave it as a habit

The overnight review measured what happens without it. 46 mutations against a 257-test suite: 36 bit, 9 were vacuous — and five of those passed the entire suite while D3's reattachment edge sat completely unguarded. That code had shipped, been reviewed, and looked tested. "The suite is green" was true and meant nothing.

The convention names why it is enforceable, not just desirable

  • Unit-testable is broader than it looks here. The pure-seam pattern that work/FailureOutcome.kt documents turns "needs a device" into a pure function plus a thin edge. Robolectric is in the JVM source set now, compose-ui-test-junit4 with it, so Compose screens are unit testable too.
  • E2E is genuinely runnable locally. tools/local-emulator/run-e2e.sh does API 33–36 on this machine. Until the SELinux/renderer cause was found, "run the instrumented suite" was not a request anyone could act on — so the e2e half of this norm would have been a wish.
  • A test has to bite. Revert the line, confirm red, restore. Green is not evidence.

It also says to name what you did not cover and why. Genuine exemptions exist — a clamp that only fires when free space is below the headroom cannot be arranged against a real volume, and that was documented rather than papered over with an assertion that could never fail. Implied coverage is the problem, not admitted gaps.

What it deliberately does not say

Nothing about a coverage percentage. Measured across this work, line coverage fell 30.7% → 29.5% while the suite went 11 → 43 test files, because the fixes added setForeground/SAF/coroutine-dispatch lines the JVM cannot reach. Tying the norm to a number would punish exactly the framework-edge work that most needed doing.

🤖 Generated with Claude Code

Adds one convention to `CLAUDE.md`: **if a piece is unit testable it gets unit tests, and if it is e2e testable it gets e2e tests, before it counts as done.** Both clauses — a change that is both needs both. ## Why record it rather than leave it as a habit The overnight review measured what happens without it. **46 mutations against a 257-test suite: 36 bit, 9 were vacuous** — and five of those passed the *entire* suite while D3's reattachment edge sat completely unguarded. That code had shipped, been reviewed, and looked tested. "The suite is green" was true and meant nothing. ## The convention names why it is enforceable, not just desirable - **Unit-testable is broader than it looks here.** The pure-seam pattern that `work/FailureOutcome.kt` documents turns "needs a device" into a pure function plus a thin edge. Robolectric is in the JVM source set now, `compose-ui-test-junit4` with it, so Compose screens are unit testable too. - **E2E is genuinely runnable locally.** `tools/local-emulator/run-e2e.sh` does API 33–36 on this machine. Until the SELinux/renderer cause was found, "run the instrumented suite" was not a request anyone could act on — so the e2e half of this norm would have been a wish. - **A test has to bite.** Revert the line, confirm red, restore. Green is not evidence. It also says to **name what you did not cover and why**. Genuine exemptions exist — a clamp that only fires when free space is below the headroom cannot be arranged against a real volume, and that was documented rather than papered over with an assertion that could never fail. Implied coverage is the problem, not admitted gaps. ## What it deliberately does not say **Nothing about a coverage percentage.** Measured across this work, line coverage *fell* 30.7% → 29.5% while the suite went 11 → 43 test files, because the fixes added `setForeground`/SAF/coroutine-dispatch lines the JVM cannot reach. Tying the norm to a number would punish exactly the framework-edge work that most needed doing. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.