CLAUDE.md carried three claims about instrumented tests. All three were false, one of
them contradicted a paragraph forty lines below it in the same file, and a subagent
working on #58 hit the contradiction and had to stop and flag it rather than trust the
project's own instructions. That is the cost being paid here: this file is what every
contributor and every agent reads first.
"Instrumented tests do not run locally" -- they do, API 33-36, since 22c7914.
"Emulators segfault on this host" -- solved 2026-08-22; it was SwiftShader's
Reactor JIT meeting SELinux execheap, not a
broken machine, and another renderer avoids
it. docs/local-emulator.md is titled
"Emulators do run on this host".
"CI's matrix therefore stops at API 36" -- the matrix has been 33/34/35/36/37 since
#56 merged, with a gating API 37 leg.
The contradiction was the worst of it. The testing-norm section added in #51 says "E2E is
runnable locally now", so the file simultaneously told you the emulator works and that it
segfaults on every AVD. A reader has no way to tell which half is current, and the wrong
half is the one that stops work: an agent that believes emulators are impossible here does
not try, and the local e2e half of the definition-of-done in #51 quietly stops being
enforceable.
The replacement says what is true now and names what is still true and why -- API 37 still
needs the manual Pixel check before a release, because the two advisory tests are the one
thing CI cannot answer for. It also states plainly that the advisory job is red on every
PR by design, which is the other thing agents keep rediscovering the hard way: three
separate subagents have now flagged that failure as possibly theirs.
The norm bullet now points at the section rather than re-arguing it, so there is one place
to correct next time rather than two that can drift apart again.
Same defect class as R14, R15, R20 and R25, all of which were documentation claims this
repo's own review falsified. The pattern is not that the docs were careless; it is that
they were written at a moment and the moment moved.
JMR-dev