The verification dispatch of the reworked harness (34011072884) came back
4 expected / 3 received / 3 failed, where the one before it (34008889182) had
been 4/4/4 on the identical configuration. Nothing about the test list changed
between them: the abort landed one test earlier and the picker test never
started.
The baseline check would have called that "one now passes", which is the wrong
reading and the kind of notice #120 is about -- a deviation that is wrong often
enough to teach everyone to skim past deviation notices. So `failed` is compared
only when `completed cleanly` is yes, and `expected` is compared always, because
`Starting N tests` is printed before anything can abort and is what actually
answers "is the marked set the size the baseline says".
Two cases in e2e-report-shape-test.sh, as a pair: a truncated run short by one is
not a deviation, and a CLEAN run short by one still is -- so the first cannot have
bought its quiet by disabling the check.
This is a consequence of adding the fourth marker rather than a pre-existing bug
worth its own ticket: with three, the advisory leg had been completing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
JMR-devandClaude Opus 5
557b3edab4