Announce a baseline it cannot read, rather than falling quiet
"A comparison was asked for" and "a number was found to compare against" were one variable, and collapsing them put the report one refactor away from being the thing #83 filed. The sed that reads FAILS_ON_EMULATOR_API37_BASELINE is anchored at the line start, so indenting the const into an object -- or renaming it, or moving it -- empties it, and the old code then skipped the whole comparison while the table kept printing exactly as before. Silent, and indistinguishable from a run that matched. Now an unreadable baseline is itself a deviation, with the notice naming the const so the fix is obvious. Verified against the real captured log of run 32865281555 three ways: baseline file absent, const indented into an object, and the committed file unchanged -- the first two announce, the third stays silent.
This commit is contained in:
@@ -31,9 +31,11 @@ annotation class FailsOnEmulatorApi37
|
||||
* How many tests carry [FailsOnEmulatorApi37] — the advisory API 37 job's committed baseline.
|
||||
*
|
||||
* **No Kotlin reads this, and it is not stray config.** `.github/scripts/e2e-report-shape.sh`
|
||||
* parses it out of this file by name, and the advisory job compares the run it just did against
|
||||
* it: this many tests should start, and all of them should fail. Deleting it makes that
|
||||
* comparison silently stop happening — the report keeps printing, with nothing to compare to.
|
||||
* parses it out of this file by name, with a line-anchored pattern, and the advisory job compares
|
||||
* the run it just did against it: this many tests should start, and all of them should fail.
|
||||
* Deleting it, renaming it, or indenting it into a class stops the comparison — the report would
|
||||
* keep printing with nothing to compare to, so it announces that it could not read the baseline
|
||||
* rather than falling quiet. If you see that notice, this line is what it means.
|
||||
*
|
||||
* **One number, both checks, and that is what the marker means.** A test carrying it cannot pass
|
||||
* on this image, so the count is simultaneously how many the advisory leg runs and how many fail.
|
||||
|
||||
Reference in New Issue
Block a user