Merge branch 'main' into test/join-failure-message-on-device

This commit is contained in:
Jason Ross
2026-09-06 08:43:20 -05:00
committed by GitHub
17 changed files with 1498 additions and 3 deletions
+40
View File
@@ -310,6 +310,46 @@ install for code that can never run — and on API 37 the full APK does not fit
#218 and carries the unfixed scope. **Prefer a mutation that must go red to a repetition count**
when a fix is for something intermittent.
**Every number above is `testDebugUnitTest` only, and on 2026-09-05 the instrumented suite got its
first read for that reason** — `docs/e2e-read-findings.md`, entries **E1-E6**, tickets
**#223-#230**. Four waves had been steered by a figure that **cannot see `app/src/androidTest` at
all**, so nothing had ever asked what those 60 device tests pin, only that they were green.
**It found one test that passes while testing nothing, and it is the one that matters most.**
`HardwareFallbackTest` is the only automated check of the hardware→software fallback against a
*real* codec failure, and on run `34004304566` the API 33, 34, 35 and 37 legs each log
`Routing sample_h264_444.mp4 -> ... via FFMPEG (NO_HARDWARE_ENCODER)` (API 36's logcat artifact on
that run is truncated, so it is unread rather than different): emulators expose no
hardware encoder, so the job never reaches Media3 and the `catch` it exists to prove is never
entered. Its two assertions — succeeded, output non-empty — are true anyway, and it finishes in
448 ms. **Deleting that `catch` reddens nothing on any leg** (#223).
Two things generalise from it. **A test can assert and still not reach**, which no coverage
number and no "does it assert something" review would catch — the filter that works is *does this
test's premise hold on the machine that runs it?*. And the codebase **already knew**: the sibling
`ForcedFailureTest` pins `DeviceCodecs.PERMISSIVE` against exactly this hazard and writes out why,
as does `ConversionWorkerTest`. The difference is that their assertions are about the *path*, so
without the pin they would fail loudly; `HardwareFallbackTest`'s are about the *output*, so it
passes quietly. **Prefer asserting the path over asserting the artefact** where the two differ.
The read was a triage, not a test push, and six of its seven findings are prose rather than code —
the suite itself is in good shape. What had drifted is its self-description.
**Working the tickets then found the thing the read could not: one production defect.** #238 —
joining files picked through the system picker failed outright on the stream-copy path. The
concat demuxer whitelists protocols separately from `-safe 0`, and `ffkitsaf` was not on the
list; only `STREAM_COPY` feeds it a list file, and every existing join test passed
`Uri.fromFile`, so **the one broken combination was the only one a user could reach**. Not a
missed line and not an unasserted value — two covered things no test put together, which is the
gap shape a coverage number is worst at.
**E7 is the other reusable result**, because it re-scoped its own ticket. A real
`DocumentsProvider` cannot be reached without the picker: an unprotected one is refused at
install, instrumentation runs in the app's uid so the test APK's identity is no help, and shell
identity is denied too — each denial naming `ACTION_OPEN_DOCUMENT`. So #226 has no cheap headless
half. But the *input* bridge needs no documents provider at all, which is what kept #225 headless
and is how #238 surfaced.
- **Testable code is not done until it is tested.** If a piece is unit testable, it gets unit
tests before it counts as done. If it is e2e testable, it gets e2e tests. Both clauses apply —
a change that is both needs both.