Compare commits

..
Author SHA1 Message Date
JMR-dev 4ff44be1d7 Give the Robolectric choice a reason that is still true
Two test classes justified using Robolectric by asserting that the alternative does not
exist:

  AppRootRestorationTest       "The instrumented tests cannot run on the development
                                host at all (see CLAUDE.md)"
  OutputPublisherStagingTest   "The instrumented suite cannot run on the development
                                host, so this is the only place [it] can be caught"

Both were true when written and stopped being true on 2026-08-22, when the segfault was
traced to SwiftShader's Reactor JIT against SELinux execheap rather than to the machine.
tools/local-emulator/run-e2e.sh has run API 33-36 here since.

The first one cites CLAUDE.md as its authority, and PR #73 corrected CLAUDE.md to say the
opposite. So it was no longer merely stale: a reader who followed the reference found the
contradiction, with the citation making the wrong half look verified. That is the worst
version of this -- R14, R15, R20 and R25 were all the same defect, and this is the fifth.

The choice itself was never wrong, which is why the fix is not to move these tests. Both
belong on the JVM, and the honest reason is cost rather than impossibility: neither needs
anything a device supplies, and both run inside the same ./gradlew invocation as every
other unit test instead of booting an emulator. That argument survives the correction; the
premise did not.

The old line also has a second failure mode worth naming. "Nobody can execute this" invites
a reader to skip the local run and let CI decide, which is the opposite of what the
definition-of-done in #51 asks for.

Verified: the string appears nowhere in app/src now, and testDebugUnitTest, ktlintCheck and
detekt are green.

Closes #46.
2026-08-24 22:45:02 -05:00
Jason Ross ad28293b72 Merge pull request #82 from JMR-dev/docs/api37-advisory-counts
Say three where a third test joined, and stop the name claiming to be exact
2026-08-24 22:06:10 -05:00
JMR-dev b9abe85580 Say three where a third test joined, and stop the name claiming to be exact
#80 added SafPickerRoundTripTest's rotation case to @FailsOnEmulatorApi37, because a
real rotation aborts the framework on android-37.0. Three tests carry the marker now --
two in Media3EngineTest, one in SafPickerRoundTripTest -- and five statements still
described two.

Four were counts, and wrong:

  status_check.yml  "notAnnotation removes the two tests that do not pass"
  status_check.yml  "The two API 37 tests the gating row above excludes"
  CLAUDE.md         "the gating leg runs the other 55"          (59 - 3 = 56)
  CLAUDE.md         "do not read a green run as evidence those two tests pass"

The fifth was worse, because it was not a count. The advisory job's header justified its
name with an invariant:

  "It is named for WHAT IT RUNS, deliberately. Both tests drive a full H.264 -> H.265
   hardware transcode through Media3Engine"

The rotation case drives no transcode. So the comment did not merely miscount -- it
asserted a property of the job's contents that had stopped being true, and that property
was the entire argument for the name.

The name is unchanged, deliberately, and the header now says so instead of implying the
question never arose. This is not a required context, it is red on every PR by design, and
it is one people have learned to look for; renaming a check costs more than the imprecision
does. What replaced the invariant is the honest rule: THE MARKER IS THE DEFINITION, NOT THE
NAME -- this job holds the tests that cannot pass on the API 37 emulator image, whatever
their subject.

Two things stay as they were because they are still true. "the two Media3EngineTest cases
that pass here" is correct: that class has four tests and two carry the marker. And the
decoder theory is still a claim about the Media3 pair alone, so it now says so rather than
being read as covering a rotation failure it has nothing to do with.

Nothing about the job's behaviour changes: same name, same continue-on-error, same marker,
same selection on both rows. Verified: yaml parses, five jobs, matrix still 33/34/35/36/37.

The check to re-run when a test next joins or leaves the marker, which is the event that
broke this twice:

  grep -rn "@FailsOnEmulatorApi37" app/src/androidTest --include='*.kt' | grep -v import | grep -c FailsOn

It must equal the number every corrected comment states. It is 3.

Closes #81.
2026-08-24 21:58:33 -05:00
Jason Ross 4375a377bc Merge pull request #80 from JMR-dev/test/r38-8-saf-e2e
Pick a file through the real system picker, then rotate the phone
2026-08-24 21:23:36 -05:00
4 changed files with 44 additions and 21 deletions
+21 -9
View File
@@ -266,7 +266,7 @@ jobs:
# api-level must be "37.0". A bare 37 is not an SDK package and fails
# during setup, which cost a run to discover.
#
# notAnnotation removes the two tests that do not pass on this image; they
# notAnnotation removes the three tests that do not pass on this image; they
# run in the advisory job below, off the same marker so they cannot end up
# in both or neither. docs/api-37-emulator-crash.md has the measurements.
- label: "37"
@@ -367,7 +367,7 @@ jobs:
if-no-files-found: ignore
# ---------------------------------------------------------------------------
# The two API 37 tests the gating row above excludes, run on their own so they
# The three API 37 tests the gating row above excludes, run on their own so they
# stay visible instead of disappearing behind a notAnnotation.
#
# continue-on-error: it reports, it never blocks. That is the whole reason it is
@@ -375,17 +375,29 @@ jobs:
# job's `E2E API <label>` name, and a check cannot be both required and advisory
# under one name.
#
# It is named for WHAT IT RUNS, deliberately. Both tests drive a full H.264 ->
# H.265 hardware transcode through Media3Engine -- which is exactly what
# separates them from the two Media3EngineTest cases that pass here, since those
# two never decode video. The current theory about why they fail is in the next
# paragraph, where it can be corrected without renaming a check that people have
# already learned to look for.
# It was named for WHAT IT RUNS, and that is now APPROXIMATE rather than exact.
# When this job was created it held two tests, both driving a full H.264 -> H.265
# hardware transcode through Media3Engine -- which is exactly what separated them
# from the two Media3EngineTest cases that pass here, since those two never decode
# video. Since 2026-08-25 it also holds SafPickerRoundTripTest's rotation case,
# which drives no transcode at all: a real rotation rebuilds every surface at once
# and takes the framework down on this image (INSTRUMENTATION_ABORTED), which is a
# different failure from the decoder one below.
#
# The name is kept anyway, and that is a decision rather than an oversight. This is
# not a required context, it is red on every PR by design, and it is one people
# have learned to look for -- renaming a check costs more than the imprecision
# does. **The marker is the definition, not the name:** what this job holds is the
# tests that cannot pass on the API 37 emulator image, whatever their subject. The
# theory about the Media3 pair is in the next paragraph, where it can be corrected
# without touching the name.
#
# THEORY, NOT SETTLED: the exception surfaces at `dequeueOutputBuffer` on
# `c2.goldfish.h264.decoder`, the emulator's own codec, which gets its frames out
# of a host-side colour buffer -- the same readback machinery that aborts
# surfaceflinger on this image. What is MEASURED is narrower: these two fail on
# surfaceflinger on this image. It is about the MEDIA3 PAIR only; the rotation
# case above fails for its own reason. What is MEASURED is narrower: those two
# fail on
# the API 37 emulator image; pass at API 36 on this runner under the same renderer
# AND the same SystemUI-disable path; pass at API 33-36 without that path at all,
# since nothing below 37 needs it; and pass on a physical Pixel 10 Pro XL at 37. That the decoder is the culprit rather than something else
+13 -7
View File
@@ -76,16 +76,22 @@ days. Read it as the current answer, and see the git history if you need the old
`angle_indirect` and `swangle_indirect` all boot, while `auto`, `off`, `guest` and
`swiftshader_indirect` do not. `docs/local-emulator.md` has the evidence and the per-API renderer
table.
- **CI runs API 37, and it gates.** The matrix is 33/34/35/36/37. Two Media3 hardware-transcode
tests fail inside the emulator's own `c2.goldfish.h264.decoder` rather than on anything this app
does; they carry `@FailsOnEmulatorApi37` and run in a separate `continue-on-error` job,
`E2E API 37 Media3 hardware transcode (advisory)`. The gating leg runs the other 55.
**That advisory job is red on every PR, by design** — do not read it as your change breaking
something, and do not read a green run as evidence those two tests pass.
- **CI runs API 37, and it gates.** The matrix is 33/34/35/36/37. **Three** of the 59 instrumented
tests cannot pass on that image, for two unrelated reasons: two Media3 hardware transcodes fail
inside the emulator's own `c2.goldfish.h264.decoder`, and one SAF test takes the framework down
when it rotates the display. All three carry `@FailsOnEmulatorApi37` and run in a separate
`continue-on-error` job; the gating leg runs the other 56.
That job is still called `E2E API 37 Media3 hardware transcode (advisory)`, which no longer
describes everything in it. The name is kept deliberately — it is not a required context and
people have learned to look for it — so **read the marker, not the name**, for what it holds.
**It is red on every PR, by design**: do not read it as your change breaking something, and do
not read a green run as evidence those three tests pass.
`docs/api-37-emulator-crash.md` has the measurements.
Still true, and the reason the advisory job is not simply deleted: **API 37 needs a manual check on
the Pixel 10 Pro XL before each release.** The advisory pair is the one thing CI cannot answer for.
the Pixel 10 Pro XL before each release.** Those three tests are the one thing CI cannot answer
for.
On a device or emulator, build only the ABI it can execute:
@@ -33,9 +33,12 @@ import org.robolectric.RobolectricTestRunner
* representation survives a `Bundle` round trip. A JVM round-trip test on the
* saver covers the representation.
*
* Robolectric rather than the instrumented suite, deliberately. The instrumented tests
* cannot run on the development host at all (see CLAUDE.md), and a red test nobody can
* execute is not a loop anyone can work in.
* Robolectric rather than the instrumented suite, deliberately -- but not because the
* instrumented suite is unavailable. It runs on this host for API 33-36
* (`tools/local-emulator/run-e2e.sh`), and CI runs 33-37. The reason is cost: this test
* needs a composition and a saved-state round trip, nothing a device supplies, and it runs
* in the same `./gradlew` invocation as every other JVM test instead of booting an
* emulator. A loop measured in seconds is a loop people stay inside.
*/
@UnstableApi
@RunWith(RobolectricTestRunner::class)
@@ -18,8 +18,10 @@ import java.util.UUID
* the actual filesystem — the same calls `reset()` makes, without needing a ViewModel (both
* of those construct a `WorkManager`, which is not initialised on the JVM classpath).
*
* The instrumented suite cannot run on the development host, so this is the only place the
* "Start over leaks a full-size copy" defect can be caught before CI.
* The instrumented suite could also catch the "Start over leaks a full-size copy" defect --
* it runs on this host for API 33-36 (`tools/local-emulator/run-e2e.sh`) and on CI for
* 33-37. Here rather than there because a real `cacheDir` is all the defect needs, and
* finding it costs an emulator boot there and a few seconds here.
*/
@RunWith(RobolectricTestRunner::class)
class OutputPublisherStagingTest {