diff --git a/.github/scripts/e2e-run.sh b/.github/scripts/e2e-run.sh index 1b3e89e..d6ea1d2 100755 --- a/.github/scripts/e2e-run.sh +++ b/.github/scripts/e2e-run.sh @@ -52,58 +52,72 @@ WEDGE_TIMEOUT=1200 # the same shape as E2E_EXTRA_GRADLE_ARGS below. The other four E2E legs run byte-identical # commands with it unset. # -# WHY IT RUNS HERE, BEFORE THE LOGCAT STREAM: `adb shell stop` ends the `adb logcat` started -# below, and nothing restarts it, so a disable performed after that point would cost this leg -# its whole diagnostic story for the part of the run that matters. Everything this function -# counts comes from `adb logcat -d -b crash`, which is a fresh read each time and independent -# of the stream. +# WHY IT RUNS HERE, BEFORE THE LOGCAT STREAM: it is a 45-second wait, and the stream below is +# meant to cover the suite rather than the wait. Everything this function counts comes from +# `adb logcat -d -b crash`, a fresh read each time and independent of the stream. (The original +# reason was stronger and no longer applies: `adb shell stop` would have ended the streamed +# `adb logcat` and nothing restarts it. There is no `stop` here any more -- see below.) # -# WHAT IT IS FOR: the android-37.x images abort surfaceflinger from RegionSamplingThread inside -# their own gralloc mapper (docs/api-37-emulator-crash.md). surfaceflinger is a critical service, -# so init SIGKILLs zygote with it and the framework restarts under the run -- Gradle then reports +# WHAT IT IS FOR -- AND THE NAME IS NOW WRONG, WHICH IS WHY THIS PARAGRAPH IS LONG. +# The android-37.x images abort surfaceflinger from RegionSamplingThread inside their own gralloc +# mapper (docs/api-37-emulator-crash.md). surfaceflinger is a critical service, so init SIGKILLs +# zygote with it and the framework restarts under the run -- Gradle then reports # `cmd: Can't find service: package` and `Starting 0 tests`. RegionSamplingThread exists only -# because SystemUI registers a nav-bar luma-sampling listener, so removing the package removes -# the whole chain. Measured cadence of those kills: 20-90 s apart, median 60-70 s, three to five -# in a four-minute window -- fast enough that install and instrumentation start-up do not fit -# inside one gap. +# because SystemUI registers a nav-bar luma-sampling listener, so this was written to remove the +# package and with it the whole chain. Measured cadence of those kills on `-gpu host`: 20-90 s +# apart, median 60-70 s, three to five in a four-minute window. +# +# **THE DISABLE HALF OF THAT HAS NEVER WORKED, AND THE QUIET WINDOW IS WHAT THE LEG ACTUALLY +# GETS.** Measured 2026-09-05, two ways that agree: +# +# - On CI, in the gating leg of run 34006456986: `pm disable-user` is accepted at 02:28:37.9 and +# `com.android.systemui` really is in `pm list packages -d` at 02:29:33 -- and SystemUI is +# started anyway at 02:28:39.5 and again at 02:28:52.3, the second of which (pid 4275) is +# alive for the whole instrumentation run, logging `WindowManagerShell ... +# app=com.android.systemui` minutes after this function prints its final line. +# - Locally on android-37.0, with the package verified disabled before AND after a deliberate +# `stop; start`: `com.android.systemui` comes up 3 s after `system_server` regardless. +# +# So `pm disable-user --user 0 com.android.systemui` does not stop SystemUI starting on this +# image, whatever else happens. The name `E2E_DISABLE_SYSTEM_UI` and the name of this function are +# kept because the matrix row, both workflows and two documents refer to them, and a rename would +# touch all of that to no benefit -- read this comment, not the name. +# +# WHAT IS LEFT IS LOAD-BEARING, so do not delete the function as dead weight. It is the 45-second +# window with zero new `hasReadColorBufferDma` aborts. The boot-time aborts land close together -- +# 02:28:18 and 02:28:43 in that same run -- and the wait is what puts instrumentation (02:32:42) +# after them rather than inside one. That is what stops a leg reporting `Starting 0 tests`, and it +# is why the three-round retry stays. +# +# THE `pm disable-user` CALL STAYS TOO, for a narrower reason than it was written for: every green +# leg and every measurement quoted anywhere about this row was taken with it applied and SystemUI +# running. Removing it would change the configuration the numbers came from, which is not a change +# to make while fixing a flake. +# +# AND THE FRAMEWORK RESTART IS GONE, having been measured to be worse than nothing. It was written +# as `adb shell stop; adb shell start`, which are root-only; adbd is not root, so every leg printed +# `Must be root` twice and restarted nothing. Adding `adb root` made it real, and api37-debug run +# 34010167885 is what that looks like: `pm disable-user` reports success, the stop lands ~2 s later +# and kills system_server before PackageManager has flushed its delayed write of package +# restrictions, so the state is gone on the way back up -- `NOT DISABLED after the restart`, three +# rounds, `final state: SystemUI STILL ENABLED`, and the leg then reported `expected: 0, +# received: 0`. A 15 s pause before the stop does make the state survive (bisected locally), and it +# still does not help, because of the two measurements above. So the restart is removed rather than +# repaired: it cost the leg every test it had, and there is nothing for it to buy. # # NOTHING HERE TRUSTS A COMMAND'S OWN REPORT, and that is not paranoia: of four runs of an # earlier one-shot version, one (32646029143) reported `new state: disabled-user` and then -# started SystemUI eight more times, with ten more aborts. `pm disable-user` can be accepted by -# a system_server that is SIGKILLed before the state is written, and `pm disable-user` does not -# retract SystemUI's existing region-sampling registration either -- by the time boot completes -# it has already registered, so only a framework restart brings back a SystemUI-less -# surfaceflinger. Hence: disable, take the framework DOWN and confirm system_server is really -# gone (an earlier probe asked `service check` 0.3 s after `stop` and got `found` from the -# system_server that was still exiting, so its wait was not a wait), bring it back, verify the -# package against `pm list packages -d`, and require a 45 s window with zero new aborts. -# Three rounds, because one is not reliable and the failure is silent. -# -# THE RESTART NEEDED ROOT, AND DID NOT HAVE IT UNTIL 2026-09-05. `adb shell stop` and -# `adb shell start` are root-only, so every API 37 leg ever run printed `Must be root` twice and -# restarted nothing -- green legs and red ones alike. SystemUI therefore stayed up for the whole -# run, which the logcat shows directly (`WindowManagerShell ... app=com.android.systemui`, from a -# live SystemUI pid, minutes after the "final state: SystemUI disabled" line). The paragraph above -# says why the `pm disable-user` on its own buys nothing: it does not retract the registration. -# -# The images are userdebug -- `google/sdk_gphone64_x86_64/emu64xa:17/...:userdebug/dev-keys` -- so -# `adb root` is available and was the only thing missing. Measured on the local android-37.0 AVD, -# same image and fingerprint as CI: `stop` -> `Must be root`; `adb root` -> `whoami` says `root`; -# `stop` -> exit 0 and `pidof system_server` empty; `start` -> exit 0. -# -# Root is dropped again before the suite runs. Everything Gradle does afterwards -- install, -# instrument, uninstall -- has to be what the other four legs do, and `adb root` changes the uid -# every later `adb shell` runs as. Both calls restart adbd, hence `wait-for-device` after each. +# started SystemUI eight more times. So this reports what `pm list packages -d` says AND what +# `pidof` says, side by side, rather than one line implying both. # --------------------------------------------------------------------------- count_aborts() { adb logcat -d -b crash 2> /dev/null | grep -c 'hasReadColorBufferDma'; } systemui_disabled() { adb shell pm list packages -d 2> /dev/null | grep -q 'com.android.systemui'; } -adb_as_root() { adb root > /dev/null 2>&1; adb wait-for-device; } -adb_as_shell() { adb unroot > /dev/null 2>&1; adb wait-for-device; } +systemui_pid() { adb shell pidof com.android.systemui 2> /dev/null | tr -d '\r\n'; } disable_region_sampling() { - local round=1 i out down back before after + local round=1 i out pid before after while [ "$round" -le 3 ]; do - echo "--- SystemUI disable, round $round ---" + echo "--- round $round ---" for i in $(seq 1 10); do out="$(adb shell pm disable-user --user 0 com.android.systemui 2>&1 | tr -d '\r')" echo " pm attempt $i: $out" @@ -111,62 +125,20 @@ disable_region_sampling() { sleep 5 done - echo " restarting the framework" - adb_as_root - echo " adbd is running as $(adb shell whoami 2>&1 | tr -d '\r')" - out="$(adb shell stop 2>&1 | tr -d '\r')" - [ -n "$out" ] && echo " stop said: $out" - # `down` rather than reading `i` afterwards: the loop leaves `i` at 20 whether it broke on the - # process being gone or simply ran out, and the old version printed that as "system_server down - # after ~40 s" for a stop that had done nothing at all. A wait that did not observe the thing - # it was waiting for has to say so. - down=no - for i in $(seq 1 20); do - if [ -z "$(adb shell pidof system_server 2> /dev/null | tr -d '\r\n')" ]; then - down=yes - break - fi - sleep 2 - done - if [ "$down" = "yes" ]; then - echo " system_server down after ~$((i * 2)) s" - else - echo " system_server STILL RUNNING after ~$((i * 2)) s -- the stop did not take" - fi - out="$(adb shell start 2>&1 | tr -d '\r')" - [ -n "$out" ] && echo " start said: $out" - adb_as_shell - # Same flag, same reason as `down` above: this loop also used to report its own exhaustion as - # an elapsed time, so "services back after ~150 s" and "services never came back" printed the - # same line. - back=no - for i in $(seq 1 30); do - if adb shell service check package 2> /dev/null | grep -q ': found' \ - && adb shell service check activity 2> /dev/null | grep -q ': found' \ - && [ -n "$(adb shell pidof system_server 2> /dev/null | tr -d '\r\n')" ]; then - back=yes - break - fi - sleep 5 - done - if [ "$back" = "yes" ]; then - echo " services back after ~$((i * 5)) s" - else - echo " services NOT back after ~$((i * 5)) s -- package, activity or system_server missing" - fi - if systemui_disabled; then - echo " verified: com.android.systemui is in pm list packages -d" + echo " pm list packages -d: com.android.systemui is in it" else - echo " NOT DISABLED after the restart -- the package state did not survive" - round=$((round + 1)) - continue + echo " pm list packages -d: com.android.systemui is NOT in it" fi + # Printed next to the line above precisely because the two disagree on this image, and a + # reader who sees only the first will believe something that is not true. + pid="$(systemui_pid)" + echo " com.android.systemui pid: ${pid:-none} (expected: a pid -- see the header)" before="$(count_aborts)" sleep 45 after="$(count_aborts)" - echo " abort rate, SystemUI disabled: $((after - before)) new in 45 s (total ${after:-0})" + echo " aborts: $((after - before)) new in 45 s (total ${after:-0})" [ "$((after - before))" -eq 0 ] && break echo " still aborting after round $round" round=$((round + 1)) @@ -175,16 +147,16 @@ disable_region_sampling() { # A warning rather than an exit. If the disable did not take, the run is about to report # `Starting 0 tests` and fail on its own -- and it will do so with the logcat, the crash # buffer and the diagnostics attached, which is more useful than dying here with none of it. - if systemui_disabled; then - echo " final state: SystemUI disabled" + if [ "$((after - before))" -eq 0 ]; then + echo " final state: 45 s with no new aborts -- the suite starts here" else - echo "::warning::E2E api${LABEL}: SystemUI is still enabled -- expect INSTRUMENTATION_ABORTED" + echo "::warning::E2E api${LABEL}: still aborting after three rounds -- expect INSTRUMENTATION_ABORTED" fi return 0 } if [ "${E2E_DISABLE_SYSTEM_UI:-}" = "1" ]; then - echo "::group::E2E api${LABEL} -- removing the region-sampling listener" + echo "::group::E2E api${LABEL} -- waiting out the boot-time gralloc aborts" disable_region_sampling echo "::endgroup::" fi diff --git a/.github/workflows/api37-debug.yml b/.github/workflows/api37-debug.yml index b9cbe66..533648b 100644 --- a/.github/workflows/api37-debug.yml +++ b/.github/workflows/api37-debug.yml @@ -32,9 +32,11 @@ name: API 37 debug # The SystemUI disable below is the exception, and it is a real one: this workflow # drives it from its own probe step so `disable_system_ui` can be turned off for a # dispatch, where the real leg gets it through `E2E_DISABLE_SYSTEM_UI`. Two copies of -# that logic therefore exist, and on 2026-09-05 both carried the same defect -- no -# `adb root`, so `adb shell stop` answered `Must be root` and nothing restarted. Fix -# one and the other needs the same change in the same diff. +# that logic therefore exist and must be changed together. **This instrument is also +# what established that the disable half of it does nothing** -- run 34010167885, in +# which making its framework restart real cost the leg every test it had. Read +# .github/scripts/e2e-run.sh's header for the measurements; the restart is gone from +# both copies and what remains is the 45-second quiet window. # - It does not change status_check.yml. If a configuration here turns out to work, # the change to the real matrix is proposed separately. # @@ -298,72 +300,30 @@ jobs: sleep 5 done - # pm disable-user does not retract SystemUI's existing region-sampling - # registration -- by the time boot completes it has already registered. Only a - # framework restart brings back a SystemUI-less SurfaceFlinger. See - # disable_region_sampling in .github/scripts/e2e-run.sh, which this mirrors. - # - # `adb root` first, because `stop` and `start` are root-only and adbd is not root - # on a freshly booted emulator: without it both printed `Must be root` and the - # restart never happened, on this workflow and on the real leg alike, for as long - # as either has existed. Root is dropped again before the suite runs so Gradle's - # install and instrument happen as an unprivileged adb does them. Both flags below - # exist because each loop used to print its own exhaustion as an elapsed time -- - # "system_server down after 40 s" was what a stop that did nothing looked like. - echo " restarting the framework" - adb root > /dev/null 2>&1; adb wait-for-device - echo " adbd is running as $(adb shell whoami 2>&1 | tr -d '\r')" - out="$(adb shell stop 2>&1 | tr -d '\r')" - [ -n "$out" ] && echo " stop said: $out" - down=no - for i in $(seq 1 20); do - if [ -z "$(adb shell pidof system_server 2> /dev/null | tr -d '\r\n')" ]; then - down=yes - break - fi - sleep 2 - done - if [ "$down" = "yes" ]; then - echo " system_server down after $((i * 2)) s" - else - echo " system_server STILL RUNNING after $((i * 2)) s -- the stop did not take" - fi - out="$(adb shell start 2>&1 | tr -d '\r')" - [ -n "$out" ] && echo " start said: $out" - adb unroot > /dev/null 2>&1; adb wait-for-device - back=no - for i in $(seq 1 30); do - if adb shell service check package 2> /dev/null | grep -q ': found' \ - && adb shell service check activity 2> /dev/null | grep -q ': found' \ - && [ -n "$(adb shell pidof system_server 2> /dev/null | tr -d '\r\n')" ]; then - back=yes - break - fi - sleep 5 - done - if [ "$back" = "yes" ]; then - echo " services back after $((i * 5)) s" - else - echo " services NOT back after $((i * 5)) s" - fi - + # NO FRAMEWORK RESTART. There was one here, and making it work (it needed + # `adb root`) is what proved the whole disable is ineffective on this image: + # SystemUI starts anyway, measured on CI and locally, and the restart itself + # loses the package state to PackageManager's delayed write and leaves the leg + # reporting `Starting 0 tests`. e2e-run.sh's header carries the measurements. + # What is left, and what is load-bearing, is the quiet window below. if systemui_disabled; then - echo " verified: com.android.systemui is in pm list packages -d" + echo " pm list packages -d: com.android.systemui is in it" else - echo " NOT DISABLED after the restart -- the package state did not survive" - round=$((round + 1)) - continue + echo " pm list packages -d: com.android.systemui is NOT in it" fi + # Beside it, because the two disagree on this image and the first line alone + # reads as a claim about the process that is not true. + echo " com.android.systemui pid: $(adb shell pidof com.android.systemui 2> /dev/null | tr -d '\r\n')" before="$(count_aborts)" sleep 45 after="$(count_aborts)" - echo "--- abort rate, SystemUI disabled: $((after - before)) new in 45 s (total ${after:-0}) ---" + echo "--- aborts: $((after - before)) new in 45 s (total ${after:-0}) ---" [ "$((after - before))" -eq 0 ] && break echo " still aborting after round $round" round=$((round + 1)) done - systemui_disabled && echo "final state: SystemUI disabled" || echo "final state: SystemUI STILL ENABLED -- expect Starting 0 tests" + systemui_disabled && echo "final state: com.android.systemui is disabled in pm (it still runs)" || echo "final state: com.android.systemui is not even disabled in pm" fi echo "--- crash buffer (tail 60) ---" diff --git a/.github/workflows/status_check.yml b/.github/workflows/status_check.yml index 7745e24..db71918 100644 --- a/.github/workflows/status_check.yml +++ b/.github/workflows/status_check.yml @@ -254,31 +254,31 @@ jobs: api-level: "36" # API 37, and it is NOT the same device as the four rows above it. # - # CAVEAT, read this before trusting a green here: this leg runs with - # SystemUI disabled and the framework restarted under it. No other leg - # and no Pixel run uses that configuration. It is defensible only because - # nothing THIS LEG RUNS touches system UI -- Media3, FFmpeg and - # WorkManager tests -- and because the alternative is no CI coverage of - # the level this app targets. **Anything that ever does depend on system - # UI must not trust this row.** E2E_DISABLE_SYSTEM_UI is what does it; - # .github/scripts/e2e-run.sh explains the mechanism and why every step of - # it is verified rather than assumed. + # THE CAVEAT THAT USED TO BE HERE IS WITHDRAWN, 2026-09-05, and the + # withdrawal is good news. It said this leg "runs with SystemUI disabled + # and the framework restarted under it", that no other leg or Pixel run + # uses that configuration, and that anything depending on system UI must + # not trust this row. **None of that was ever true.** Measured: the + # framework restart is two root-only adb commands that answered `Must be + # root` on every leg ever run, and `pm disable-user` does not stop SystemUI + # starting on this image anyway -- in run 34006456986 the package is + # verified disabled at 02:29:33 and SystemUI (pid 4275) is up from 02:28:52 + # for the whole run. So this row's device configuration is the same as the + # other four's, and a green here means what a green on 33-36 means. # - # "this leg" and not "this suite", since 2026-08-24, and the difference is - # now load-bearing: SafPickerRoundTripTest DOES touch system UI. It drives - # DocumentsUI and rotates the display, and both reach the gralloc mapper - # this image aborts in -- disabling SystemUI removes the IDLE trigger, not - # those. Measured per method on android-37.0: the ROTATION test takes the - # framework down (INSTRUMENTATION_ABORTED) and carries - # @FailsOnEmulatorApi37, so notAnnotation below keeps it off this row; the - # PICKER test passes and runs here like anything else. A rotation rebuilds - # every surface at once, and starting another app's activity does not. + # E2E_DISABLE_SYSTEM_UI still exists and still runs, because what it + # actually buys is a 45-second window with no new gralloc aborts before the + # suite starts -- the boot-time ones land close together and instrumentation + # has to begin after them, not between them. The name is stale and kept: + # read .github/scripts/e2e-run.sh's header, which carries the measurements. # - # So this row does now run one test that depends on system UI, and the - # caveat above still applies to it: a green here is not evidence the picker - # works on a device with SystemUI running -- the Pixel release check is. - # docs/api-37-emulator-crash.md has the per-method measurements, and the - # correction that produced them. + # notAnnotation below keeps four tests off this row, not two, and one of + # them is new. SafPickerRoundTripTest's PICKER test was measured on + # 2026-08-24 as passing here and was left on the leg; four gating logcats + # read on 2026-09-05 show it aborting system_server from the task-snapshot + # path on every single run, pass or fail, which is what had been failing + # unrelated PRs (#108). Both of that class's tests now carry the marker. + # docs/api-37-emulator-crash.md has the timings and the correction. # # api-level must be a POINT release. A bare 37 is not an SDK package and # fails during setup, which cost a run to discover. `37.0` is the choice diff --git a/CLAUDE.md b/CLAUDE.md index c5f7bd7..75ef00e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -93,6 +93,17 @@ days. Read it as the current answer, and see the git history if you need the old #108, it cost roughly a third of the gating legs over the wave-4 landings (#190), and a marker is what it needed. `docs/api-37-emulator-crash.md` has the timings. + **A second thing came out of those logcats, and it withdraws a caveat rather than adding one.** + The API 37 row was documented as the one leg running "with SystemUI disabled and the framework + restarted under it", which nothing else does. Neither half was ever happening: `adb shell stop` + and `start` are root-only and answered `Must be root` on every leg ever run, and `pm + disable-user` does not stop SystemUI starting on this image anyway — measured on CI and locally, + with and without a real restart. **So this row's device configuration is the same as the other + four's, and a green here means what a green at 33–36 means.** `E2E_DISABLE_SYSTEM_UI` is kept + under its now-stale name because what it really buys is a 45-second window with no new gralloc + aborts before the suite starts, which is load-bearing; `.github/scripts/e2e-run.sh`'s header is + where that is written down. + That job is still called `E2E API 37 Media3 hardware transcode (advisory)`, which no longer describes everything in it. The name is kept deliberately — it is not a required context and people have learned to look for it — so **read the marker, not the name**, for what it holds. diff --git a/docs/api-37-emulator-crash.md b/docs/api-37-emulator-crash.md index 7ea7e07..4b8cc0d 100644 --- a/docs/api-37-emulator-crash.md +++ b/docs/api-37-emulator-crash.md @@ -317,77 +317,88 @@ So what is reliably achieved is a **rate collapse** — from roughly one abort e seconds to one every forty-five — which a 47-second Gradle run survives and a five-minute one might not. -**Read every number in this section knowing that the restart it credits was not happening.** -Found 2026-09-05: `stop` and `start` are root-only, adbd was not root in any of the three copies -of this logic, and `run-e2e.sh` sent both to `/dev/null`, so its `Must be root` was never even -printed. What the local runs almost certainly had instead is the image restarting its own -framework — on API 37 that happens every minute or so — and a restart landing after a successful -`pm disable-user` brings back a SystemUI-less zygote by itself. That would produce exactly the -rate collapse recorded above, by accident and on the image's schedule rather than the harness's, -which also explains why the same code bought nothing at all on CI's far quieter -`swiftshader_indirect` legs: there the logcat shows SystemUI alive for the whole run. The -measurements above are still what was observed; what they are evidence *of* is narrower than it -was written to be. All three copies now take root. The 180-second zero above is one measurement on a device that had been up for twelve -minutes and had already cycled its framework several times. The harness prints the quiet-check -delta on every run precisely so this is visible rather than assumed. +**And that restart has never happened — which is how the disable turned out not to work either.** +Corrected 2026-09-05; this replaces the two paragraphs above rather than qualifying them. -One ordering detail cost a whole run and is now encoded in `disable_region_sampling`: by the time -`sys.boot_completed` flips, SystemUI has **already registered**, and `pm disable-user` does not -retract an existing registration — it only stops the package being started again. Disabling it -and proceeding straight to the tests fails exactly as before. The harness therefore does -`stop; start` afterwards, so the framework that comes back never starts SystemUI at all. +`adb shell stop` and `start` are root-only, adbd is not root on a booted emulator, and all three +copies of this logic called them without `adb root`. On CI both printed `Must be root`, between +lines that read as if the restart had happened; `run-e2e.sh` sent them to `/dev/null`, so its +`Must be root` was never even visible. Neither number in those logs was an observation either — +the `pidof` loop breaks when the process is gone and otherwise falls out at its last iteration, +and the old code printed the iteration count either way, so `system_server down after ~40 s` is +what a stop that did nothing looks like. -**And on CI that `stop; start` did nothing at all until 2026-09-05.** Both are root-only commands, -adbd on a freshly booted emulator is not root, and the step log had been saying so on every API 37 -leg since the function was written — `Must be root`, twice, between lines that read as if the -restart had happened: +Adding `adb root` made the restart real, and **that is what proved the disable ineffective**. +`api37-debug` run 34010167885, `disable_system_ui=true`: ``` +--- disable round 1 --- + pm attempt 1: Package com.android.systemui new state: disabled-user restarting the framework -Must be root - system_server down after ~40 s -Must be root - services back after ~5 s - verified: com.android.systemui is in pm list packages -d + adbd is running as root + system_server down after 2 s + services back after 10 s + NOT DISABLED after the restart -- the package state did not survive ``` -Neither number was an observation. The `pidof` loop breaks when the process is gone and otherwise -falls out at its last iteration, and the old code printed the iteration count either way — so -"down after ~40 s" is what a stop that did nothing looks like. The paragraph above is what makes -this matter rather than merely untidy: without the restart the disable buys nothing, and the -logcat confirms it directly — SystemUI is alive for the whole run, logging -`WindowManagerShell ... app=com.android.systemui` minutes after `final state: SystemUI disabled`. +Three rounds of that, then `final state: SystemUI STILL ENABLED`, and the leg reported +`expected: 0, received: 0` — `Starting 0 tests`, the exact failure this function exists to +prevent. -The images are userdebug, so `adb root` is all that was missing. Measured on the local -`android-37.0` AVD, same fingerprint as CI -(`google/sdk_gphone64_x86_64/emu64xa:17/CE2A.260420.019/15611780:userdebug/dev-keys`): +Bisected locally on `android-37.0`, which explains the lost state and nothing else: + +| arm | sequence | disabled after the restart? | +|---|---|---| +| A | `pm disable-user`, then `stop` at once | **no** | +| B | `pm disable-user`, wait 15 s, then `stop` | **yes** | + +That is PackageManager's delayed write of package restrictions: the stop kills `system_server` +before the settings are flushed, and arm A is what CI did. **Arm B does not help either**, which +is the measurement that matters. With the package verified `disabled-user` before *and* after a +further clean restart: ``` -adb shell stop -> Must be root -adb root -> restarting adbd as root -adb shell whoami -> root -adb shell stop -> exit 0; adb shell pidof system_server -> (empty) -adb shell start -> exit 0 + package still disabled? YES + processes: + 9275 00:17 system_server + 9695 00:14 com.android.systemui <- started 3 s after system_server ``` -`disable_region_sampling` now takes root for the restart and drops it again with `adb unroot` -before Gradle runs, so install, instrument and uninstall happen as the other four legs do it. Both -waits report whether they observed what they were waiting for instead of printing their own -exhaustion as an elapsed time. +CI's own logcat says the same without any restart at all. In the gating leg of run 34006456986, +`pm disable-user` is accepted at 02:28:37.9 and the package really is in `pm list packages -d` at +02:29:33 — and SystemUI is started at 02:28:39.5 and again at 02:28:52.3, the second of which +(pid 4275) is alive for the whole instrumentation run. -**It does not touch the picker test's abort**, and it was never going to: that one is -WindowManager inside `system_server`, not SystemUI. What it fixes is the *idle* trigger this -section is about, which had been left running on every leg. +**So `pm disable-user --user 0 com.android.systemui` does not stop SystemUI starting on this +image**, with or without a framework restart, on CI or locally. The premise this section was +built on — "the framework that comes back never starts SystemUI at all" — is false. + +Two things follow, pointing in opposite directions. + +- **The restart is removed rather than repaired**, in all three copies. It cost a leg every test + it had and there is nothing for it to buy. What is kept is the 45-second window with zero new + aborts, which was always the part doing the work: in that same run the boot aborts land at + 02:28:18 and 02:28:43, and the wait is what puts instrumentation at 02:32:42 — after them + rather than inside one. The `pm disable-user` call is kept too, for a narrower reason than it + was written for: every green leg and every number quoted about this row was measured with it + applied, and changing the configuration while fixing a flake is not a trade worth making. +- **The rate collapse recorded above is not evidence of what it says.** Both arms of that + comparison had SystemUI running. What it measured is a device twelve minutes into its uptime + against one that had just booted — a real difference, and a different claim. The quiet gate is + still worth having on exactly that reading. ### The two deviations, stated plainly 1. **The renderer is ANGLE, not the host GPU.** Shared with nothing else in the matrix — API 33–36 run `-gpu host` locally, and CI runs `swiftshader_indirect`. -2. **SystemUI is disabled.** The API 37 leg does not run the same device configuration as any - other leg or as the Pixel. It was defensible here because nothing in this suite touched - system UI — Media3, FFmpeg and WorkManager tests — and because the alternative is no local - API 37 coverage at all. **Anything that ever does depend on system UI must not trust this - leg.** Something now does; see the section below. +2. **SystemUI is asked to be disabled, and runs anyway.** This was written as the deviation that + mattered — "anything that ever does depend on system UI must not trust this leg" — and the + measurements above say the deviation does not exist: the package is marked `disabled-user` and + `com.android.systemui` is up for the whole leg regardless. **The correction is good news + rather than bad.** This row is *more* comparable to API 33–36 and to the Pixel than it has + been claiming, not less, and the test that depends on system UI (see the section below) was + never running in the exotic configuration this bullet describes. What `pm disable-user` leaves + behind is a package-manager flag nothing acts on. ### Something does depend on system UI now, and half of it is excluded @@ -395,12 +406,13 @@ Added 2026-08-24, and the first entry on this page that is not a codec. `SafPickerRoundTripTest` drives the real system file picker and rotates the display. Both reach the gralloc mapper — DocumentsUI is another app's windows, and a rotation rebuilds every surface -on screen — and **disabling SystemUI does not help**, because it removes the *idle* trigger -(RegionSamplingThread's nav-bar luma sampling) and not this one. +on screen — and **disabling SystemUI does not help**. Two reasons now, and only the first was +known when this was written: it removes the *idle* trigger (RegionSamplingThread's nav-bar luma +sampling) and not this one, and — see the section above — it does not remove SystemUI either. -Measured one method per fresh emulator, `android-37.0`, `swangle_indirect`, SystemUI disabled and -verified quiet — separately, because inferring the second from the first is the mistake this -page's opening correction is about: +Measured one method per fresh emulator, `android-37.0`, `swangle_indirect`, with the disable +applied and verified quiet — separately, because inferring the second from the first is the +mistake this page's opening correction is about: | test | result on android-37.0 | `hasReadColorBufferDma` aborts in the window | |---|---|---| diff --git a/tools/local-emulator/run-e2e.sh b/tools/local-emulator/run-e2e.sh index 4afd1f9..04a2424 100755 --- a/tools/local-emulator/run-e2e.sh +++ b/tools/local-emulator/run-e2e.sh @@ -355,12 +355,10 @@ boot_emulator() { # may be in one of its restarts and `pm` is simply not published yet. The first attempt at this # failed exactly that way, with `cmd: Can't find service: package`. # -# The framework restart at the end is not optional, and finding that out cost a run. By the -# time `sys.boot_completed` flips, SystemUI has already registered its region-sampling listener, -# and `pm disable-user` does not retract a registration that already happened -- it only stops -# the package being started again. So the first attempt disabled SystemUI, reported success, and -# then died exactly as before with `Starting 0 tests` and four more aborts. `stop; start` cycles -# zygote deliberately, and the framework that comes back up does not start SystemUI at all. +# This used to end with a framework restart, described here as "not optional". It was neither +# optional nor happening -- see the block inside the function. What the first attempt's +# `Starting 0 tests` and four more aborts actually showed is that a `pm disable-user` on its own +# buys nothing, which is still true; what was wrong is the conclusion that a restart would. disable_region_sampling() { local api="$1" out i before after ready case "$api" in 37 | 37.*) ;; *) return 0 ;; esac @@ -383,32 +381,18 @@ disable_region_sampling() { return 0 fi - echo " restarting the framework so the region-sampling listener goes with it" - # `adb root` first, and the two redirects below used to hide why. `stop` and `start` are - # root-only, adbd is not root on a booted emulator, and both were answering `Must be root` - # into /dev/null -- so this restarted nothing, here and in both CI copies, for as long as - # any of them has existed (2026-09-05). Measured on a local android-37.0 AVD: `stop` alone - # says `Must be root`; after `adb root`, `whoami` says root, `stop` returns 0 and - # `pidof system_server` comes back empty. + # NO FRAMEWORK RESTART, and the two lines that used to be here are why this comment is long. + # They were `emu_adb shell stop` and `emu_adb shell start`, both redirected to /dev/null, and + # both root-only -- so what they printed there was `Must be root` and what they did was nothing, + # here and in the two CI copies alike. Making them real (2026-09-05) is what established that + # the disable never worked in the first place: with the package verified `disabled-user` before + # AND after a clean restart on android-37.0, `com.android.systemui` comes up 3 s after + # `system_server` regardless, and the same is visible in CI's own logcat. The restart also loses + # the package state to PackageManager's delayed write if it lands too soon after the `pm` call, + # which cost api37-debug run 34010167885 every test in the leg. # - # Read the abort table in docs/api-37-emulator-crash.md with that in mind: on API 37 the - # image restarts its own framework every minute or so, and a restart AFTER a successful - # `pm disable-user` brings back a SystemUI-less zygote by itself. That is the likeliest - # reason the disable appeared to work here while doing nothing on CI's much quieter - # swiftshader legs, where the logcat shows SystemUI alive for the whole run. - # - # Output is kept rather than discarded now, for the same reason. - emu_adb root > /dev/null 2>&1 - emu_adb wait-for-device - emu_adb shell stop 2>&1 | sed 's/^/ stop: /' - emu_adb shell start 2>&1 | sed 's/^/ start: /' - emu_adb unroot > /dev/null 2>&1 - emu_adb wait-for-device - # There is no property worth waiting on here, and an earlier version of this only looked - # like it was waiting on one: `stop` does not clear sys.boot_completed, so it still reads - # `1` throughout the restart and any loop over it returns at once. The loop below is the - # wait -- and it polls the better thing anyway, since `Can't find service: package` is the - # failure it exists to prevent. + # So the useful part of this function is the quiet window below, not the disable. See + # .github/scripts/e2e-run.sh's header, and docs/api-37-emulator-crash.md. ready=0 for i in $(seq 1 30); do if emu_adb shell service check package 2> /dev/null | grep -q ': found' \