Find out why the emulators segfault, and make them run #9

Merged
JMR-dev merged 2 commits from tools/local-emulator into main 2026-08-23 02:15:16 +00:00
JMR-dev commented 2026-08-23 01:45:51 +00:00 (Migrated from github.com)

CLAUDE.md has said "Emulators segfault on this host — qemu dies on every AVD" since the E2E matrix landed, and the PR that introduced it called the failure "exit 139 across three AVDs and both GPU backends, environmental". That is accurate about the symptom and wrong about the cause. The cost of being wrong was the entire instrumented suite being unrunnable locally.

Root cause

SwiftShader's Reactor JIT writes generated GLES shader code onto the heap and then mprotects it executable. Fedora's SELinux policy denies that — execheap is not granted to unconfined_t and selinuxuser_execheap is off by default — so the mprotect fails and the emulator takes SIGSEGV the instant it calls the routine it just generated. The AVC denial and the core are the same event, one second apart, every time:

audit[655856]: AVC avc: denied { execheap } for pid=655856 comm="RenderThread"
  scontext=unconfined_u:unconfined_r:unconfined_t:s0 tclass=process permissive=0
655856 SIGSEGV present

The predictor is mechanical and held 7 for 7: a run crashes if and only if it dlopens lib64/gles_swiftshader/libGLESv2.so.

-gpu Result
host, angle_indirect, swangle_indirect boots, 20–26 s
auto, off, guest, swiftshader_indirect SIGSEGV, exit 139

auto is the default, and headless it resolves to SwiftShader GLES even on a machine with a real GPU. That is why the failure looked universal rather than renderer-specific, and why "both GPU backends" was a reasonable but wrong inference.

The suite now runs locally

API Android Boot Tests Fail Skip
33 13 50 s 49 0 2
34 14 55 s 49 0 2
35 15 40 s 49 0 2
36 16 90 s 49 0 2

13 m 29 s total, every level matching the Pixel 10 Pro XL baseline exactly. The two skips are the same assumption-guarded RealMediaBenchmark tests each time.

What is in here

tools/local-emulator/run-e2e.sh creates the AVD, picks a renderer that works and refuses the ones that do not, applies CI's disk-size/ram-size pins, then hands off to .github/scripts/e2e-run.sh rather than forking it, so local and CI diagnostics cannot drift.

The one change to CI's script adds an optional E2E_EXTRA_GRADLE_ARGS, unset in CI, so CI runs byte-identical commands. It exists because the local runner does several API levels against one checkout, where Gradle would otherwise mark connectedDebugAndroidTest up-to-date and report the previous level's results as the current one's — the failure mode commit 356c04d is about.

Not applied here

docs/local-emulator.md proposes a CLAUDE.md correction rather than making it. That claim is load-bearing in several places — it is the stated reason compileDebugAndroidTestKotlin is in the pre-commit list — and the wording deserves review. The advice survives even though the reasoning weakens: compiling androidTest is still the fast check, and nobody wants to boot four emulators to find a syntax error.

The API 37 image is still broken (docs/api-37-emulator-crash.md), unrelated to the renderer above, and still needs the manual Pixel check before release.

🤖 Generated with Claude Code

`CLAUDE.md` has said *"Emulators segfault on this host — qemu dies on every AVD"* since the E2E matrix landed, and the PR that introduced it called the failure *"exit 139 across three AVDs and both GPU backends, environmental"*. That is accurate about the symptom and wrong about the cause. The cost of being wrong was the entire instrumented suite being unrunnable locally. ## Root cause SwiftShader's Reactor JIT writes generated GLES shader code onto the **heap** and then `mprotect`s it executable. Fedora's SELinux policy denies that — `execheap` is not granted to `unconfined_t` and `selinuxuser_execheap` is off by default — so the `mprotect` fails and the emulator takes `SIGSEGV` the instant it calls the routine it just generated. The AVC denial and the core are the same event, one second apart, every time: ``` audit[655856]: AVC avc: denied { execheap } for pid=655856 comm="RenderThread" scontext=unconfined_u:unconfined_r:unconfined_t:s0 tclass=process permissive=0 655856 SIGSEGV present ``` The predictor is mechanical and held **7 for 7**: a run crashes **if and only if** it `dlopen`s `lib64/gles_swiftshader/libGLESv2.so`. | `-gpu` | Result | |---|---| | `host`, `angle_indirect`, `swangle_indirect` | boots, 20–26 s | | `auto`, `off`, `guest`, `swiftshader_indirect` | SIGSEGV, exit 139 | **`auto` is the default, and headless it resolves to SwiftShader GLES even on a machine with a real GPU.** That is why the failure looked universal rather than renderer-specific, and why "both GPU backends" was a reasonable but wrong inference. ## The suite now runs locally | API | Android | Boot | Tests | Fail | Skip | |---|---|---|---|---|---| | 33 | 13 | 50 s | 49 | 0 | 2 | | 34 | 14 | 55 s | 49 | 0 | 2 | | 35 | 15 | 40 s | 49 | 0 | 2 | | 36 | 16 | 90 s | 49 | 0 | 2 | 13 m 29 s total, every level matching the Pixel 10 Pro XL baseline exactly. The two skips are the same assumption-guarded `RealMediaBenchmark` tests each time. ## What is in here `tools/local-emulator/run-e2e.sh` creates the AVD, picks a renderer that works and refuses the ones that do not, applies CI's `disk-size`/`ram-size` pins, then **hands off to `.github/scripts/e2e-run.sh`** rather than forking it, so local and CI diagnostics cannot drift. The one change to CI's script adds an optional `E2E_EXTRA_GRADLE_ARGS`, unset in CI, so **CI runs byte-identical commands**. It exists because the local runner does several API levels against one checkout, where Gradle would otherwise mark `connectedDebugAndroidTest` up-to-date and report the previous level's results as the current one's — the failure mode commit `356c04d` is about. ## Not applied here `docs/local-emulator.md` **proposes** a `CLAUDE.md` correction rather than making it. That claim is load-bearing in several places — it is the stated reason `compileDebugAndroidTestKotlin` is in the pre-commit list — and the wording deserves review. The advice survives even though the reasoning weakens: compiling androidTest is still the fast check, and nobody wants to boot four emulators to find a syntax error. The API 37 image is still broken (`docs/api-37-emulator-crash.md`), unrelated to the renderer above, and still needs the manual Pixel check before release. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.