From f54e9c67fa1be74886e6dcbd3f3e632eea443012 Mon Sep 17 00:00:00 2001 From: Jason Ross Date: Sat, 4 Jul 2026 19:58:14 -0500 Subject: [PATCH] test-infra: port local_instrumented.sh to Python + rewire preflight off GMD (#281) Port the last bash dev-script (local_instrumented.sh) to a cross-platform, stdlib-only Python 3 script (local_instrumented.py), matching api37_e2e.py's style, and rewire the /preflight skill's local E2E off the GMD apiXXDebugAndroidTest tasks (which fail locally under AEHD 2.2) onto it. - local_instrumented.py preserves the .sh's behavior exactly: comma-separated test-class CLI arg, pre-boot orphan-kill, manual cold-boot of the dev36 AVD (no GMD, no snapshot), targeted connectedDebugAndroidTest, and the EXIT-trap teardown (now try/finally + atexit + SIGINT/SIGTERM handlers, idempotent). Exit codes 0/2/3/4 preserved. - Cross-platform process kill abstracted per-OS: taskkill /F /IM on Windows, pkill -f qemu-system on *nix; process listing via tasklist / ps ax. - Teardown hardened vs the .sh: it now also reaps the emulator *launcher* image, not just qemu -- the Windows -no-window emulator spawns a sibling emulator.exe that briefly outlives the qemu VM, which a qemu-only sweep left as an orphan on return (caught by the smoke run). - SKILL.md + CLAUDE.md: replace the local api35/api36 GMD E2E steps with local_instrumented.py; CI's own multi-API matrix is untouched. CLAUDE.md documents the Python-first dev-script convention. Validated: py_compile, argparse (--help / no-arg exit 2), and a guarded emulator smoke run of org.libremail.data.local.DatabaseEncryptionTest -- boots, passes, and tears down clean (no qemu/emulator orphan on return). Co-Authored-By: Claude Opus 4.8 --- .claude/skills/preflight/SKILL.md | 75 +-- .../preflight/local_instrumented.README.md | 50 +- .../skills/preflight/local_instrumented.py | 451 ++++++++++++++++++ .../skills/preflight/local_instrumented.sh | 253 ---------- CLAUDE.md | 37 +- 5 files changed, 549 insertions(+), 317 deletions(-) create mode 100644 .claude/skills/preflight/local_instrumented.py delete mode 100755 .claude/skills/preflight/local_instrumented.sh diff --git a/.claude/skills/preflight/SKILL.md b/.claude/skills/preflight/SKILL.md index 47a8343..f7e637d 100644 --- a/.claude/skills/preflight/SKILL.md +++ b/.claude/skills/preflight/SKILL.md @@ -1,6 +1,6 @@ --- name: preflight -description: Run LibreMail's fast CI gate locally (assembleDebug + testDebugUnitTest + compileDebugAndroidTestKotlin + lintDebug + ktlintCheck + detekt) plus the top-of-matrix emulator E2E (currently API 35 + 36 via Gradle Managed Devices, then API 37 preview via the hand-provisioning script) before pushing or opening a PR. Mirrors the merge gate; the rest of the multi-API matrix stays CI-only. Use before treating a change as done. +description: Run LibreMail's fast CI gate locally (assembleDebug + testDebugUnitTest + compileDebugAndroidTestKotlin + lintDebug + ktlintCheck + detekt) plus the local emulator E2E — the instrumented test class(es) you changed via local_instrumented.py (cold-boot, no Gradle Managed Devices), then the API 37 preview via api37_e2e.py — before pushing or opening a PR. Mirrors the merge gate; CI runs the full multi-API matrix. Use before treating a change as done. --- # /preflight @@ -13,20 +13,24 @@ Run the same fast checks CI enforces on every PR, in order, and report the outco with a JDK/AGP version mismatch, check `java -version` / `JAVA_HOME` and point it at a 17–21 JDK (e.g. Android Studio's bundled JBR) before retrying. - PowerShell: invoke the wrapper as `.\gradlew`. Git Bash / the Bash tool: `./gradlew`. -- The final three E2E steps each boot an emulator, so the host needs a **free hardware +- The final two E2E steps each boot an emulator, so the host needs a **free hardware hypervisor** (Intel VT-x / AMD-V, exposed as WHPX on Windows, KVM on Linux, HVF on macOS). Shut down VirtualBox, Hyper-V-based VMs, WSL2, Docker Desktop, or any other emulator first — a VM holding the hypervisor starves the AVD, and it hangs at 0% CPU and never reaches - `sys.boot_completed`. The api35/api36 steps run through Gradle Managed Devices (Gradle - downloads the image and boots/tears down each AVD itself); the api37 step is hand-provisioned - by `api37_e2e.py` (see Steps). The first run per API level is slow while its system image + `sys.boot_completed`. Both steps are hand-provisioned by cross-platform Python scripts (**not** + Gradle Managed Devices, which fail locally on this box — see Steps): `local_instrumented.py` + cold-boots one existing AVD and runs the instrumented class(es) you changed; `api37_e2e.py` + installs + boots the API 37 preview image. The first API 37 run is slow while its system image downloads. If the host has no accelerated emulator and a device cannot boot, report the E2E step as not run rather than treating the gate as green. -- The api37 step is a stdlib-only, cross-platform **Python 3** script and needs `python3` plus - the Android SDK command-line tools (`sdkmanager`/`avdmanager`) and `emulator` on the machine, - located via `ANDROID_SDK_ROOT`/`ANDROID_HOME` (or the per-OS default: - `%LOCALAPPDATA%\Android\Sdk` on Windows, `~/Library/Android/sdk` on macOS, `~/Android/Sdk` on - Linux). The script installs the preview system image itself on first run. +- Both E2E steps are stdlib-only, cross-platform **Python 3** scripts. `local_instrumented.py` + needs `python3` plus the Android SDK `emulator` + `adb` on `PATH` and an existing AVD (any local + `apiXXDebugAndroidTest` run creates one; override with `LOCAL_INSTRUMENTED_AVD` / + `ANDROID_AVD_HOME`), and it pins `JAVA_HOME` to a JDK 17–21 itself (override with + `LOCAL_INSTRUMENTED_JDK`). `api37_e2e.py` additionally needs the Android SDK command-line tools + (`sdkmanager`/`avdmanager`), located via `ANDROID_SDK_ROOT`/`ANDROID_HOME` (or the per-OS + default: `%LOCALAPPDATA%\Android\Sdk` on Windows, `~/Library/Android/sdk` on macOS, + `~/Android/Sdk` on Linux); it installs the preview system image itself on first run. ## Steps @@ -38,8 +42,7 @@ Run these, stopping at the first failure: ./gradlew :app:compileDebugAndroidTestKotlin ./gradlew :app:lintDebug ./gradlew :app:ktlintCheck :app:detekt -./gradlew :app:api35DebugAndroidTest # top-of-matrix emulator E2E (2nd-highest stable level) -./gradlew :app:api36DebugAndroidTest # top-of-matrix emulator E2E (highest stable level) +python3 .claude/skills/preflight/local_instrumented.py [,,...] # local instrumented/E2E, cold-boot (no GMD) python3 .claude/skills/preflight/api37_e2e.py # api37 preview E2E (hand-provisioned; on Windows: py or python) ``` @@ -54,12 +57,19 @@ merge gate even when the build and lint are green. Add `--continue` to any comma `:app:ktlintCheck :app:detekt --continue`) to collect every failure in one pass instead of stopping at the first. -The three E2E steps run last because they are the slowest. `api35DebugAndroidTest` and -`api36DebugAndroidTest` run the full instrumented/E2E suite on `api35`, then `api36` — the top -two stable levels in the E2E matrix — each via its own Gradle Managed Device, which Gradle -provisions, boots, and tears down automatically. +The two E2E steps run last because they are the slowest, and they run through **cross-platform +Python scripts, not Gradle Managed Devices (GMD)**. GMD's `apiXXDebugAndroidTest` tasks fail +locally on this box — GMD's AVD-snapshot step times out under the AEHD 2.2 hypervisor +(`AvdSnapshotHandler$EmulatorSnapshotCannotCreatedException`), cycling for hours — so preflight +does **not** call them (issue #269/#281). `local_instrumented.py` instead cold-boots one existing +AVD by hand with `-no-snapshot` (the exact `connectedDebugAndroidTest` technique CI and +`api37_e2e.py` use), runs `:app:connectedDebugAndroidTest` filtered to the instrumented class(es) +you pass, then tears the emulator down and verifies no orphaned `qemu` process is left behind +(exit 3 if one survives). Pass the instrumented/E2E class(es) you actually changed +(comma-separated, no spaces) — the full ~114-test suite tends to wedge mid-run locally, so +targeted runs are deliberate; the whole suite across every API level is CI's job. -`api37_e2e.py` then runs the same suite on the **API 37 preview** emulator. API 37 has no Gradle +`api37_e2e.py` then runs the instrumented/E2E suite on the **API 37 preview** emulator. API 37 has no Gradle Managed Device — its only published system image is the nonstandard `android-37.0` / `google_apis_ps16k` pairing, which neither `ManagedVirtualDevice`'s `apiLevel` (Int) nor `apiPreview` (codename) DSL resolves (see the comment above `testOptions.managedDevices` in @@ -72,26 +82,27 @@ match `e2e-preview` with one deliberate local exception: the **GPU mode**. CI us local run uses `-gpu auto-no-window`, which renders on the host GPU — faster, and the mode that boots cleanly on a dev machine. -All three levels are the E2E that preflight runs locally and all three must pass; CI fans the -same suite out across the whole matrix (API 29–36 in `e2e`, plus API 37 in `e2e-preview`). Keep -`api35DebugAndroidTest` / `api36DebugAndroidTest` in lockstep with the top of the managed-device -list in `app/build.gradle.kts`, and keep `api37_e2e.py` in lockstep with the `e2e-preview` job in -`.github/workflows/ci.yml` (same image string + emulator flags, apart from the intentional GPU-mode -difference noted above) — when a newer API level is added there, run the new top levels instead. +Both E2E steps are the E2E that preflight runs locally and both must pass; CI then fans the full +instrumented/E2E suite out across the whole matrix (API 29–36 in `e2e`, plus API 37 in +`e2e-preview`). Keep `local_instrumented.py` pointed at the instrumented class(es) you changed, +and keep `api37_e2e.py` in lockstep with the `e2e-preview` job in `.github/workflows/ci.yml` +(same image string + emulator flags, apart from the intentional GPU-mode difference noted above). +The local gate no longer runs the GMD `apiXXDebugAndroidTest` tasks (they are unusable locally — +see above); full multi-API coverage stays CI's job. ## Reporting -- If everything passes, say so plainly (e.g. "preflight green: build, unit tests, lint, ktlint, detekt, api35+api36+api37 E2E"). +- If everything passes, say so plainly (e.g. "preflight green: build, unit tests, lint, ktlint, detekt, local + api37 E2E"). - On failure, surface the actual Gradle error and point at the relevant report: - unit tests → `app/build/reports/tests/testDebugUnitTest/` - lint → `app/build/reports/lint-results-debug.html` - ktlint → `app/build/reports/ktlint/` (per source set, e.g. `ktlintTestSourceSetCheck/`) - detekt → `app/build/reports/detekt/` - - api35/api36 E2E → `app/build/reports/androidTests/managedDevice/` (per-device HTML, e.g. - `.../api35/`, `.../api36/`) - - api37 E2E → `app/build/reports/androidTests/connected/` (the `connectedDebugAndroidTest` - report the script drives); the emulator's own boot log is at the temp path the script prints. -- Run the **top two stable-API** levels (`api35DebugAndroidTest` + `api36DebugAndroidTest`) plus - the **API 37 preview** via `api37_e2e.py`; the rest of the multi-API matrix (API 29–34) stays - CI's job. If the host has no accelerated emulator and a device cannot boot (see the hypervisor - note in Preconditions), report that E2E could not run rather than treating the gate as green. + - local + api37 E2E → `app/build/reports/androidTests/connected/` (the `connectedDebugAndroidTest` + report both scripts drive — the later run overwrites the earlier); each emulator's own boot log + is at the temp path the script prints (`local_instrumented.py` also exits **3** if it leaves an + orphaned `qemu`, **4** if the emulator never booted). +- Run the instrumented class(es) you changed via `local_instrumented.py`, plus the **API 37 + preview** via `api37_e2e.py`; the full multi-API matrix (API 29–37) stays CI's job. If the host + has no accelerated emulator and a device cannot boot (see the hypervisor note in Preconditions), + report that E2E could not run rather than treating the gate as green. diff --git a/.claude/skills/preflight/local_instrumented.README.md b/.claude/skills/preflight/local_instrumented.README.md index 44ece27..c94b69c 100644 --- a/.claude/skills/preflight/local_instrumented.README.md +++ b/.claude/skills/preflight/local_instrumented.README.md @@ -1,25 +1,28 @@ -# `local_instrumented.sh` — reliable local instrumented/E2E runs +# `local_instrumented.py` — reliable local instrumented/E2E runs A helper for running LibreMail's instrumented / E2E tests **locally** without Gradle -Managed Devices (GMD). Companion to `api37_e2e.py`; born from issue #269. +Managed Devices (GMD). Cross-platform, pure Python 3 standard library (Windows / Linux / +macOS). Companion to `api37_e2e.py`; born from issue #269, ported from bash to Python in +issue #281 so it runs the same on the Windows primary dev box and on \*nix — no Git Bash, +no `jq`, no `taskkill`-vs-`kill` gaps. ## Usage ```bash -# in Git Bash, from anywhere — invoke the script by path: -.claude/skills/preflight/local_instrumented.sh org.libremail.ui.compose.ComposeScreenE2ETest +# from anywhere — invoke the script by path (Windows: use `py` or `python`): +python .claude/skills/preflight/local_instrumented.py org.libremail.ui.compose.ComposeScreenE2ETest # multiple classes (comma-separated, no spaces): -.claude/skills/preflight/local_instrumented.sh org.libremail.a.FooTest,org.libremail.b.BarTest +python .claude/skills/preflight/local_instrumented.py org.libremail.a.FooTest,org.libremail.b.BarTest ``` The script is CWD-independent: it resolves its own repo/worktree root from its script -location (three directories up from `.claude/skills/preflight`) and `cd`s there before -invoking gradlew, so it always builds *that* tree's `:app` — never whatever tree your -shell happens to be sitting in. This matters most when you have several worktrees -checked out side by side; run the copy of this script that lives inside the worktree you -want to test, regardless of your current directory (issue #284). +location (three directories up from `.claude/skills/preflight`) and runs gradlew there, so +it always builds *that* tree's `:app` — never whatever tree your shell happens to be +sitting in. This matters most when you have several worktrees checked out side by side; run +the copy of this script that lives inside the worktree you want to test, regardless of your +current directory (issue #284). It cold-boots **one** emulator (`-no-snapshot`, no GMD), waits for `sys.boot_completed`, runs `:app:connectedDebugAndroidTest` filtered to the class(es) you pass, then tears the @@ -35,16 +38,25 @@ emulator down and verifies no orphaned `qemu` process is left behind (exit **3** - **Keep runs targeted.** The full ~114-test suite tends to wedge mid-run on this machine; small, targeted class sets do not. That's why the script requires an explicit class list — run only what you changed. The full matrix is CI's job. -- **Emulator hygiene is mandatory.** A hung `adb emu kill` leaves a detached - `qemu-system-x86_64-headless.exe`; accumulated orphans have frozen this machine. The - script force-kills stragglers before booting and after tearing down, and fails loudly if - a zombie survives. +- **Emulator hygiene is mandatory.** A hung `adb emu kill` leaves a detached qemu VM + (`qemu-system-x86_64-headless.exe` on Windows, a `qemu-system-*` process on \*nix); + accumulated orphans have frozen this machine. The script force-kills stragglers before + booting and after tearing down, and fails loudly (exit 3) if a zombie survives. The + orphan-kill is abstracted per-OS (`taskkill /F /IM …` on Windows, `pkill -f qemu-system` + on \*nix), and teardown always runs — even on Ctrl-C / error / SIGTERM (try/finally + + atexit + SIGINT/SIGTERM handlers). -See the header comment of `local_instrumented.sh` for the full rationale, requirements, and -the `LOCAL_INSTRUMENTED_*` environment overrides (AVD name, JDK home, boot timeout, …). +Exit codes: **0** pass · **2** usage/precondition failure · **3** a qemu zombie survived +teardown · **4** emulator never booted · any other non-zero = `connectedDebugAndroidTest`'s +own test-failure exit code. + +See the module docstring at the top of `local_instrumented.py` for the full rationale, +requirements, and the `LOCAL_INSTRUMENTED_*` environment overrides (AVD name, JDK home, +boot timeout, …). ## Requirements -Git Bash; Android SDK `emulator` + `adb` on `PATH`; a JDK **17–21** (AGP 9.2 fails on 25+ — -the script pins `JAVA_HOME` to a known JDK 21, overridable via `LOCAL_INSTRUMENTED_JDK`); and -a free hardware hypervisor (shut down VirtualBox / other VMs first). +`python3` (Windows: `py`/`python`); Android SDK `emulator` + `adb` on `PATH`; a JDK +**17–21** (AGP 9.2 fails on 25+ — the script pins `JAVA_HOME` to a known JDK 21, overridable +via `LOCAL_INSTRUMENTED_JDK`); and a free hardware hypervisor (shut down VirtualBox / other +VMs first). diff --git a/.claude/skills/preflight/local_instrumented.py b/.claude/skills/preflight/local_instrumented.py new file mode 100644 index 0000000..b31eeab --- /dev/null +++ b/.claude/skills/preflight/local_instrumented.py @@ -0,0 +1,451 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-or-later +"""local_instrumented.py -- reliable LOCAL instrumented / E2E test runner for LibreMail. + +Usage: local_instrumented.py [,,...] +Example: + python .claude/skills/preflight/local_instrumented.py \ + org.libremail.ui.compose.ComposeScreenE2ETest + python .claude/skills/preflight/local_instrumented.py \ + org.libremail.ui.compose.ComposeScreenE2ETest,org.libremail.ui.compose.RecipientChipTest + +Cross-platform (Windows / Linux / macOS), pure standard library. Companion to +``api37_e2e.py``; ported from the original ``local_instrumented.sh`` (issue #281) so the +helper runs the same on the Windows primary dev box and on *nix -- no Git Bash, no ``jq``, +no ``taskkill`` vs ``kill`` portability gaps. + +WHY THIS SCRIPT EXISTS (issue #269) +------------------------------------ +On this machine (Windows + the AEHD 2.2 hypervisor) the Gradle Managed Device (GMD) +instrumented tasks -- ``apiXXDebugAndroidTest`` -- FAIL during setup. GMD tries to +save/load an AVD *snapshot* and AEHD 2.2 cannot complete it: + + AvdSnapshotHandler$EmulatorSnapshotCannotCreatedException: Snapshot creation timed out + +GMD retries the snapshot ~5x, rebooting the AVD each time -- that endless reboot is the +"cycling" that eats hours. The emulator ITSELF is healthy (8 GB RAM, sys.boot_completed=1, +shell-responsive); only GMD's snapshot step is broken. So every LOCAL GMD task is affected: +the coverage lanes and the /preflight api35/api36 steps. CI is unaffected -- it uses +reactivecircus/android-emulator-runner + ``connectedDebugAndroidTest``, never GMD. + +THE RELIABLE LOCAL PATH (this script): + Cold-boot ONE emulator by hand with ``-no-snapshot`` (no GMD, no snapshot machinery), + then run ``:app:connectedDebugAndroidTest`` -- the exact technique CI and ``api37_e2e.py`` + already use. We reuse a GMD-provisioned AVD by name so we don't re-download a system + image; GMD re-provisions its own copy on its next run, so the ``-wipe-data`` cold boot + here does not disturb it. + +KEEP RUNS TARGETED -- THE ~114-TEST MID-SUITE WEDGE +--------------------------------------------------- +Running the WHOLE instrumented suite (~114 tests) via ``connectedDebugAndroidTest`` on this +box tends to wedge partway through -- the emulator stops making progress mid-run. Small, +targeted class sets do NOT hit that wedge. That is why this helper takes an explicit +``[,...]`` argument and filters the run with +``-Pandroid.testInstrumentationRunnerArguments.class=...`` instead of running everything. +Run the class(es) you actually changed; do not use this to run the full suite (that is +CI's / preflight's job across the API matrix). + +FREEZE / HYGIENE RATIONALE -- WHY THE ORPHAN-KILL + TEARDOWN VERIFY ARE MANDATORY +-------------------------------------------------------------------------------- +A hung ``adb emu kill`` (or an interrupted run) leaves a detached qemu VM process behind +(``qemu-system-x86_64-headless.exe`` on Windows; a ``qemu-system-*`` process on *nix). +These orphans do not show up in ``adb devices``, they keep holding the hypervisor + RAM, +and accumulated orphans have FROZEN this machine outright. So this script: + * PREAMBLE -- force-kills any pre-existing qemu/emulator processes and resets the adb + server BEFORE booting, so we always start from a clean slate. + * TEARDOWN -- ``adb emu kill``, kill the launcher we spawned, then re-check for ANY + surviving emulator/qemu process and force-kill it (the ``-no-window`` emulator + can leave a sibling ``emulator.exe`` that briefly outlives the qemu VM). Teardown + runs even on Ctrl-C / error / SIGTERM (try/finally + atexit + SIGINT/SIGTERM + handlers) and is idempotent. + * VERIFY -- if a qemu process is STILL alive after the force-kill, the script exits + non-zero (code 3) so the leak is never silently ignored. +Never leave an emulator running after this script; if it exits 3, hunt the zombie down by +hand (Windows: ``tasklist | findstr qemu`` then ``taskkill /F /IM +qemu-system-x86_64-headless.exe``; *nix: ``pgrep -fa qemu-system`` then ``pkill -f +qemu-system``). + +CROSS-PLATFORM PROCESS KILL +--------------------------- +Listing and force-killing the emulator/qemu processes is abstracted per-OS (see +``list_procs`` / ``force_kill``): Windows uses ``tasklist`` + ``taskkill /F /IM ``; +*nix uses ``ps ax`` + ``pkill -f qemu-system`` (alongside the graceful ``adb emu kill``). +The qemu VM is the freeze-causing orphan on every platform. + +EXIT CODES (preserved from local_instrumented.sh) + 0 tests passed + 2 usage / precondition failure + 3 a qemu zombie survived teardown -- clean it up by hand before the next run + 4 emulator never reached sys.boot_completed + connectedDebugAndroidTest's own non-zero exit code (test failures) + +REQUIREMENTS + * Android SDK ``emulator`` + ``adb`` on PATH. + * A JDK 17-21 for the Gradle daemon -- AGP 9.2 fails on JDK 25+. This script pins + JAVA_HOME to a known JDK 21 (override with LOCAL_INSTRUMENTED_JDK) because the ambient + JAVA_HOME on the primary box points at JDK 25. + * A free hardware hypervisor (VT-x/WHPX/AEHD/KVM/HVF). Shut down VirtualBox / other VMs + first or the AVD hangs at 0% CPU and never reaches sys.boot_completed. + +Overridable via environment (defaults target the primary Windows dev box): + LOCAL_INSTRUMENTED_AVD AVD name to boot (dev36_google_apis_x86_64_Pixel_2) + ANDROID_AVD_HOME AVD home dir (C:/Users/jasonross/.android/avd/gradle-managed) + LOCAL_INSTRUMENTED_JDK JDK 17-21 home (Eclipse Adoptium jdk-21.0.11.10-hotspot) + LOCAL_INSTRUMENTED_SERIAL adb serial (emulator-5554) + LOCAL_INSTRUMENTED_BOOT_TIMEOUT boot wait seconds (300) +""" + +from __future__ import annotations + +import argparse +import atexit +import os +import shutil +import signal +import subprocess +import sys +import tempfile +import time +from pathlib import Path + +IS_WINDOWS = os.name == "nt" + +# ---- configuration (env-overridable; defaults are correct for the primary dev box) ------ +AVD_NAME = os.environ.get("LOCAL_INSTRUMENTED_AVD", "dev36_google_apis_x86_64_Pixel_2") +AVD_HOME = os.environ.get("ANDROID_AVD_HOME", "C:/Users/jasonross/.android/avd/gradle-managed") +JDK_HOME = os.environ.get( + "LOCAL_INSTRUMENTED_JDK", "C:/Program Files/Eclipse Adoptium/jdk-21.0.11.10-hotspot" +) +SERIAL = os.environ.get("LOCAL_INSTRUMENTED_SERIAL", "emulator-5554") +BOOT_TIMEOUT = int(os.environ.get("LOCAL_INSTRUMENTED_BOOT_TIMEOUT", "300")) + +# Windows qemu/emulator image names (see FREEZE / HYGIENE above). The ``-headless`` variant is +# what a ``-no-window`` emulator launches; the plain qemu name is swept too, belt-and-suspenders. +QEMU_IMAGE = "qemu-system-x86_64-headless.exe" +QEMU_IMAGE_ALT = "qemu-system-x86_64.exe" +EMULATOR_IMAGE = "emulator.exe" + +REPO_ROOT = Path(__file__).resolve().parents[3] # .claude/skills/preflight -> repo root +GRADLEW = REPO_ROOT / ("gradlew.bat" if IS_WINDOWS else "gradlew") +EMU_LOG = Path(tempfile.gettempdir()) / "libremail-local-instrumented-emulator.log" + + +class _RunState: + """Mutable run state shared by main(), teardown(), the atexit hook and the signal + handlers -- mirrors the bash globals EMU_PID / TEST_EXIT / ZOMBIE / TEARDOWN_DONE.""" + + def __init__(self) -> None: + self.proc: subprocess.Popen | None = None + self.test_exit = 1 + self.zombie = False + self.teardown_done = False + + +_STATE = _RunState() + + +def log(msg: str) -> None: + print(f"\n=== {msg} ===") + + +def warn(msg: str) -> None: + print(f"WARNING: {msg}", file=sys.stderr) + + +def die(msg: str) -> None: + """Print an error and exit 2 (usage / precondition failure). Called before the teardown + backstops are armed, so nothing has booted and there is nothing to tear down.""" + print(f"ERROR: {msg}", file=sys.stderr) + sys.exit(2) + + +def cmd(tool: str, *args: str) -> list[str]: + """Build an argv list, wrapping Windows ``.bat``/``.cmd`` launchers (e.g. gradlew.bat) + through ``cmd /c`` -- matching api37_e2e.py. ``.exe`` tools pass through unchanged.""" + if IS_WINDOWS and tool.lower().endswith((".bat", ".cmd")): + return ["cmd", "/c", tool, *args] + return [tool, *args] + + +def _run_quiet(argv: list[str]) -> None: + """Run a command, discarding output and swallowing any error -- teardown/kill helpers + must always make progress.""" + try: + subprocess.run(argv, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False) + except (OSError, subprocess.SubprocessError): + pass + + +def _run_capture(argv: list[str]) -> str: + try: + return subprocess.run(argv, capture_output=True, text=True, check=False).stdout or "" + except (OSError, subprocess.SubprocessError): + return "" + + +def list_procs(*needles: str) -> str: + """Return the lines of currently-running processes whose name/command line contains any + of ``needles`` (case-insensitive); empty string if none. Cross-platform stand-in for the + .sh's ``tasklist | grep``: ``tasklist`` on Windows, ``ps ax`` on *nix.""" + out = _run_capture(["tasklist"] if IS_WINDOWS else ["ps", "ax"]) + lowered = [n.lower() for n in needles] + return "\n".join(ln for ln in out.splitlines() if any(n in ln.lower() for n in lowered)) + + +def list_qemu() -> str: + return list_procs("qemu") + + +def list_emu_procs() -> str: + return list_procs("qemu", "emulator") + + +def force_kill(win_images: list[str], nix_patterns: list[str]) -> None: + """Best-effort force-kill. Windows: ``taskkill /F /IM ...``. *nix: ``pkill -f + `` per pattern. Never raises -- teardown must always make progress.""" + if IS_WINDOWS: + argv = ["taskkill", "/F"] + for image in win_images: + argv += ["/IM", image] + _run_quiet(argv) + else: + for pattern in nix_patterns: + _run_quiet(["pkill", "-f", pattern]) + + +def tail(path: Path, lines: int = 40) -> None: + try: + with open(path, "r", errors="replace") as handle: + content = handle.readlines()[-lines:] + print("".join(content), file=sys.stderr, end="") + except OSError: + pass + + +def teardown() -> None: + """Kill the emulator and verify no orphaned emulator/qemu process remains. Idempotent -- + safe to call from the finally block, the atexit hook and the signal handlers (mirrors the + .sh TEARDOWN_DONE guard). Sets _STATE.zombie if a *qemu* process survives the force-kill -- + the machine-freezing case (exit 3).""" + if _STATE.teardown_done: + return + _STATE.teardown_done = True + + log("Teardown: killing emulator and verifying no orphaned emulator/qemu remains") + adb = shutil.which("adb") + if adb: + _run_quiet(cmd(adb, "-s", SERIAL, "emu", "kill")) + time.sleep(2) + + # Belt-and-suspenders: kill the emulator launcher process we started, if still alive. + proc = _STATE.proc + if proc is not None and proc.poll() is None: + proc.terminate() + try: + proc.wait(timeout=1) + except subprocess.TimeoutExpired: + proc.kill() + + # Reap any lingering emulator/qemu process, then verify. Unlike the original .sh -- which + # swept qemu ONLY -- we also force-kill the emulator *launcher* image: on Windows the + # ``-no-window`` emulator spawns a sibling ``emulator.exe`` that is NOT the Popen child we + # tracked and outlives both it and the qemu VM by a few seconds, so a qemu-only sweep + # returns while it is still shutting down -- an orphan the freeze-safety rule forbids. So we + # trigger on any emulator-or-qemu survivor and taskkill the launcher too. + if list_emu_procs(): + warn("emulator/qemu still present after 'adb emu kill'; force-killing:") + print(list_emu_procs(), file=sys.stderr) + force_kill([QEMU_IMAGE, QEMU_IMAGE_ALT, EMULATOR_IMAGE], ["qemu-system"]) + time.sleep(2) + # A surviving QEMU is the machine-freezing zombie (exit 3); a stray launcher is not. + remaining = list_qemu() + if remaining: + warn("qemu ZOMBIE survived teardown -- kill it by hand or the machine may freeze:") + print(remaining, file=sys.stderr) + _STATE.zombie = True + + if adb: + _run_quiet(cmd(adb, "kill-server")) + + +def orphan_kill_preamble(adb: str) -> None: + """Force-kill any pre-existing qemu/emulator processes and reset the adb server, so we + always cold-boot from a clean slate.""" + log("Orphan-kill preamble: ensuring a clean slate before boot") + existing = list_emu_procs() + if existing: + warn("Pre-existing emulator/qemu processes found -- force-killing them first:") + print(existing, file=sys.stderr) + force_kill([QEMU_IMAGE, EMULATOR_IMAGE], ["qemu-system"]) + time.sleep(2) + else: + print("No pre-existing qemu/emulator processes.") + _run_quiet(cmd(adb, "kill-server")) + _run_quiet(cmd(adb, "start-server")) + + +def start_emulator(emulator: str) -> subprocess.Popen: + """Cold-boot ONE emulator by hand (no GMD, no snapshot), logging to EMU_LOG. Records the + launcher process in _STATE so teardown can reap it even if we are interrupted next.""" + log(f"Cold-booting @{AVD_NAME} (no GMD, no snapshot); log -> {EMU_LOG}") + flags = [ + f"@{AVD_NAME}", + "-no-window", "-no-snapshot", "-no-boot-anim", "-no-audio", + "-gpu", "auto-no-window", "-cores", "8", "-wipe-data", + ] + logf = open(EMU_LOG, "wb") # noqa: SIM115 - handed to the child; parent copy closed below + try: + proc = subprocess.Popen(cmd(emulator, *flags), stdout=logf, stderr=subprocess.STDOUT) + finally: + logf.close() # the child inherited its own fd; the parent's copy is no longer needed + _STATE.proc = proc + print(f"emulator launcher pid={proc.pid}") + return proc + + +def wait_for_boot(adb: str, proc: subprocess.Popen, timeout: int) -> bool: + """Poll ``adb get-state`` + ``getprop sys.boot_completed`` until the emulator is up, or the + launcher dies, or ``timeout`` seconds elapse. Mirrors the .sh boot loop.""" + print(f"Waiting up to {timeout}s for sys.boot_completed on {SERIAL}...") + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + if proc.poll() is not None: + warn("emulator process exited during boot; last log lines:") + tail(EMU_LOG, 40) + return False + state = _run_capture(cmd(adb, "-s", SERIAL, "get-state")).strip() + if state == "device": + booted = _run_capture( + cmd(adb, "-s", SERIAL, "shell", "getprop", "sys.boot_completed") + ).strip() + if booted == "1": + return True + time.sleep(3) + return False + + +def dismiss_keyguard(adb: str) -> None: + # Dismiss the keyguard (mirrors CI + api37_e2e.py). Best-effort: a cold -wipe-data boot + # rarely needs it, and the input service can lose a race right after boot. + _run_quiet(cmd(adb, "-s", SERIAL, "shell", "input", "keyevent", "82")) + + +def run_tests(test_classes: str) -> int: + """Run :app:connectedDebugAndroidTest filtered to ``test_classes`` from the repo root + (JAVA_HOME / ANDROID_AVD_HOME are already in the environment).""" + log(f"Running :app:connectedDebugAndroidTest for: {test_classes}") + print(f"JAVA_HOME={os.environ.get('JAVA_HOME', '')}") + return subprocess.run( + cmd( + str(GRADLEW), + ":app:connectedDebugAndroidTest", + f"-Pandroid.testInstrumentationRunnerArguments.class={test_classes}", + "--stacktrace", + ), + cwd=str(REPO_ROOT), + check=False, + ).returncode + + +def main() -> int: + parser = argparse.ArgumentParser( + prog="local_instrumented.py", + formatter_class=argparse.RawDescriptionHelpFormatter, + description=( + "Cold-boot ONE emulator (no GMD, no snapshot) and run " + ":app:connectedDebugAndroidTest filtered to the given instrumented test class(es)." + ), + epilog=( + "Keep the class set small and targeted -- the full ~114-test suite tends to wedge\n" + "mid-run on this box (see the module docstring). The full matrix is CI's job.\n" + "Example:\n" + " python .claude/skills/preflight/local_instrumented.py \\\n" + " org.libremail.ui.compose.ComposeScreenE2ETest,org.libremail.ui.compose.RecipientChipTest" + ), + ) + parser.add_argument( + "test_classes", + metavar="TEST_CLASSES", + help=( + "Comma-separated fully-qualified instrumented test class(es), no spaces " + "(e.g. org.libremail.a.FooTest,org.libremail.b.BarTest)." + ), + ) + args = parser.parse_args() + + # ---- preconditions (before arming teardown; nothing has booted yet) ------------------ + emulator = shutil.which("emulator") + adb = shutil.which("adb") + if not emulator: + die("emulator not on PATH (install Android SDK emulator).") + if not adb: + die("adb not on PATH (install Android SDK platform-tools).") + kill_tool = "taskkill" if IS_WINDOWS else "pkill" + if not shutil.which(kill_tool): + die(f"{kill_tool} not found -- required to force-kill orphaned emulator/qemu processes.") + if not GRADLEW.is_file(): + die(f"gradlew not found at {GRADLEW}.") + if not os.path.isdir(JDK_HOME): + die(f"JDK 17-21 not found at '{JDK_HOME}'. Set LOCAL_INSTRUMENTED_JDK.") + if not os.path.isfile(os.path.join(AVD_HOME, AVD_NAME + ".ini")): + die( + f"AVD '{AVD_NAME}' not found under '{AVD_HOME}'. " + "Set LOCAL_INSTRUMENTED_AVD / ANDROID_AVD_HOME. " + "(GMD AVDs are created by any local apiXXDebugAndroidTest run.)" + ) + + # Move gradlew's working dir to this tree's repo root (below) and pin JAVA_HOME/AVD home, + # exactly like the .sh -- the ambient JAVA_HOME on this box points at JDK 25 (AGP-incompatible). + os.environ["JAVA_HOME"] = JDK_HOME + os.environ["ANDROID_AVD_HOME"] = AVD_HOME + + # ---- arm teardown backstops BEFORE touching the emulator ----------------------------- + # try/finally is the primary path; atexit covers sys.exit()/unhandled-exception exits; the + # signal handlers make SIGINT/SIGTERM tear down too (Python does not raise on SIGTERM by + # default). teardown() is idempotent, so firing from several paths is safe (mirrors the + # .sh's ``trap teardown EXIT INT TERM`` + TEARDOWN_DONE guard). + atexit.register(teardown) + + def _signal_teardown(signum: int, _frame: object) -> None: + teardown() + sys.exit(128 + signum) + + signal.signal(signal.SIGINT, _signal_teardown) + if hasattr(signal, "SIGTERM"): + signal.signal(signal.SIGTERM, _signal_teardown) + + boot_failed = False + try: + orphan_kill_preamble(adb) + proc = start_emulator(emulator) + if wait_for_boot(adb, proc, BOOT_TIMEOUT): + print("Emulator booted.") + dismiss_keyguard(adb) + _STATE.test_exit = run_tests(args.test_classes) + else: + warn(f"Emulator did not reach sys.boot_completed within {BOOT_TIMEOUT}s.") + tail(EMU_LOG, 40) + boot_failed = True + finally: + teardown() + + if boot_failed: + return 4 + if _STATE.zombie: + warn( + "Exiting 3: a qemu zombie was left behind (see above) -- " + "clean it up before the next run." + ) + return 3 + if _STATE.test_exit != 0: + warn( + f"connectedDebugAndroidTest failed (exit {_STATE.test_exit}). " + "Report: app/build/reports/androidTests/connected/" + ) + return _STATE.test_exit + log(f"PASS -- instrumented tests green for: {args.test_classes}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/skills/preflight/local_instrumented.sh b/.claude/skills/preflight/local_instrumented.sh deleted file mode 100755 index 328ac63..0000000 --- a/.claude/skills/preflight/local_instrumented.sh +++ /dev/null @@ -1,253 +0,0 @@ -#!/usr/bin/env bash -# SPDX-License-Identifier: GPL-3.0-or-later -# -# local_instrumented.sh — reliable LOCAL instrumented / E2E test runner for LibreMail. -# -# Usage: local_instrumented.sh [,,...] -# Example: -# .claude/skills/preflight/local_instrumented.sh \ -# org.libremail.ui.compose.ComposeScreenE2ETest -# .claude/skills/preflight/local_instrumented.sh \ -# org.libremail.ui.compose.ComposeScreenE2ETest,org.libremail.ui.compose.RecipientChipTest -# -# ============================================================================= -# WHY THIS SCRIPT EXISTS (issue #269) -# ----------------------------------------------------------------------------- -# On this machine (Windows + the AEHD 2.2 hypervisor) the Gradle Managed Device -# (GMD) instrumented tasks — `apiXXDebugAndroidTest` — FAIL during setup. GMD tries -# to save/load an AVD *snapshot* and AEHD 2.2 cannot complete it: -# -# AvdSnapshotHandler$EmulatorSnapshotCannotCreatedException: Snapshot creation timed out -# -# GMD retries the snapshot ~5x, rebooting the AVD each time — that endless reboot is -# the "cycling" that eats hours. The emulator ITSELF is healthy (8 GB RAM, -# `sys.boot_completed=1`, shell-responsive); only GMD's snapshot step is broken. So -# every LOCAL GMD task is affected: coverage lanes 3/5 (#248/#250) and the /preflight -# api35/api36 steps (#266). CI is unaffected — it uses reactivecircus/android-emulator-runner -# + `connectedDebugAndroidTest`, never GMD. -# -# THE RELIABLE LOCAL PATH (this script): -# Cold-boot ONE emulator by hand with `-no-snapshot` (no GMD, no snapshot machinery), -# then run `:app:connectedDebugAndroidTest` — the exact technique CI and -# `api37_e2e.py` already use. We reuse a GMD-provisioned AVD by name so we don't have -# to re-download a system image; GMD re-provisions its own copy on its next run, so -# the `-wipe-data` cold boot here does not disturb it. -# -# ============================================================================= -# KEEP RUNS TARGETED — THE ~114-TEST MID-SUITE WEDGE -# ----------------------------------------------------------------------------- -# Running the WHOLE instrumented suite (~114 tests) via `connectedDebugAndroidTest` -# on this box tends to wedge partway through — the emulator stops making progress -# mid-run. Small, targeted class sets do NOT hit that wedge. That is why this helper -# takes an explicit `[,...]` argument and filters the run -# with `-Pandroid.testInstrumentationRunnerArguments.class=...` instead of running -# everything. Run the class(es) you actually changed; do not use this to run the full -# suite (that is CI's / preflight's job across the API matrix). -# -# ============================================================================= -# FREEZE / HYGIENE RATIONALE — WHY THE ORPHAN-KILL + TEARDOWN VERIFY ARE MANDATORY -# ----------------------------------------------------------------------------- -# A hung `adb emu kill` (or an interrupted run) leaves a detached -# `qemu-system-x86_64-headless.exe` behind. These orphans do not show up in -# `adb devices`, they keep holding the hypervisor + RAM, and accumulated orphans have -# FROZEN this machine outright. So this script: -# * PREAMBLE — force-kills any pre-existing qemu/emulator processes and resets the -# adb server BEFORE booting, so we always start from a clean slate. -# * TEARDOWN — `adb emu kill`, then re-checks `tasklist` for qemu and force-kills any -# survivor. The teardown runs even on Ctrl-C / error (EXIT/INT/TERM trap). -# * VERIFY — if a qemu process is STILL alive after the force-kill, the script exits -# non-zero (code 3) so the leak is never silently ignored. -# Never leave an emulator running after this script; if it exits 3, hunt the zombie -# down by hand (`tasklist | grep -i qemu`; `taskkill //F //IM qemu-system-x86_64-headless.exe`). -# -# ============================================================================= -# REQUIREMENTS -# * Git Bash (this is a bash script; it shells out to Windows `tasklist`/`taskkill`). -# * Android SDK `emulator` + `adb` on PATH (SDK at C:\Android here). -# * A JDK 17–21 for the Gradle daemon — AGP 9.2 fails on JDK 25+. This script pins -# JAVA_HOME to a known JDK 21 (override with LOCAL_INSTRUMENTED_JDK) because the -# ambient JAVA_HOME on this box points at JDK 25. -# * A free hardware hypervisor (VT-x/WHPX/AEHD). Shut down VirtualBox / other VMs first -# or the AVD hangs at 0% CPU and never reaches sys.boot_completed. -# -# Overridable via environment (defaults target THIS machine): -# LOCAL_INSTRUMENTED_AVD AVD name to boot (dev36_google_apis_x86_64_Pixel_2) -# ANDROID_AVD_HOME AVD home dir (C:/Users/jasonross/.android/avd/gradle-managed) -# LOCAL_INSTRUMENTED_JDK JDK 17–21 home (Eclipse Adoptium jdk-21.0.11.10-hotspot) -# LOCAL_INSTRUMENTED_SERIAL adb serial (emulator-5554) -# LOCAL_INSTRUMENTED_BOOT_TIMEOUT boot wait seconds (300) -# ============================================================================= - -set -uo pipefail - -# ---- configuration (env-overridable; defaults are correct for this machine) ----------- -AVD_NAME="${LOCAL_INSTRUMENTED_AVD:-dev36_google_apis_x86_64_Pixel_2}" -AVD_HOME="${ANDROID_AVD_HOME:-C:/Users/jasonross/.android/avd/gradle-managed}" -JDK_HOME="${LOCAL_INSTRUMENTED_JDK:-C:/Program Files/Eclipse Adoptium/jdk-21.0.11.10-hotspot}" -SERIAL="${LOCAL_INSTRUMENTED_SERIAL:-emulator-5554}" -BOOT_TIMEOUT="${LOCAL_INSTRUMENTED_BOOT_TIMEOUT:-300}" -QEMU_IMAGE="qemu-system-x86_64-headless.exe" - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)" # .claude/skills/preflight -> repo root -GRADLEW="${REPO_ROOT}/gradlew" -EMU_LOG="${TMPDIR:-/tmp}/libremail-local-instrumented-emulator.log" - -EMU_PID="" -TEST_EXIT=1 -ZOMBIE=0 -TEARDOWN_DONE=0 - -log() { printf '\n=== %s ===\n' "$*"; } -warn() { printf 'WARNING: %s\n' "$*" >&2; } -die() { printf 'ERROR: %s\n' "$*" >&2; exit 2; } - -# All emulator/qemu processes Windows currently sees (empty string if none). -list_emu_procs() { tasklist 2>/dev/null | grep -iE 'qemu|emulator' || true; } -list_qemu() { tasklist 2>/dev/null | grep -i 'qemu' || true; } - -# ---- argument parsing ----------------------------------------------------------------- -TEST_CLASSES="${1:-}" -if [[ -z "${TEST_CLASSES}" ]]; then - cat >&2 <<'USAGE' -usage: local_instrumented.sh [,,...] - -Cold-boots ONE emulator (no GMD, no snapshot) and runs :app:connectedDebugAndroidTest -filtered to the given instrumented test class(es). Keep the set small and targeted — -see the header for the ~114-test mid-suite wedge. -USAGE - exit 2 -fi - -# ---- preconditions -------------------------------------------------------------------- -command -v emulator >/dev/null 2>&1 || die "emulator not on PATH (install Android SDK emulator)." -command -v adb >/dev/null 2>&1 || die "adb not on PATH (install Android SDK platform-tools)." -command -v tasklist >/dev/null 2>&1 || die "tasklist not found — this helper targets Windows/Git Bash." -[[ -f "${GRADLEW}" ]] || die "gradlew not found at ${GRADLEW}." -[[ -d "${JDK_HOME}" ]] || die "JDK 17-21 not found at '${JDK_HOME}'. Set LOCAL_INSTRUMENTED_JDK." -[[ -f "${AVD_HOME}/${AVD_NAME}.ini" ]] || \ - die "AVD '${AVD_NAME}' not found under '${AVD_HOME}'. Set LOCAL_INSTRUMENTED_AVD / ANDROID_AVD_HOME. - (GMD AVDs are created by any local apiXXDebugAndroidTest run.)" - -# Move into the resolved repo/worktree root before invoking gradlew. GRADLEW above is an -# absolute path, but the gradlew wrapper script picks the *project* to build from the -# process's current directory, not from its own script location — so without this `cd`, -# running this helper from a different tree (e.g. another worktree, or the main repo -# while iterating on a worktree's copy of this script) silently builds the CALLER's CWD -# tree instead of this one (issue #284). -cd "${REPO_ROOT}" || die "Could not cd to repo root '${REPO_ROOT}'." - -export JAVA_HOME="${JDK_HOME}" -export ANDROID_AVD_HOME="${AVD_HOME}" - -# ---- teardown: always runs (normal exit, error, or Ctrl-C) ---------------------------- -teardown() { - [[ "${TEARDOWN_DONE}" == "1" ]] && return 0 - TEARDOWN_DONE=1 - - log "Teardown: killing emulator and verifying no orphaned qemu remains" - adb -s "${SERIAL}" emu kill >/dev/null 2>&1 || true - sleep 2 - - # Belt-and-suspenders: kill the emulator launcher process we started, if still alive. - if [[ -n "${EMU_PID}" ]] && kill -0 "${EMU_PID}" 2>/dev/null; then - kill "${EMU_PID}" 2>/dev/null || true - sleep 1 - kill -9 "${EMU_PID}" 2>/dev/null || true - fi - - # Verify: any surviving qemu is a machine-freezing zombie — force-kill and re-check. - local remaining - remaining="$(list_qemu)" - if [[ -n "${remaining}" ]]; then - warn "qemu still present after 'adb emu kill'; force-killing:" - printf '%s\n' "${remaining}" >&2 - taskkill //F //IM "${QEMU_IMAGE}" >/dev/null 2>&1 || true - # Sweep any other stray qemu-system image name, too. - taskkill //F //IM "qemu-system-x86_64.exe" >/dev/null 2>&1 || true - sleep 2 - remaining="$(list_qemu)" - if [[ -n "${remaining}" ]]; then - warn "qemu ZOMBIE survived teardown — kill it by hand or the machine may freeze:" - printf '%s\n' "${remaining}" >&2 - ZOMBIE=1 - fi - fi - - adb kill-server >/dev/null 2>&1 || true -} -trap teardown EXIT INT TERM - -# ---- 1. orphan-kill preamble ---------------------------------------------------------- -log "Orphan-kill preamble: ensuring a clean slate before boot" -existing="$(list_emu_procs)" -if [[ -n "${existing}" ]]; then - warn "Pre-existing emulator/qemu processes found — force-killing them first:" - printf '%s\n' "${existing}" >&2 - taskkill //F //IM "${QEMU_IMAGE}" //IM "emulator.exe" >/dev/null 2>&1 || true - sleep 2 -else - echo "No pre-existing qemu/emulator processes." -fi -adb kill-server >/dev/null 2>&1 || true -adb start-server >/dev/null 2>&1 || true - -# ---- 2. cold-boot ONE emulator (no snapshot) ------------------------------------------ -log "Cold-booting @${AVD_NAME} (no GMD, no snapshot); log -> ${EMU_LOG}" -emulator "@${AVD_NAME}" \ - -no-window -no-snapshot -no-boot-anim -no-audio \ - -gpu auto-no-window -cores 8 -wipe-data \ - >"${EMU_LOG}" 2>&1 & -EMU_PID=$! -echo "emulator launcher pid=${EMU_PID}" - -echo "Waiting up to ${BOOT_TIMEOUT}s for sys.boot_completed on ${SERIAL}..." -deadline=$(( $(date +%s) + BOOT_TIMEOUT )) -booted=0 -while (( $(date +%s) < deadline )); do - if ! kill -0 "${EMU_PID}" 2>/dev/null; then - warn "emulator process exited during boot; last log lines:" - tail -n 40 "${EMU_LOG}" >&2 || true - break - fi - state="$(adb -s "${SERIAL}" get-state 2>/dev/null | tr -d '\r')" - if [[ "${state}" == "device" ]]; then - bc="$(adb -s "${SERIAL}" shell getprop sys.boot_completed 2>/dev/null | tr -d '\r\n ')" - if [[ "${bc}" == "1" ]]; then booted=1; break; fi - fi - sleep 3 -done - -if [[ "${booted}" != "1" ]]; then - warn "Emulator did not reach sys.boot_completed within ${BOOT_TIMEOUT}s." - tail -n 40 "${EMU_LOG}" >&2 || true - # teardown runs via the EXIT trap; surface a boot failure distinctly. - exit 4 -fi -echo "Emulator booted." - -# Dismiss the keyguard (mirrors CI + api37_e2e.py). Best-effort: a cold -wipe-data boot -# rarely needs it, and the input service can lose a race right after boot. -adb -s "${SERIAL}" shell input keyevent 82 >/dev/null 2>&1 || true - -# ---- 3. run the targeted instrumented tests ------------------------------------------- -log "Running :app:connectedDebugAndroidTest for: ${TEST_CLASSES}" -echo "JAVA_HOME=${JAVA_HOME}" -"${GRADLEW}" :app:connectedDebugAndroidTest \ - "-Pandroid.testInstrumentationRunnerArguments.class=${TEST_CLASSES}" \ - --stacktrace -TEST_EXIT=$? - -# ---- 4. teardown + verify, then exit -------------------------------------------------- -teardown - -if (( ZOMBIE != 0 )); then - warn "Exiting 3: a qemu zombie was left behind (see above) — clean it up before the next run." - exit 3 -fi -if (( TEST_EXIT != 0 )); then - warn "connectedDebugAndroidTest failed (exit ${TEST_EXIT}). Report: app/build/reports/androidTests/connected/" - exit "${TEST_EXIT}" -fi -log "PASS — instrumented tests green for: ${TEST_CLASSES}" -exit 0 diff --git a/CLAUDE.md b/CLAUDE.md index 1acc90b..e96e56a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,23 +28,34 @@ or via Gradle Managed Devices `./gradlew e2eGroupDebugAndroidTest` (whole matrix **Before treating a change as done**, run the fast CI gate: `assembleDebug` + `testDebugUnitTest` + `compileDebugAndroidTestKotlin` + `lintDebug` + `ktlintCheck` + -`detekt` + the top-of-matrix emulator E2E `api35DebugAndroidTest` + `api36DebugAndroidTest` + -the API 37 preview E2E via `python3 .claude/skills/preflight/api37_e2e.py` (the `/preflight` -skill does all of this). `compileDebugAndroidTestKotlin` compiles the `androidTest` source set +`detekt` + the local emulator E2E — the instrumented test class(es) you changed via +`python3 .claude/skills/preflight/local_instrumented.py ` + the API 37 preview E2E via +`python3 .claude/skills/preflight/api37_e2e.py` (the `/preflight` skill does all of this). +`compileDebugAndroidTestKotlin` compiles the `androidTest` source set that the static part of the gate skips, catching E2E/instrumented-test compile errors before they surface only in CI. `ktlintCheck`/`detekt` cover the `test`/`androidTest` source sets that `lintDebug` skips, so they catch style violations that would otherwise fail CI's Static analysis -gate. `api35DebugAndroidTest` and `api36DebugAndroidTest` run the instrumented/E2E suite on the -top two stable API levels in the E2E matrix, each via its own Gradle Managed Device (Gradle boots -and tears down each emulator automatically). API 37 (preview) has no Gradle Managed Device — its +gate. The local E2E does **not** use Gradle Managed Devices (`apiXXDebugAndroidTest`): GMD's +snapshot step fails locally under the AEHD 2.2 hypervisor. Instead `local_instrumented.py` +cold-boots one existing AVD by hand (no GMD, no snapshot) and runs `connectedDebugAndroidTest` +filtered to the class(es) you pass — run the ones you changed; the full ~114-test suite wedges +mid-run locally. API 37 (preview) has no Gradle Managed Device — its only image is the nonstandard `android-37.0` / `google_apis_ps16k` pairing (see the comment above `testOptions.managedDevices` in `app/build.gradle.kts`) — so `api37_e2e.py` hand-provisions it, mirroring CI's `e2e-preview` job (same image + emulator flags, except it uses host-GPU `-gpu auto-no-window` locally vs CI's headless `-gpu swiftshader_indirect`), boots it headless, -runs `connectedDebugAndroidTest`, and tears it down. Running all three levels locally is required; the -rest of the multi-API matrix (API 29–34) stays CI's job. Emulators need a free hardware +runs `connectedDebugAndroidTest`, and tears it down. Both local E2E steps are required; the +full multi-API matrix (API 29–37) stays CI's job. Emulators need a free hardware hypervisor (VT-x/WHPX), so shut down VirtualBox/other VMs first or the AVD hangs at 0% CPU. +**Dev scripts: prefer Python (stdlib).** Auxiliary dev / CI-helper scripts — like the preflight +E2E runners (`.claude/skills/preflight/local_instrumented.py`, `api37_e2e.py`) and +`.claude/hooks/check-spdx.py` — are written in **Python 3, standard library only**, for +cross-platform portability. The primary dev box is Windows, where bash-only helpers need Git Bash +and hit gaps (`jq` missing, `taskkill` vs `kill`, path/quoting). **Do not add new bash-only +(`.sh`) or PowerShell-only dev scripts**; write new helpers in Python (or extend the existing +ones). Scope is auxiliary tooling only — product code stays Kotlin and Gradle stays Kotlin DSL. + ## Build-config gotchas - **Built-in Kotlin (AGP 9.x).** Kotlin compilation is handled by AGP's built-in Kotlin; @@ -77,11 +88,11 @@ pulled in as a real dependency for unit tests because `android.jar`'s version is A change is not done until it ships with passing **unit tests** and **E2E/instrumented tests** that exercise the new or changed behaviour. Writing and committing that E2E/instrumented test is a required part of every task — and the test must actually **run and pass**, not merely -compile: preflight runs the top-of-matrix emulator E2E locally — `api35DebugAndroidTest` + -`api36DebugAndroidTest` (Gradle Managed Devices) plus the API 37 preview via -`api37_e2e.py` (hand-provisioned, mirroring CI's `e2e-preview` job) — and all three must be -green before the change is done. CI then runs the full multi-API matrix plus the API 37 preview -job. +compile: preflight runs the changed instrumented test class(es) on a locally cold-booted +emulator via `local_instrumented.py` (no Gradle Managed Devices — they fail locally) plus the +API 37 preview via `api37_e2e.py` (hand-provisioned, mirroring CI's `e2e-preview` job) — and both +must be green before the change is done. CI then runs the full multi-API matrix plus the API 37 +preview job. ## Repo etiquette