feat(scripts): device-testing perf harness
Add a cross-platform, standard-library-only Python package under scripts/device-testing/ that replicates LibreMail's on-device performance-test scenarios and logging capture, codifying the methodology run by hand on 2026-07-05 (Pixel 10 Pro XL). Modules: - breadcrumbs.py: a pure, unit-tested parser for the ImapPerf / MailReader / Reader / MailBackfiller breadcrumbs, plus open-correlation that reproduces the manual timing-tables.md figures exactly. - adb.py: a safety-guarded adb wrapper -- an allow-list of adb subcommands and a deny-list + assertions on shell commands. The only sanctioned app-state mutation is clearing LibreMail's own cache/ (exact-match); no pm clear / uninstall / data wipe, and no touching databases/ files/ shared_prefs/ datastore/ can be constructed. - uidump.py: uiautomator XML parser + screen recognition (mailbox rows with the cached "Available offline" flag, reader, and the keyguard / foreign-app guards). - scenarios.py: cold-open, message-open (uncached), back-nav, prefetch A/B (fetch-policy toggle) and cross-provider, each keyguard-guarded and driven through the guarded wrapper. - report.py + perf_harness.py: aggregates, a timing-tables.md renderer mirroring the manual write-up, and the CLI (timestamped run dir with the raw logcat, a filtered breadcrumb extract, and the tables). --dry-run prints the exact command plan without touching device state. Tests (stdlib unittest, 68 cases) validate the parser, guardrails, UI recognition and report against the manual run's real captures (fixtures include a verbatim perf-extract slice and the reader/lockscreen/alarm dumps). Track the *.log fixture past the gitignore *.log rule via a scoped negation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -30,6 +30,8 @@ secrets.properties
|
||||
|
||||
# Log Files
|
||||
*.log
|
||||
# ...but keep the device-testing parser fixtures (verbatim logcat slices used as test inputs)
|
||||
!scripts/device-testing/tests/fixtures/*.log
|
||||
|
||||
# Android Studio / IntelliJ
|
||||
.idea/
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
|
||||
# LibreMail device-testing perf harness
|
||||
|
||||
A cross-platform, **standard-library-only** Python tool that replicates LibreMail's
|
||||
on-device performance-test scenarios and logging capture. It codifies the methodology that
|
||||
was first run by hand (2026-07-05, Pixel 10 Pro XL) and written up in
|
||||
`scratchpad/perf/perf_summary.md`, `causation-report.md`, and `ab-run/timing-tables.md`.
|
||||
|
||||
Everything runs against **LibreMail only** plus read-only system-log/settings/dumpsys
|
||||
collection, behind hard device-safety guardrails (see [Device safety](#device-safety)).
|
||||
|
||||
## Requirements
|
||||
|
||||
- **Python 3.8+** (standard library only — no `pip install`, no third-party deps).
|
||||
- **`adb`** on `PATH` (or pass `--adb /path/to/adb`).
|
||||
- A connected device with:
|
||||
- **LibreMail installed** as a **debuggable** build (the cache-clear uses `run-as`, which
|
||||
only works on debuggable APKs), and
|
||||
- **at least one account signed in** with some **not-yet-cached** messages in the inbox
|
||||
(message bodies are fetched on first open), and
|
||||
- the screen **unlocked** (the harness keeps it awake during a run and guards every step
|
||||
against the keyguard, but it cannot get you *past* a secure lock screen).
|
||||
|
||||
No build step. Run it straight from the repo.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python scripts/device-testing/perf_harness.py <scenario> [options]
|
||||
```
|
||||
|
||||
Scenarios (each independently selectable):
|
||||
|
||||
| Scenario | What it does |
|
||||
|------------------|--------------|
|
||||
| `cold-open` | Force-stop LibreMail, clear **only** its `cache/`, `am start -W` ×N, parse `TotalTime`/`WaitTime`. |
|
||||
| `message-open` | Open N distinct **uncached** messages one at a time; time spinner→content from the breadcrumbs. |
|
||||
| `back-nav` | Time reader→mailbox back transitions ×N (dump-latency-bound; see caveat in the report). |
|
||||
| `prefetch-ab` | Run `message-open` under **Fetch all on Wi-Fi** (prefetch ON) vs **Always on-demand** (prefetch OFF), cache cleared between conditions. |
|
||||
| `cross-provider` | Open N messages from the (unified) inbox and tabulate per provider (`imap:…` vs `outlook:…`) from the breadcrumb account refs. |
|
||||
|
||||
Common options:
|
||||
|
||||
| Option | Default | Meaning |
|
||||
|--------|---------|---------|
|
||||
| `--serial <id>` | auto (if exactly one device) | choose the device |
|
||||
| `-n, --count <N>` | per-scenario | samples / runs (per condition for `prefetch-ab`) |
|
||||
| `--out <dir>` | `./device-perf-runs` | output root; a timestamped subdir is created per run |
|
||||
| `--package <pkg>` | `org.libremail.app` | target package |
|
||||
| `--component <c>` | `org.libremail.app/org.libremail.MainActivity` | launcher component |
|
||||
| `--adb <path>` | `adb` | path to the adb executable |
|
||||
| `--dry-run` | off | **print the exact command plan without changing device state** |
|
||||
|
||||
**Always start with `--dry-run`** to review the command plan a scenario will issue:
|
||||
|
||||
```bash
|
||||
python scripts/device-testing/perf_harness.py prefetch-ab --dry-run
|
||||
python scripts/device-testing/perf_harness.py cold-open -n 5 --serial 5C310DLCQ000G3
|
||||
```
|
||||
|
||||
## What each scenario measures
|
||||
|
||||
Timing comes **primarily from the on-device breadcrumbs** (PII-free), with uiautomator used
|
||||
only as a "content is ready" signal so the driver knows when to move on:
|
||||
|
||||
- `MailReader: openMessage <acctRef> folder=<label> fetchedBody=<bool> took=<ms>ms`
|
||||
— end-to-end reader open. `fetchedBody=true` ⇒ a real network body fetch (uncached).
|
||||
- `Reader: reader ready took=<ms>ms html=<bool> inline=<n>` — spinner→content.
|
||||
- `ImapPerf: <op> connect=<ms>ms work=<ms>ms live=<N>` and
|
||||
`ImapPerf: body-fetch select=<ms>ms body=<ms>ms flag=<ms>ms rfc822=<n>B chars=<n> att=<n>`
|
||||
— connection + phase split; `body KB/s` is `rfc822 / body_ms`.
|
||||
- `MailBackfiller: backfill … pages=<n> complete=<bool>` / `backfill slice…` — backfill activity.
|
||||
|
||||
`cold-open` instead parses `am start -W`'s `TotalTime` / `WaitTime`.
|
||||
|
||||
## Output
|
||||
|
||||
Each run writes a timestamped directory under `--out`:
|
||||
|
||||
```
|
||||
device-perf-runs/20260706-131612-cold-open/
|
||||
├── timing-tables.md # per-scenario tables + aggregates (mirrors the manual timing-tables.md)
|
||||
├── session-raw.log # the full `adb logcat -b all -v threadtime` stream for the run
|
||||
├── perf-extract.log # the ImapPerf|MailReader|Reader|MailBackfiller subset of the raw log
|
||||
└── driver.log # what the harness did, step by step
|
||||
```
|
||||
|
||||
## Device safety
|
||||
|
||||
Every device call goes through a guarded `adb` wrapper (`adb.py`). Two independent layers
|
||||
mean a dangerous command **cannot be constructed**:
|
||||
|
||||
- **Allow-list** of adb subcommands: `devices`, `get-state`, `install`, `shell`, `logcat`,
|
||||
`wait-for-device`, `start-server`. Anything else (`uninstall`, `root`, `remount`,
|
||||
`reboot`, `disable-verity`, `emu`, `push`, `pull`, …) is refused.
|
||||
- **Deny-list + assertions** on every `shell` command: no `pm clear` / `pm uninstall`, no
|
||||
reboot/remount/root/verity/factory-reset, **no touching the app's `databases/` / `files/`
|
||||
/ `shared_prefs/` / `datastore/`**, no output redirects, and `run-as` / `am force-stop` /
|
||||
`am start` are constrained to the target package.
|
||||
|
||||
The **only** sanctioned mutation of app state is clearing LibreMail's own **`cache/`**
|
||||
(`run-as org.libremail.app sh -c 'rm -rf cache/*'`) — an exact-match allow-list; any other
|
||||
`rm`/`mv`/`dd`/… is refused. There is **no** `pm clear`, uninstall, or data wipe anywhere.
|
||||
|
||||
The screen is kept awake (`svc power stayon true` + `KEYCODE_WAKEUP`) for the run and
|
||||
restored afterwards, and every uiautomator/input step is guarded against the keyguard and
|
||||
against a foreign app being in the foreground — a sample taken against either is **skipped**,
|
||||
not measured (the manual run hit exactly these: a lock-screen dump and a deskclock alarm).
|
||||
|
||||
## Tests
|
||||
|
||||
Pure-logic modules (the breadcrumb parser, the uiautomator parser, the safety guardrails,
|
||||
the report renderers) are unit-tested with the standard-library `unittest` against the
|
||||
**real captures** from the manual run:
|
||||
|
||||
```bash
|
||||
python -m unittest discover -s scripts/device-testing/tests -p "test_*.py"
|
||||
```
|
||||
|
||||
`tests/fixtures/perf-extract-sample.log` is a verbatim slice of the manual run's
|
||||
`perf-extract-ALL.log`, so the parser tests assert the harness reproduces the exact figures
|
||||
in the hand-written `timing-tables.md` (e.g. Gmail A1: `took=31227 ms`, `rfc822=60457 B`,
|
||||
`4.2 KB/s`; Outlook O1: `took=2934 ms`, `139.5 KB/s`). The UI fixtures include the reader,
|
||||
plus the lockscreen and deskclock-alarm negatives the keyguard/foreground guards must catch.
|
||||
|
||||
## Validated vs. needs the live run
|
||||
|
||||
**Validated offline** (by the unit tests, no device):
|
||||
|
||||
- Breadcrumb parsing + open-correlation reproduce the manual `timing-tables.md` figures.
|
||||
- The safety guardrails accept the known-good commands and refuse every forbidden one.
|
||||
- Screen recognition (mailbox rows + cached flag, reader, lockscreen, foreign app).
|
||||
- The report renders the same tables/aggregates as the manual write-up.
|
||||
- `--dry-run` emits the correct command plan for all five scenarios.
|
||||
|
||||
**Needs the first monitored live run** (a device makes the state real):
|
||||
|
||||
- End-to-end timing capture on hardware (streamed logcat → per-sample breadcrumb tailing).
|
||||
- **Settings-screen navigation for `prefetch-ab`.** No uiautomator dump of the settings
|
||||
screen was captured in the manual run, so `set_fetch_policy` navigates by the on-screen
|
||||
option text (`"Fetch all on Wi-Fi"`, `"Always on-demand"`, from `res/values/strings.xml`)
|
||||
with a scroll fallback. The bottom-nav "Settings" tap target is confirmed from
|
||||
`ui_mailbox.xml`; the option rows themselves need one live confirmation.
|
||||
- Row selection under a live, scrolling list and the auto-lock recovery path.
|
||||
|
||||
## Notes / design decisions
|
||||
|
||||
- **Uncached opens.** Message bodies live in the Room DB (`libremail.db`), **not** in
|
||||
`cache/`, so bodies can't be force-uncached without touching `databases/` (forbidden).
|
||||
The harness therefore opens **naturally-uncached** messages and verifies each was a real
|
||||
network fetch via the `fetchedBody=true` breadcrumb — exactly as the manual run did.
|
||||
- **Cross-provider** uses the unified inbox: a single mixed pass yields both providers, and
|
||||
the harness buckets rows by the breadcrumb account ref (`imap:…` vs `outlook:…`) — no
|
||||
account switching required.
|
||||
- **Back-nav** timings are dominated by the ~2.5–3 s uiautomator-dump latency floor; the
|
||||
report labels them accordingly (true in-app back is sub-second and not resolvable via adb
|
||||
UI polling under load).
|
||||
- Dev-script convention: Python 3, standard library only, cross-platform (Windows-primary),
|
||||
matching `.claude/skills/preflight/*.py`.
|
||||
@@ -0,0 +1,370 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""adb.py -- a deliberately narrow, safety-guarded wrapper around ``adb``.
|
||||
|
||||
Every device interaction in the harness goes through :class:`Adb`. The wrapper enforces
|
||||
two independent layers so a dangerous command *cannot be constructed*, even by a buggy
|
||||
caller:
|
||||
|
||||
* a TOP-LEVEL ALLOW-LIST of adb subcommands (``devices``, ``get-state``, ``install``,
|
||||
``shell``, ``logcat``, ``wait-for-device``) -- anything else raises :class:`AdbSafetyError`;
|
||||
* a DENY-LIST + assertions applied to every ``shell`` command: no ``pm clear`` / ``pm
|
||||
uninstall``, no ``reboot`` / ``remount`` / ``root`` / ``disable-verity`` / factory reset,
|
||||
no touching the app's ``databases/`` / ``files/`` / ``shared_prefs/`` / ``datastore/``,
|
||||
no output redirects, and the ONLY sanctioned destructive operation is clearing the app's
|
||||
own ``cache/`` (an exact-match allow-list of ``run-as <pkg> sh -c 'rm -rf cache/*'``).
|
||||
|
||||
``am force-stop`` / ``am start`` and ``run-as`` are constrained to the target package
|
||||
(default ``org.libremail.app``). All checks run in ``--dry-run`` mode too, so the guardrails
|
||||
are unit-testable without a device.
|
||||
|
||||
Scope: this harness operates ONLY on LibreMail plus read-only system-log/settings/dumpsys
|
||||
collection. It never writes app data other than clearing the cache.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import List, Optional, Sequence
|
||||
|
||||
DEFAULT_PACKAGE = "org.libremail.app"
|
||||
DEFAULT_COMPONENT = "org.libremail.app/org.libremail.MainActivity"
|
||||
|
||||
# adb subcommands the harness is ever allowed to invoke.
|
||||
_ALLOWED_SUBCOMMANDS = frozenset(
|
||||
{"devices", "get-state", "install", "shell", "logcat", "wait-for-device", "start-server"}
|
||||
)
|
||||
|
||||
# Substrings that must never appear anywhere in a shell command line.
|
||||
_DENY_SUBSTRINGS = (
|
||||
"pm clear",
|
||||
"pm uninstall",
|
||||
"pm disable",
|
||||
"pm hide",
|
||||
"disable-verity",
|
||||
"set-verity",
|
||||
"remount",
|
||||
"reboot",
|
||||
"bootloader",
|
||||
"fastboot",
|
||||
"factory",
|
||||
"wipe-data",
|
||||
"wipe_data",
|
||||
"mkfs",
|
||||
"format ",
|
||||
# App-private dirs we must never read or write (cache/ is the sole exception, handled
|
||||
# separately). Banning the segment names outright is defence-in-depth for this harness,
|
||||
# which never has a legitimate reason to touch them.
|
||||
"databases",
|
||||
"shared_prefs",
|
||||
"datastore",
|
||||
"/files",
|
||||
"files/",
|
||||
)
|
||||
|
||||
# Verbs that mutate the filesystem. Detected as whole *words* anywhere in the shell command
|
||||
# (they can be buried inside a ``sh -c '<payload>'`` token); permitted ONLY for the exact
|
||||
# sanctioned cache-clear (see _is_sanctioned_cache_clear).
|
||||
_DESTRUCTIVE_VERBS = ("rm", "rmdir", "mv", "dd", "truncate", "shred", "unlink", "mkfs", "chmod", "chown")
|
||||
_DESTRUCTIVE_RE = re.compile(r"\b(?:" + "|".join(_DESTRUCTIVE_VERBS) + r")\b")
|
||||
|
||||
# The exact ``sh -c`` payloads permitted for the cache clear -- nothing else.
|
||||
_CACHE_CLEAR_PAYLOADS = ("rm -rf cache/*", "rm -rf cache")
|
||||
|
||||
|
||||
class AdbSafetyError(RuntimeError):
|
||||
"""Raised when a command would violate the harness's device-safety guardrails."""
|
||||
|
||||
|
||||
class AdbError(RuntimeError):
|
||||
"""Raised when an adb command fails (non-zero exit) and the caller wanted a check."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class AdbResult:
|
||||
"""Outcome of one adb invocation."""
|
||||
|
||||
args: List[str]
|
||||
returncode: int
|
||||
stdout: str
|
||||
stderr: str
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return self.returncode == 0
|
||||
|
||||
|
||||
def _shell_command_string(args: Sequence[str]) -> str:
|
||||
"""Join the tokens *after* ``shell`` into a single inspection string."""
|
||||
return " ".join(args[1:]) if len(args) > 1 else ""
|
||||
|
||||
|
||||
def _is_sanctioned_cache_clear(package: str, args: Sequence[str]) -> bool:
|
||||
"""True iff ``args`` is exactly ``shell run-as <package> sh -c '<cache-clear>'``."""
|
||||
return (
|
||||
len(args) == 6
|
||||
and args[0] == "shell"
|
||||
and args[1] == "run-as"
|
||||
and args[2] == package
|
||||
and args[3] == "sh"
|
||||
and args[4] == "-c"
|
||||
and args[5] in _CACHE_CLEAR_PAYLOADS
|
||||
)
|
||||
|
||||
|
||||
def assert_safe(package: str, args: Sequence[str]) -> None:
|
||||
"""Raise :class:`AdbSafetyError` if ``args`` (an adb argv, sans ``adb``/``-s``) is unsafe.
|
||||
|
||||
Pure and side-effect-free so the guardrails can be unit-tested exhaustively.
|
||||
"""
|
||||
if not args:
|
||||
raise AdbSafetyError("empty adb command")
|
||||
|
||||
subcommand = args[0]
|
||||
if subcommand not in _ALLOWED_SUBCOMMANDS:
|
||||
raise AdbSafetyError(f"adb subcommand {subcommand!r} is not on the allow-list")
|
||||
|
||||
# 'install' may only take known-safe flags plus a path; it is never 'uninstall'.
|
||||
if subcommand == "install":
|
||||
for tok in args[1:]:
|
||||
if tok.startswith("-") and tok not in ("-r", "-t", "-g", "-d", "-i"):
|
||||
raise AdbSafetyError(f"install flag {tok!r} is not allowed")
|
||||
return
|
||||
|
||||
if subcommand != "shell":
|
||||
# devices / get-state / logcat / wait-for-device / start-server take no risky args.
|
||||
return
|
||||
|
||||
sanctioned_cache_clear = _is_sanctioned_cache_clear(package, args)
|
||||
sh = _shell_command_string(args)
|
||||
lowered = sh.lower()
|
||||
|
||||
for bad in _DENY_SUBSTRINGS:
|
||||
if bad in lowered:
|
||||
raise AdbSafetyError(f"shell command contains forbidden text {bad!r}: {sh!r}")
|
||||
|
||||
if ">" in sh:
|
||||
raise AdbSafetyError(f"shell output redirection is not allowed: {sh!r}")
|
||||
|
||||
tokens = list(args[1:])
|
||||
if _DESTRUCTIVE_RE.search(sh) and not sanctioned_cache_clear:
|
||||
raise AdbSafetyError(
|
||||
"the only permitted destructive shell command is clearing the app cache; "
|
||||
f"refusing: {sh!r}"
|
||||
)
|
||||
|
||||
# run-as must target our package and (unless the sanctioned cache clear) be a read.
|
||||
if "run-as" in tokens:
|
||||
idx = tokens.index("run-as")
|
||||
target = tokens[idx + 1] if idx + 1 < len(tokens) else None
|
||||
if target != package:
|
||||
raise AdbSafetyError(f"run-as may only target {package!r}, got {target!r}: {sh!r}")
|
||||
|
||||
# am force-stop / am start may only reference our package/component.
|
||||
if "am" in tokens:
|
||||
if "force-stop" in tokens:
|
||||
i = tokens.index("force-stop")
|
||||
target = tokens[i + 1] if i + 1 < len(tokens) else None
|
||||
if target != package:
|
||||
raise AdbSafetyError(f"am force-stop must target {package!r}: {sh!r}")
|
||||
if "-n" in tokens: # component: <package>/<activity>
|
||||
i = tokens.index("-n")
|
||||
component = tokens[i + 1] if i + 1 < len(tokens) else ""
|
||||
comp_pkg = component.split("/", 1)[0]
|
||||
if comp_pkg != package:
|
||||
raise AdbSafetyError(f"am start component must be in {package!r}: {sh!r}")
|
||||
for pkg_flag in ("-p", "--package"):
|
||||
if pkg_flag in tokens:
|
||||
i = tokens.index(pkg_flag)
|
||||
target = tokens[i + 1] if i + 1 < len(tokens) else None
|
||||
if target != package:
|
||||
raise AdbSafetyError(f"am {pkg_flag} must be {package!r}: {sh!r}")
|
||||
|
||||
|
||||
class Adb:
|
||||
"""Guarded adb wrapper bound to a single device serial and target package."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
serial: Optional[str] = None,
|
||||
package: str = DEFAULT_PACKAGE,
|
||||
adb_path: str = "adb",
|
||||
dry_run: bool = False,
|
||||
default_timeout: float = 60.0,
|
||||
logger=None,
|
||||
) -> None:
|
||||
self.serial = serial
|
||||
self.package = package
|
||||
self.adb_path = adb_path
|
||||
self.dry_run = dry_run
|
||||
self.default_timeout = default_timeout
|
||||
self._log = logger or (lambda msg: None)
|
||||
|
||||
# -- core ---------------------------------------------------------------- #
|
||||
def _argv(self, args: Sequence[str]) -> List[str]:
|
||||
prefix = [self.adb_path]
|
||||
if self.serial:
|
||||
prefix += ["-s", self.serial]
|
||||
return prefix + list(args)
|
||||
|
||||
def run(
|
||||
self,
|
||||
args: Sequence[str],
|
||||
check: bool = False,
|
||||
timeout: Optional[float] = None,
|
||||
) -> AdbResult:
|
||||
"""Validate and execute an adb command (``args`` excludes ``adb`` and ``-s <serial>``)."""
|
||||
assert_safe(self.package, args)
|
||||
argv = self._argv(args)
|
||||
printable = " ".join(argv)
|
||||
if self.dry_run:
|
||||
self._log(f"[dry-run] {printable}")
|
||||
return AdbResult(list(args), 0, "", "")
|
||||
self._log(f"$ {printable}")
|
||||
completed = subprocess.run(
|
||||
argv,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout if timeout is not None else self.default_timeout,
|
||||
)
|
||||
result = AdbResult(list(args), completed.returncode, completed.stdout, completed.stderr)
|
||||
if check and not result.ok:
|
||||
raise AdbError(f"adb failed ({result.returncode}): {printable}\n{result.stderr}")
|
||||
return result
|
||||
|
||||
# -- device discovery ---------------------------------------------------- #
|
||||
def devices(self) -> List[str]:
|
||||
"""Return the serials of attached, ready devices."""
|
||||
out = self.run(["devices"]).stdout
|
||||
return parse_devices(out)
|
||||
|
||||
def get_state(self) -> str:
|
||||
return self.run(["get-state"]).stdout.strip()
|
||||
|
||||
# -- lifecycle (LibreMail only) ------------------------------------------ #
|
||||
def force_stop(self) -> AdbResult:
|
||||
return self.run(["shell", "am", "force-stop", self.package])
|
||||
|
||||
def start_activity(
|
||||
self, component: str = DEFAULT_COMPONENT, wait: bool = True
|
||||
) -> AdbResult:
|
||||
args = ["shell", "am", "start"]
|
||||
if wait:
|
||||
args.append("-W")
|
||||
args += ["-n", component]
|
||||
return self.run(args)
|
||||
|
||||
def clear_cache(self) -> AdbResult:
|
||||
"""Clear ONLY the app's ``cache/`` via run-as. The sole sanctioned mutation."""
|
||||
return self.run(["shell", "run-as", self.package, "sh", "-c", "rm -rf cache/*"])
|
||||
|
||||
def list_cache(self) -> AdbResult:
|
||||
"""Read-only listing of the app cache dir (for verification/logging)."""
|
||||
return self.run(["shell", "run-as", self.package, "ls", "-la", "cache"])
|
||||
|
||||
# -- input / UI ---------------------------------------------------------- #
|
||||
def input_tap(self, x: int, y: int) -> AdbResult:
|
||||
return self.run(["shell", "input", "tap", str(x), str(y)])
|
||||
|
||||
def input_swipe(self, x1: int, y1: int, x2: int, y2: int, ms: int = 300) -> AdbResult:
|
||||
return self.run(
|
||||
["shell", "input", "swipe", str(x1), str(y1), str(x2), str(y2), str(ms)]
|
||||
)
|
||||
|
||||
def input_keyevent(self, keycode: str) -> AdbResult:
|
||||
return self.run(["shell", "input", "keyevent", str(keycode)])
|
||||
|
||||
def uiautomator_dump(self) -> str:
|
||||
"""Return the current window's uiautomator XML (via ``dump /dev/tty``)."""
|
||||
out = self.run(["shell", "uiautomator", "dump", "/dev/tty"], timeout=90).stdout
|
||||
return _extract_xml(out)
|
||||
|
||||
# -- screen / keyguard --------------------------------------------------- #
|
||||
def stay_on(self, on: bool = True) -> AdbResult:
|
||||
return self.run(["shell", "svc", "power", "stayon", "true" if on else "false"])
|
||||
|
||||
def wake(self) -> AdbResult:
|
||||
return self.input_keyevent("KEYCODE_WAKEUP")
|
||||
|
||||
def dumpsys(self, service: str, *extra: str) -> AdbResult:
|
||||
return self.run(["shell", "dumpsys", service, *extra])
|
||||
|
||||
def settings_get(self, namespace: str, key: str) -> str:
|
||||
return self.run(["shell", "settings", "get", namespace, key]).stdout.strip()
|
||||
|
||||
# -- logcat -------------------------------------------------------------- #
|
||||
def clear_logcat(self) -> AdbResult:
|
||||
return self.run(["shell", "logcat", "-c"])
|
||||
|
||||
def start_logcat(self, out_file) -> Optional[subprocess.Popen]:
|
||||
"""Start streaming the full logcat to an already-open file handle.
|
||||
|
||||
Returns the :class:`subprocess.Popen` (or ``None`` in dry-run). Stop it with
|
||||
:meth:`stop_logcat`.
|
||||
"""
|
||||
args = ["logcat", "-b", "all", "-v", "threadtime"]
|
||||
assert_safe(self.package, args)
|
||||
argv = self._argv(args)
|
||||
if self.dry_run:
|
||||
self._log(f"[dry-run] {' '.join(argv)} > <session-raw.log>")
|
||||
return None
|
||||
self._log(f"$ {' '.join(argv)} > <session-raw.log>")
|
||||
return subprocess.Popen(argv, stdout=out_file, stderr=subprocess.DEVNULL)
|
||||
|
||||
def settle(self, seconds: float) -> None:
|
||||
"""Sleep to let device state settle -- a no-op in dry-run so previews are instant."""
|
||||
if not self.dry_run and seconds > 0:
|
||||
time.sleep(seconds)
|
||||
|
||||
@staticmethod
|
||||
def stop_logcat(proc: Optional[subprocess.Popen]) -> None:
|
||||
if proc is None:
|
||||
return
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Pure parsers / helpers (unit-tested)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def parse_devices(output: str) -> List[str]:
|
||||
"""Parse ``adb devices`` output into a list of ready serials (state ``device``)."""
|
||||
serials: List[str] = []
|
||||
for line in output.splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("List of devices"):
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) >= 2 and parts[1] == "device":
|
||||
serials.append(parts[0])
|
||||
return serials
|
||||
|
||||
|
||||
_AM_FIELD_RE = re.compile(r"^\s*(TotalTime|WaitTime|ThisTime):\s*(\d+)\s*$")
|
||||
|
||||
|
||||
def parse_am_start(output: str) -> dict:
|
||||
"""Extract ``TotalTime`` / ``WaitTime`` / ``ThisTime`` (ms) from ``am start -W`` output."""
|
||||
result: dict = {}
|
||||
for line in output.splitlines():
|
||||
m = _AM_FIELD_RE.match(line)
|
||||
if m:
|
||||
result[m.group(1)] = int(m.group(2))
|
||||
return result
|
||||
|
||||
|
||||
def _extract_xml(raw: str) -> str:
|
||||
"""Pull the ``<?xml ...</hierarchy>`` payload out of ``uiautomator dump /dev/tty`` output."""
|
||||
start = raw.find("<?xml")
|
||||
if start == -1:
|
||||
start = raw.find("<hierarchy")
|
||||
end = raw.rfind("</hierarchy>")
|
||||
if start == -1 or end == -1:
|
||||
return raw.strip()
|
||||
return raw[start : end + len("</hierarchy>")]
|
||||
@@ -0,0 +1,422 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""breadcrumbs.py -- pure parser for LibreMail on-device perf breadcrumbs.
|
||||
|
||||
This module is the *core* of the device-testing harness and is intentionally free of any
|
||||
I/O, ``adb``, or device dependency so it can be unit-tested against the saved logcat
|
||||
captures from the manual perf run (see ``tests/test_breadcrumbs.py`` and
|
||||
``tests/fixtures/perf-extract-sample.log``).
|
||||
|
||||
It parses two things:
|
||||
|
||||
1. A raw ``adb logcat -v threadtime`` line into its fields (:func:`parse_logcat_line`).
|
||||
2. The message payload of the four perf tags into typed events
|
||||
(:func:`parse_breadcrumb` and the per-tag helpers).
|
||||
|
||||
Breadcrumb formats (verbatim from the app source, all PII-free):
|
||||
|
||||
* ``ImapPerf: <op> connect=<ms>ms work=<ms>ms live=<N>``
|
||||
-- ``ImapClient`` generic per-operation timing; ``op`` is a short label
|
||||
(``prefetch-body``, ``backfill-page``, ``body-fetch``, ``imap`` ...).
|
||||
* ``ImapPerf: body-fetch select=<ms>ms body=<ms>ms flag=<ms>ms rfc822=<n>B chars=<n> att=<n>``
|
||||
-- ``ImapClient`` detailed body-fetch phase split (emitted just before the generic
|
||||
``body-fetch connect=.. work=.. live=..`` line for the same fetch).
|
||||
* ``MailReader: openMessage <acctRef> folder=<label> fetchedBody=<bool> took=<ms>ms``
|
||||
-- ``MailRepositoryImpl`` reader-open; ``fetchedBody=true`` means a real network body
|
||||
fetch happened (i.e. the message was *not* cached).
|
||||
* ``Reader: reader ready took=<ms>ms html=<bool> inline=<n>``
|
||||
-- ``ReaderViewModel`` spinner-to-content time.
|
||||
* ``MailBackfiller: backfill <acctRef> folder=<label> pages=<n> complete=<bool>``
|
||||
* ``MailBackfiller: backfill slice: maxBatches=<n>``
|
||||
* ``MailBackfiller: backfill slice done: moreWork=<bool>``
|
||||
|
||||
The threadtime tag column is space-padded by logcat (e.g. ``Reader :``); the parser
|
||||
tolerates that and reports the trimmed tag (``Reader``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Iterable, Iterator, List, Optional, Union
|
||||
|
||||
# Tags that make up the filtered perf extract (mirrors the manual
|
||||
# `grep -E 'ImapPerf|MailReader|Reader|MailBackfiller'`). Matched on the *tag* column so a
|
||||
# stray mention of one of these words inside another tag's message is not miscounted.
|
||||
PERF_TAGS = ("ImapPerf", "MailReader", "Reader", "MailBackfiller")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Raw logcat line (``-v threadtime``)
|
||||
# --------------------------------------------------------------------------- #
|
||||
_THREADTIME_RE = re.compile(
|
||||
r"^(?P<date>\d{2}-\d{2})\s+"
|
||||
r"(?P<time>\d{2}:\d{2}:\d{2}\.\d{3})\s+"
|
||||
r"(?P<pid>\d+)\s+"
|
||||
r"(?P<tid>\d+)\s+"
|
||||
r"(?P<level>[VDIWEFS])\s+"
|
||||
r"(?P<tag>[^:]+?)\s*:\s?"
|
||||
r"(?P<message>.*)$"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LogLine:
|
||||
"""A single ``adb logcat -v threadtime`` line, split into fields."""
|
||||
|
||||
date: str
|
||||
time: str
|
||||
pid: int
|
||||
tid: int
|
||||
level: str
|
||||
tag: str
|
||||
message: str
|
||||
raw: str
|
||||
|
||||
@property
|
||||
def timestamp(self) -> str:
|
||||
"""`MM-DD HH:MM:SS.mmm` -- the wall-clock stamp (no year in logcat)."""
|
||||
return f"{self.date} {self.time}"
|
||||
|
||||
|
||||
def parse_logcat_line(line: str) -> Optional[LogLine]:
|
||||
"""Parse one threadtime logcat line; return ``None`` if it is not one."""
|
||||
m = _THREADTIME_RE.match(line.rstrip("\r\n"))
|
||||
if not m:
|
||||
return None
|
||||
return LogLine(
|
||||
date=m.group("date"),
|
||||
time=m.group("time"),
|
||||
pid=int(m.group("pid")),
|
||||
tid=int(m.group("tid")),
|
||||
level=m.group("level"),
|
||||
tag=m.group("tag").strip(),
|
||||
message=m.group("message").strip(),
|
||||
raw=line.rstrip("\r\n"),
|
||||
)
|
||||
|
||||
|
||||
def is_perf_line(line: str) -> bool:
|
||||
"""True if ``line`` is a threadtime line whose tag is one of :data:`PERF_TAGS`."""
|
||||
parsed = parse_logcat_line(line)
|
||||
return parsed is not None and parsed.tag in PERF_TAGS
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Typed breadcrumb events
|
||||
# --------------------------------------------------------------------------- #
|
||||
class Breadcrumb:
|
||||
"""Base type: every event optionally carries its source :class:`LogLine`.
|
||||
|
||||
A plain (non-dataclass) base so each subclass can declare its own fields with ``line``
|
||||
last -- this keeps the generated ``__init__`` valid on Python 3.7+ without the
|
||||
3.10-only ``kw_only`` field option (a base dataclass field with a default would force a
|
||||
"non-default argument follows default argument" error in the subclasses).
|
||||
"""
|
||||
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
@property
|
||||
def timestamp(self) -> Optional[str]:
|
||||
return self.line.timestamp if self.line else None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ImapPerfOp(Breadcrumb):
|
||||
"""``ImapPerf: <op> connect=<ms>ms work=<ms>ms live=<N>``."""
|
||||
|
||||
op: str
|
||||
connect_ms: int
|
||||
work_ms: int
|
||||
live: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BodyFetch(Breadcrumb):
|
||||
"""``ImapPerf: body-fetch select=.. body=.. flag=.. rfc822=..B chars=.. att=..``."""
|
||||
|
||||
select_ms: int
|
||||
body_ms: int
|
||||
flag_ms: int
|
||||
rfc822_bytes: int
|
||||
chars: int
|
||||
att: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
@property
|
||||
def body_kb_per_s(self) -> Optional[float]:
|
||||
"""Body-download throughput in decimal KB/s; ``None`` if body_ms==0.
|
||||
|
||||
``bytes/ms`` is already KB/s (``bytes/ms * 1000 ms/s / 1000 B/KB``), so this mirrors
|
||||
the manual ``timing-tables.md`` "body KB/s" column exactly (e.g. 60457/14253 = 4.2).
|
||||
"""
|
||||
if self.body_ms <= 0:
|
||||
return None
|
||||
return self.rfc822_bytes / self.body_ms
|
||||
|
||||
|
||||
@dataclass
|
||||
class OpenMessage(Breadcrumb):
|
||||
"""``MailReader: openMessage <ref> folder=<label> fetchedBody=<bool> took=<ms>ms``."""
|
||||
|
||||
account_ref: str
|
||||
folder: str
|
||||
fetched_body: bool
|
||||
took_ms: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ReaderReady(Breadcrumb):
|
||||
"""``Reader: reader ready took=<ms>ms html=<bool> inline=<n>``."""
|
||||
|
||||
took_ms: int
|
||||
html: bool
|
||||
inline: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackfillProgress(Breadcrumb):
|
||||
"""``MailBackfiller: backfill <ref> folder=<label> pages=<n> complete=<bool>``."""
|
||||
|
||||
account_ref: str
|
||||
folder: str
|
||||
pages: int
|
||||
complete: bool
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackfillSliceStart(Breadcrumb):
|
||||
"""``MailBackfiller: backfill slice: maxBatches=<n>``."""
|
||||
|
||||
max_batches: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackfillSliceDone(Breadcrumb):
|
||||
"""``MailBackfiller: backfill slice done: moreWork=<bool>``."""
|
||||
|
||||
more_work: bool
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
Event = Union[
|
||||
ImapPerfOp,
|
||||
BodyFetch,
|
||||
OpenMessage,
|
||||
ReaderReady,
|
||||
BackfillProgress,
|
||||
BackfillSliceStart,
|
||||
BackfillSliceDone,
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Message-payload parsers (pure; operate on the trimmed message string)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _to_bool(text: str) -> bool:
|
||||
return text == "true"
|
||||
|
||||
|
||||
_IMAP_OP_RE = re.compile(
|
||||
r"^(?P<op>\S+)\s+connect=(?P<connect>\d+)ms\s+work=(?P<work>\d+)ms\s+live=(?P<live>\d+)$"
|
||||
)
|
||||
_BODY_FETCH_RE = re.compile(
|
||||
r"^body-fetch\s+select=(?P<select>\d+)ms\s+body=(?P<body>\d+)ms\s+flag=(?P<flag>\d+)ms\s+"
|
||||
r"rfc822=(?P<bytes>\d+)B\s+chars=(?P<chars>\d+)\s+att=(?P<att>\d+)$"
|
||||
)
|
||||
_OPEN_MSG_RE = re.compile(
|
||||
r"^openMessage\s+(?P<ref>\S+)\s+folder=(?P<folder>\S+)\s+"
|
||||
r"fetchedBody=(?P<fetched>true|false)\s+took=(?P<took>\d+)ms$"
|
||||
)
|
||||
_READER_READY_RE = re.compile(
|
||||
r"^reader ready took=(?P<took>\d+)ms\s+html=(?P<html>true|false)\s+inline=(?P<inline>\d+)$"
|
||||
)
|
||||
_BACKFILL_PROGRESS_RE = re.compile(
|
||||
r"^backfill\s+(?P<ref>\S+)\s+folder=(?P<folder>\S+)\s+pages=(?P<pages>\d+)\s+"
|
||||
r"complete=(?P<complete>true|false)$"
|
||||
)
|
||||
_BACKFILL_SLICE_START_RE = re.compile(r"^backfill slice:\s+maxBatches=(?P<n>\d+)$")
|
||||
_BACKFILL_SLICE_DONE_RE = re.compile(r"^backfill slice done:\s+moreWork=(?P<more>true|false)$")
|
||||
|
||||
|
||||
def parse_imap_perf(message: str) -> Optional[Union[ImapPerfOp, BodyFetch]]:
|
||||
"""Parse an ``ImapPerf`` message (either the detailed body-fetch or the generic op)."""
|
||||
m = _BODY_FETCH_RE.match(message)
|
||||
if m:
|
||||
return BodyFetch(
|
||||
select_ms=int(m.group("select")),
|
||||
body_ms=int(m.group("body")),
|
||||
flag_ms=int(m.group("flag")),
|
||||
rfc822_bytes=int(m.group("bytes")),
|
||||
chars=int(m.group("chars")),
|
||||
att=int(m.group("att")),
|
||||
)
|
||||
m = _IMAP_OP_RE.match(message)
|
||||
if m:
|
||||
return ImapPerfOp(
|
||||
op=m.group("op"),
|
||||
connect_ms=int(m.group("connect")),
|
||||
work_ms=int(m.group("work")),
|
||||
live=int(m.group("live")),
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def parse_open_message(message: str) -> Optional[OpenMessage]:
|
||||
"""Parse a ``MailReader`` ``openMessage`` message."""
|
||||
m = _OPEN_MSG_RE.match(message)
|
||||
if not m:
|
||||
return None
|
||||
return OpenMessage(
|
||||
account_ref=m.group("ref"),
|
||||
folder=m.group("folder"),
|
||||
fetched_body=_to_bool(m.group("fetched")),
|
||||
took_ms=int(m.group("took")),
|
||||
)
|
||||
|
||||
|
||||
def parse_reader_ready(message: str) -> Optional[ReaderReady]:
|
||||
"""Parse a ``Reader`` ``reader ready`` message."""
|
||||
m = _READER_READY_RE.match(message)
|
||||
if not m:
|
||||
return None
|
||||
return ReaderReady(
|
||||
took_ms=int(m.group("took")),
|
||||
html=_to_bool(m.group("html")),
|
||||
inline=int(m.group("inline")),
|
||||
)
|
||||
|
||||
|
||||
def parse_backfill(
|
||||
message: str,
|
||||
) -> Optional[Union[BackfillProgress, BackfillSliceStart, BackfillSliceDone]]:
|
||||
"""Parse any of the three ``MailBackfiller`` messages."""
|
||||
m = _BACKFILL_PROGRESS_RE.match(message)
|
||||
if m:
|
||||
return BackfillProgress(
|
||||
account_ref=m.group("ref"),
|
||||
folder=m.group("folder"),
|
||||
pages=int(m.group("pages")),
|
||||
complete=_to_bool(m.group("complete")),
|
||||
)
|
||||
m = _BACKFILL_SLICE_START_RE.match(message)
|
||||
if m:
|
||||
return BackfillSliceStart(max_batches=int(m.group("n")))
|
||||
m = _BACKFILL_SLICE_DONE_RE.match(message)
|
||||
if m:
|
||||
return BackfillSliceDone(more_work=_to_bool(m.group("more")))
|
||||
return None
|
||||
|
||||
|
||||
# Dispatch table keyed by the trimmed logcat tag.
|
||||
_TAG_PARSERS = {
|
||||
"ImapPerf": parse_imap_perf,
|
||||
"MailReader": parse_open_message,
|
||||
"Reader": parse_reader_ready,
|
||||
"MailBackfiller": parse_backfill,
|
||||
}
|
||||
|
||||
|
||||
def parse_breadcrumb(line: str) -> Optional[Event]:
|
||||
"""Parse a full threadtime logcat line into a typed :data:`Event` (or ``None``).
|
||||
|
||||
The returned event carries its source :class:`LogLine` on ``.line`` so callers can
|
||||
order and time-box events. Lines that are not perf breadcrumbs return ``None``.
|
||||
"""
|
||||
log = parse_logcat_line(line)
|
||||
if log is None:
|
||||
return None
|
||||
parser = _TAG_PARSERS.get(log.tag)
|
||||
if parser is None:
|
||||
return None
|
||||
event = parser(log.message)
|
||||
if event is None:
|
||||
return None
|
||||
event.line = log
|
||||
return event
|
||||
|
||||
|
||||
def iter_events(lines: Iterable[str]) -> Iterator[Event]:
|
||||
"""Yield every parseable breadcrumb :data:`Event` from an iterable of logcat lines."""
|
||||
for line in lines:
|
||||
event = parse_breadcrumb(line)
|
||||
if event is not None:
|
||||
yield event
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Correlation -- group the breadcrumbs of one message-open together
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class OpenSample:
|
||||
"""One reader-open, correlating the breadcrumbs the app emits for a single open.
|
||||
|
||||
Mirrors a row of the manual ``timing-tables.md``: the ``openMessage`` marker plus the
|
||||
``body-fetch`` detail/op that preceded it and the ``reader ready`` that followed.
|
||||
"""
|
||||
|
||||
open_message: OpenMessage
|
||||
body_fetch: Optional[BodyFetch] = None
|
||||
body_fetch_op: Optional[ImapPerfOp] = None
|
||||
reader_ready: Optional[ReaderReady] = None
|
||||
|
||||
@property
|
||||
def account_ref(self) -> str:
|
||||
return self.open_message.account_ref
|
||||
|
||||
@property
|
||||
def cached(self) -> bool:
|
||||
"""A cached open did not fetch the body over the network."""
|
||||
return not self.open_message.fetched_body
|
||||
|
||||
@property
|
||||
def took_ms(self) -> int:
|
||||
return self.open_message.took_ms
|
||||
|
||||
@property
|
||||
def rfc822_bytes(self) -> Optional[int]:
|
||||
return self.body_fetch.rfc822_bytes if self.body_fetch else None
|
||||
|
||||
@property
|
||||
def body_kb_per_s(self) -> Optional[float]:
|
||||
return self.body_fetch.body_kb_per_s if self.body_fetch else None
|
||||
|
||||
|
||||
def correlate_opens(events: Iterable[Event]) -> List[OpenSample]:
|
||||
"""Group a flat, time-ordered event stream into :class:`OpenSample` records.
|
||||
|
||||
Strategy (matches how the breadcrumbs interleave in a real capture): keep the most
|
||||
recent ``body-fetch`` detail and generic ``body-fetch`` op seen; when an
|
||||
``openMessage`` marker arrives, attach those (consuming them so they are not reused);
|
||||
the first ``reader ready`` after the marker attaches to it.
|
||||
"""
|
||||
samples: List[OpenSample] = []
|
||||
pending_detail: Optional[BodyFetch] = None
|
||||
pending_op: Optional[ImapPerfOp] = None
|
||||
open_awaiting_reader: Optional[OpenSample] = None
|
||||
|
||||
for event in events:
|
||||
if isinstance(event, BodyFetch):
|
||||
pending_detail = event
|
||||
elif isinstance(event, ImapPerfOp) and event.op == "body-fetch":
|
||||
pending_op = event
|
||||
elif isinstance(event, OpenMessage):
|
||||
sample = OpenSample(open_message=event)
|
||||
if event.fetched_body:
|
||||
sample.body_fetch = pending_detail
|
||||
sample.body_fetch_op = pending_op
|
||||
pending_detail = None
|
||||
pending_op = None
|
||||
samples.append(sample)
|
||||
open_awaiting_reader = sample
|
||||
elif isinstance(event, ReaderReady):
|
||||
if open_awaiting_reader is not None and open_awaiting_reader.reader_ready is None:
|
||||
open_awaiting_reader.reader_ready = event
|
||||
open_awaiting_reader = None
|
||||
|
||||
return samples
|
||||
@@ -0,0 +1,258 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""perf_harness.py -- LibreMail on-device performance-test harness (CLI entry point).
|
||||
|
||||
Replicates the manual on-device perf methodology (2026-07-05, Pixel 10 Pro XL) as a
|
||||
repeatable, cross-platform, standard-library-only tool. Run one scenario at a time:
|
||||
|
||||
python scripts/device-testing/perf_harness.py cold-open [opts]
|
||||
python scripts/device-testing/perf_harness.py message-open [opts]
|
||||
python scripts/device-testing/perf_harness.py back-nav [opts]
|
||||
python scripts/device-testing/perf_harness.py prefetch-ab [opts]
|
||||
python scripts/device-testing/perf_harness.py cross-provider [opts]
|
||||
|
||||
Common options: ``--serial`` (auto-detected if exactly one device), ``--count/-n``,
|
||||
``--out``, ``--package``, ``--component``, ``--adb``, and ``--dry-run`` (print the exact
|
||||
command plan without touching device state -- use this to review a run before it happens).
|
||||
|
||||
Device safety is enforced by :mod:`adb` (allow-list of adb subcommands + deny-list on shell
|
||||
commands; the only sanctioned mutation is clearing LibreMail's own ``cache/``). See the
|
||||
README for the full guarantees and the required device state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as _dt
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from typing import List, Optional
|
||||
|
||||
# Flat-layout imports: this file's directory is on sys.path[0] when run as a script, and the
|
||||
# tests inject it explicitly. (The dir name contains a hyphen, so it is not an importable
|
||||
# package -- hence flat modules rather than `python -m`.)
|
||||
import breadcrumbs
|
||||
import report
|
||||
import scenarios
|
||||
from adb import Adb, DEFAULT_COMPONENT, DEFAULT_PACKAGE
|
||||
|
||||
SCENARIOS = ("cold-open", "message-open", "back-nav", "prefetch-ab", "cross-provider")
|
||||
_DEFAULT_COUNTS = {
|
||||
"cold-open": 5,
|
||||
"message-open": 6,
|
||||
"back-nav": 6,
|
||||
"prefetch-ab": 3,
|
||||
"cross-provider": 8,
|
||||
}
|
||||
|
||||
|
||||
def _make_logger(log_path: Optional[str]):
|
||||
handle = open(log_path, "a", encoding="utf-8") if log_path else None
|
||||
|
||||
def log(msg: str) -> None:
|
||||
stamp = _dt.datetime.now().strftime("%H:%M:%S")
|
||||
line = f"[{stamp}] {msg}"
|
||||
print(line, flush=True)
|
||||
if handle:
|
||||
handle.write(line + "\n")
|
||||
handle.flush()
|
||||
|
||||
return log, handle
|
||||
|
||||
|
||||
def resolve_serial(adb_path: str, requested: Optional[str]) -> Optional[str]:
|
||||
"""Return the serial to use, auto-detecting when exactly one device is attached."""
|
||||
probe = Adb(serial=None, adb_path=adb_path)
|
||||
serials = probe.devices()
|
||||
if requested:
|
||||
if serials and requested not in serials:
|
||||
print(
|
||||
f"warning: requested serial {requested!r} not in attached devices {serials}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return requested
|
||||
if len(serials) == 1:
|
||||
return serials[0]
|
||||
if not serials:
|
||||
raise SystemExit("no devices attached; connect one or pass --serial")
|
||||
raise SystemExit(f"multiple devices attached {serials}; pass --serial to choose one")
|
||||
|
||||
|
||||
def _run_dir(out_root: str, scenario: str) -> str:
|
||||
stamp = _dt.datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||
path = os.path.join(out_root, f"{stamp}-{scenario}")
|
||||
os.makedirs(path, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def _write_filtered_extract(raw_path: str, extract_path: str) -> int:
|
||||
"""Write the ImapPerf/MailReader/Reader/MailBackfiller subset of the raw log; return count."""
|
||||
if not os.path.exists(raw_path):
|
||||
return 0
|
||||
count = 0
|
||||
with open(raw_path, "r", encoding="utf-8", errors="replace") as src, open(
|
||||
extract_path, "w", encoding="utf-8", newline="\n"
|
||||
) as dst:
|
||||
for line in src:
|
||||
if breadcrumbs.is_perf_line(line):
|
||||
dst.write(line if line.endswith("\n") else line + "\n")
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def _render_sections(scenario: str, results, adb: Adb, count: int) -> List[str]:
|
||||
if scenario == "cold-open":
|
||||
return [report.render_cold_open(results)]
|
||||
if scenario == "message-open":
|
||||
return [report.render_message_open("Message open (uncached)", results)]
|
||||
if scenario == "back-nav":
|
||||
return [report.render_back_nav(results)]
|
||||
if scenario == "prefetch-ab":
|
||||
return [
|
||||
report.render_message_open(
|
||||
"Condition A - Fetch all on Wi-Fi (prefetch ON)", results["conditionA"]
|
||||
),
|
||||
report.render_message_open(
|
||||
"Condition B - Always on-demand (prefetch OFF)", results["conditionB"]
|
||||
),
|
||||
_ab_comparison(results),
|
||||
]
|
||||
if scenario == "cross-provider":
|
||||
sections = []
|
||||
for ref, rows in sorted(results.items()):
|
||||
sections.append(report.render_message_open(f"Provider {ref}", rows))
|
||||
if not sections:
|
||||
sections.append("## Cross-provider\n\nNo opens captured.\n")
|
||||
return sections
|
||||
return []
|
||||
|
||||
|
||||
def _ab_comparison(results: dict) -> str:
|
||||
def median_took(rows):
|
||||
vals = [r.took_ms for r in rows if not r.skipped and not r.cached and r.took_ms]
|
||||
return report.aggregate(vals).median
|
||||
|
||||
a = median_took(results["conditionA"])
|
||||
b = median_took(results["conditionB"])
|
||||
return (
|
||||
"## Prefetch A/B comparison\n\n"
|
||||
f"- Condition A (prefetch ON) median openMessage: {report._ms(a)} ms\n"
|
||||
f"- Condition B (prefetch OFF) median openMessage: {report._ms(b)} ms\n"
|
||||
)
|
||||
|
||||
|
||||
def _run_scenario(scenario: str, adb: Adb, tailer, args, log):
|
||||
package, component, count = args.package, args.component, args.count
|
||||
if scenario == "cold-open":
|
||||
return scenarios.cold_open(adb, component, count, log)
|
||||
if scenario == "message-open":
|
||||
return scenarios.message_open(adb, package, tailer, count, log)
|
||||
if scenario == "back-nav":
|
||||
return scenarios.back_nav(adb, package, count, log)
|
||||
if scenario == "prefetch-ab":
|
||||
return scenarios.prefetch_ab(adb, package, component, tailer, count, log)
|
||||
if scenario == "cross-provider":
|
||||
return scenarios.cross_provider(adb, package, tailer, count, log)
|
||||
raise SystemExit(f"unknown scenario {scenario!r}")
|
||||
|
||||
|
||||
def build_arg_parser() -> argparse.ArgumentParser:
|
||||
p = argparse.ArgumentParser(
|
||||
prog="perf_harness.py",
|
||||
description="LibreMail on-device performance-test harness (stdlib-only).",
|
||||
)
|
||||
p.add_argument("scenario", choices=SCENARIOS, help="which scenario to run")
|
||||
p.add_argument("--serial", help="device serial (auto-detected if exactly one attached)")
|
||||
p.add_argument("--package", default=DEFAULT_PACKAGE, help="target app package")
|
||||
p.add_argument("--component", default=DEFAULT_COMPONENT, help="launcher component")
|
||||
p.add_argument("--adb", default="adb", dest="adb_path", help="path to the adb executable")
|
||||
p.add_argument(
|
||||
"-n", "--count", type=int, default=None,
|
||||
help="samples/runs (per condition for prefetch-ab); scenario-specific default",
|
||||
)
|
||||
p.add_argument(
|
||||
"--out", default=os.path.join(os.getcwd(), "device-perf-runs"),
|
||||
help="output root; a timestamped subdir is created per run",
|
||||
)
|
||||
p.add_argument(
|
||||
"--dry-run", action="store_true",
|
||||
help="print the command plan without changing device state",
|
||||
)
|
||||
return p
|
||||
|
||||
|
||||
def main(argv: Optional[List[str]] = None) -> int:
|
||||
args = build_arg_parser().parse_args(argv)
|
||||
if args.count is None:
|
||||
args.count = _DEFAULT_COUNTS[args.scenario]
|
||||
|
||||
serial = None if args.dry_run else resolve_serial(args.adb_path, args.serial)
|
||||
if args.dry_run and args.serial:
|
||||
serial = args.serial
|
||||
|
||||
run_dir = _run_dir(args.out, args.scenario)
|
||||
driver_log_path = os.path.join(run_dir, "driver.log")
|
||||
log, log_handle = _make_logger(driver_log_path)
|
||||
raw_path = os.path.join(run_dir, "session-raw.log")
|
||||
|
||||
log(f"scenario={args.scenario} serial={serial} count={args.count} dry_run={args.dry_run}")
|
||||
log(f"run dir: {run_dir}")
|
||||
|
||||
adb = Adb(
|
||||
serial=serial,
|
||||
package=args.package,
|
||||
adb_path=args.adb_path,
|
||||
dry_run=args.dry_run,
|
||||
logger=log,
|
||||
)
|
||||
|
||||
logcat_proc = None
|
||||
raw_handle = None
|
||||
tailer: scenarios.LogTailer = scenarios._NullTailer()
|
||||
try:
|
||||
if not args.dry_run:
|
||||
adb.clear_logcat()
|
||||
# Binary handle: the logcat child writes raw bytes to this fd.
|
||||
raw_handle = open(raw_path, "wb")
|
||||
logcat_proc = adb.start_logcat(raw_handle)
|
||||
tailer = scenarios.LogTailer(raw_path)
|
||||
scenarios.ensure_awake(adb, log)
|
||||
time.sleep(1.0)
|
||||
|
||||
results = _run_scenario(args.scenario, adb, tailer, args, log)
|
||||
sections = _render_sections(args.scenario, results, adb, args.count)
|
||||
finally:
|
||||
Adb.stop_logcat(logcat_proc)
|
||||
if raw_handle:
|
||||
raw_handle.close()
|
||||
if not args.dry_run:
|
||||
adb.stay_on(False) # restore default screen-timeout behaviour
|
||||
if log_handle:
|
||||
log_handle.flush()
|
||||
|
||||
metadata = {
|
||||
"scenario": args.scenario,
|
||||
"timestamp": _dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"serial": serial or "(dry-run)",
|
||||
"package": args.package,
|
||||
"count": args.count,
|
||||
}
|
||||
document = report.build_document(metadata, sections)
|
||||
tables_path = os.path.join(run_dir, "timing-tables.md")
|
||||
with open(tables_path, "w", encoding="utf-8", newline="\n") as fh:
|
||||
fh.write(document)
|
||||
|
||||
extract_count = 0
|
||||
if not args.dry_run:
|
||||
extract_count = _write_filtered_extract(
|
||||
raw_path, os.path.join(run_dir, "perf-extract.log")
|
||||
)
|
||||
log(f"wrote {tables_path} (filtered {extract_count} breadcrumb lines)")
|
||||
if log_handle:
|
||||
log_handle.close()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,253 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""report.py -- aggregate scenario samples and render ``timing-tables.md``.
|
||||
|
||||
Pure and testable: given the samples a scenario collected, it produces the same
|
||||
per-scenario markdown tables + aggregates as the hand-written ``timing-tables.md`` from the
|
||||
manual run. No device or I/O dependency beyond writing the final file.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import statistics
|
||||
from dataclasses import dataclass
|
||||
from typing import List, Optional, Sequence
|
||||
|
||||
from breadcrumbs import OpenSample
|
||||
|
||||
# The floor below which adb uiautomator-dump timings cannot resolve an in-app transition
|
||||
# (see perf_summary.md: nav/back were dominated by the ~2.5-3 s dump latency).
|
||||
UIAUTOMATOR_LATENCY_FLOOR_MS = 3000
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Stats + markdown helpers
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class Aggregate:
|
||||
n: int
|
||||
mean: Optional[float]
|
||||
median: Optional[float]
|
||||
minimum: Optional[int]
|
||||
maximum: Optional[int]
|
||||
|
||||
|
||||
def aggregate(values: Sequence[float]) -> Aggregate:
|
||||
"""Summarise a list of numbers; safe on an empty list."""
|
||||
vals = [v for v in values if v is not None]
|
||||
if not vals:
|
||||
return Aggregate(0, None, None, None, None)
|
||||
return Aggregate(
|
||||
n=len(vals),
|
||||
mean=statistics.fmean(vals),
|
||||
median=statistics.median(vals),
|
||||
minimum=min(vals),
|
||||
maximum=max(vals),
|
||||
)
|
||||
|
||||
|
||||
def md_table(headers: Sequence[str], rows: Sequence[Sequence[object]]) -> str:
|
||||
"""Render a GitHub-flavoured markdown table."""
|
||||
head = "| " + " | ".join(str(h) for h in headers) + " |"
|
||||
sep = "| " + " | ".join("---" for _ in headers) + " |"
|
||||
body = [
|
||||
"| " + " | ".join("" if c is None else str(c) for c in row) + " |" for row in rows
|
||||
]
|
||||
return "\n".join([head, sep, *body])
|
||||
|
||||
|
||||
def _ms(value: Optional[float]) -> str:
|
||||
return "" if value is None else f"{round(value)}"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Cold open
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class ColdOpenSample:
|
||||
run: int
|
||||
total_time_ms: Optional[int]
|
||||
wait_time_ms: Optional[int]
|
||||
note: str = ""
|
||||
|
||||
|
||||
def render_cold_open(samples: Sequence[ColdOpenSample]) -> str:
|
||||
rows: List[Sequence[object]] = []
|
||||
for s in samples:
|
||||
rows.append([s.run, _ms(s.total_time_ms), _ms(s.wait_time_ms), s.note])
|
||||
total_agg = aggregate([s.total_time_ms for s in samples if s.total_time_ms is not None])
|
||||
wait_agg = aggregate([s.wait_time_ms for s in samples if s.wait_time_ms is not None])
|
||||
rows.append(
|
||||
[
|
||||
"**mean**",
|
||||
f"**{_ms(total_agg.mean)}**",
|
||||
f"**{_ms(wait_agg.mean)}**",
|
||||
f"median {_ms(total_agg.median)} / {_ms(wait_agg.median)}",
|
||||
]
|
||||
)
|
||||
table = md_table(["Run", "TotalTime (ms)", "WaitTime (ms)", "note"], rows)
|
||||
return (
|
||||
"## Cold open (am start -W, cache cleared each run)\n\n"
|
||||
+ table
|
||||
+ "\n\nColdest run is the first post-clear launch (class-load/JIT); steady state is lower.\n"
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Message open (reader body-load)
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class ReaderOpenRow:
|
||||
"""A flattened, render-ready reader-open row (mirrors a timing-tables.md line)."""
|
||||
|
||||
index: int
|
||||
label: str
|
||||
account_ref: str = ""
|
||||
cached: bool = False
|
||||
took_ms: Optional[int] = None
|
||||
reader_ready_ms: Optional[int] = None
|
||||
connect_ms: Optional[int] = None
|
||||
work_ms: Optional[int] = None
|
||||
select_ms: Optional[int] = None
|
||||
body_ms: Optional[int] = None
|
||||
flag_ms: Optional[int] = None
|
||||
live: Optional[int] = None
|
||||
rfc822_bytes: Optional[int] = None
|
||||
body_kb_per_s: Optional[float] = None
|
||||
skipped: bool = False
|
||||
reason: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_open_sample(
|
||||
cls, sample: OpenSample, index: int, label: str
|
||||
) -> "ReaderOpenRow":
|
||||
bf = sample.body_fetch
|
||||
op = sample.body_fetch_op
|
||||
return cls(
|
||||
index=index,
|
||||
label=label,
|
||||
account_ref=sample.account_ref,
|
||||
cached=sample.cached,
|
||||
took_ms=sample.took_ms,
|
||||
reader_ready_ms=sample.reader_ready.took_ms if sample.reader_ready else None,
|
||||
connect_ms=op.connect_ms if op else None,
|
||||
work_ms=op.work_ms if op else None,
|
||||
select_ms=bf.select_ms if bf else None,
|
||||
body_ms=bf.body_ms if bf else None,
|
||||
flag_ms=bf.flag_ms if bf else None,
|
||||
live=op.live if op else None,
|
||||
rfc822_bytes=bf.rfc822_bytes if bf else None,
|
||||
body_kb_per_s=sample.body_kb_per_s,
|
||||
)
|
||||
|
||||
|
||||
_OPEN_HEADERS = [
|
||||
"#",
|
||||
"message",
|
||||
"cached",
|
||||
"rfc822 B",
|
||||
"openMessage took",
|
||||
"reader ready",
|
||||
"connect",
|
||||
"work",
|
||||
"select",
|
||||
"body-dl",
|
||||
"flag",
|
||||
"live",
|
||||
"body KB/s",
|
||||
]
|
||||
|
||||
|
||||
def _open_row_cells(row: ReaderOpenRow) -> Sequence[object]:
|
||||
if row.skipped:
|
||||
return [
|
||||
row.index,
|
||||
row.label,
|
||||
"-",
|
||||
f"SKIPPED: {row.reason}",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
]
|
||||
kbps = "" if row.body_kb_per_s is None else f"{row.body_kb_per_s:.1f}"
|
||||
return [
|
||||
row.index,
|
||||
row.label,
|
||||
"yes" if row.cached else "no",
|
||||
row.rfc822_bytes if row.rfc822_bytes is not None else "",
|
||||
_fmt_ms(row.took_ms),
|
||||
_fmt_ms(row.reader_ready_ms),
|
||||
_ms(row.connect_ms),
|
||||
_ms(row.work_ms),
|
||||
_ms(row.select_ms),
|
||||
_ms(row.body_ms),
|
||||
_ms(row.flag_ms),
|
||||
_ms(row.live),
|
||||
kbps,
|
||||
]
|
||||
|
||||
|
||||
def _fmt_ms(value: Optional[int]) -> str:
|
||||
return "" if value is None else f"{value} ms"
|
||||
|
||||
|
||||
def render_message_open(title: str, rows: Sequence[ReaderOpenRow]) -> str:
|
||||
table = md_table(_OPEN_HEADERS, [_open_row_cells(r) for r in rows])
|
||||
uncached = [r for r in rows if not r.skipped and not r.cached and r.took_ms is not None]
|
||||
agg = aggregate([r.took_ms for r in uncached])
|
||||
lines = [f"## {title}", "", table, ""]
|
||||
if agg.n:
|
||||
lines.append(
|
||||
f"Uncached opens: n={agg.n}, openMessage median "
|
||||
f"{_ms(agg.median)} ms, mean {_ms(agg.mean)} ms, "
|
||||
f"range {agg.minimum}-{agg.maximum} ms."
|
||||
)
|
||||
skipped = [r for r in rows if r.skipped]
|
||||
if skipped:
|
||||
lines.append(f"Skipped samples: {len(skipped)} (see rows above).")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Back navigation
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class BackNavSample:
|
||||
index: int
|
||||
back_ms: Optional[int]
|
||||
note: str = ""
|
||||
|
||||
|
||||
def render_back_nav(samples: Sequence[BackNavSample]) -> str:
|
||||
rows = [[s.index, _fmt_ms(s.back_ms), s.note] for s in samples]
|
||||
agg = aggregate([s.back_ms for s in samples if s.back_ms is not None])
|
||||
table = md_table(["#", "back (reader->mailbox)", "note"], rows)
|
||||
caveat = (
|
||||
f"\n\n**Caveat:** these are dominated by the ~{UIAUTOMATOR_LATENCY_FLOOR_MS} ms "
|
||||
"uiautomator-dump latency floor; true in-app back is sub-second and not precisely "
|
||||
"measurable via adb UI polling under load (see perf_summary.md)."
|
||||
)
|
||||
summary = "" if not agg.n else f"\n\nBack: n={agg.n}, mean {_ms(agg.mean)} ms, median {_ms(agg.median)} ms."
|
||||
return "## Reader -> mailbox (back)\n\n" + table + summary + caveat + "\n"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Document assembly
|
||||
# --------------------------------------------------------------------------- #
|
||||
def render_header(metadata: dict) -> str:
|
||||
lines = ["# LibreMail device perf run", ""]
|
||||
for key, value in metadata.items():
|
||||
lines.append(f"- **{key}:** {value}")
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def build_document(metadata: dict, sections: Sequence[str]) -> str:
|
||||
parts = [render_header(metadata), *sections]
|
||||
return "\n".join(parts).rstrip() + "\n"
|
||||
@@ -0,0 +1,445 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""scenarios.py -- the on-device performance scenarios.
|
||||
|
||||
Each scenario drives LibreMail through the guarded :class:`adb.Adb` wrapper, times the
|
||||
behaviour primarily from the on-device breadcrumbs (with uiautomator as a fallback ready
|
||||
signal), and returns render-ready samples for :mod:`report`.
|
||||
|
||||
Timing philosophy (from the manual run): message-open and reader-ready times come from the
|
||||
``MailReader`` / ``Reader`` / ``ImapPerf`` breadcrumbs -- adb UI polling only tells us *when*
|
||||
the content is ready so we can move on, and back-nav is explicitly dump-latency-bound.
|
||||
|
||||
Safety: every device call goes through :class:`adb.Adb`, so its allow/deny guardrails apply.
|
||||
Every uiautomator/input step is guarded against the keyguard and against a foreign app being
|
||||
in the foreground; a sample taken against either is skipped, not measured.
|
||||
|
||||
``--dry-run`` executes one representative pass per scenario -- issuing the canonical command
|
||||
sequence (so the plan is auditable) without looping on a live UI.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from typing import Callable, List, Optional
|
||||
|
||||
import breadcrumbs
|
||||
import uidump
|
||||
from adb import Adb, parse_am_start
|
||||
from report import BackNavSample, ColdOpenSample, ReaderOpenRow
|
||||
|
||||
# Message-open can stall 30-75 s behind the spinner (Gmail throttle), so allow generous
|
||||
# headroom before giving up on a single open.
|
||||
OPEN_TIMEOUT_S = 150.0
|
||||
POLL_INTERVAL_S = 2.0
|
||||
SETTLE_S = 1.5
|
||||
|
||||
# Fetch-policy option labels (from res/values/strings.xml) used to drive the A/B toggle.
|
||||
FETCH_WIFI_LABEL = "Fetch all on Wi-Fi" # prefetch ON (Condition A / WIFI_ONLY)
|
||||
FETCH_ON_DEMAND_LABEL = "Always on-demand" # prefetch OFF (Condition B / ON_DEMAND)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Live-logcat tailing (per-sample breadcrumb extraction)
|
||||
# --------------------------------------------------------------------------- #
|
||||
class LogTailer:
|
||||
"""Reads newly-appended text from the growing session logcat file since the last mark."""
|
||||
|
||||
def __init__(self, path: str) -> None:
|
||||
self.path = path
|
||||
self._offset = 0
|
||||
|
||||
def mark(self) -> None:
|
||||
"""Set the read cursor to the current end of file."""
|
||||
self._offset = os.path.getsize(self.path) if os.path.exists(self.path) else 0
|
||||
|
||||
def read_new(self) -> str:
|
||||
if not os.path.exists(self.path):
|
||||
return ""
|
||||
with open(self.path, "r", encoding="utf-8", errors="replace") as fh:
|
||||
fh.seek(self._offset)
|
||||
data = fh.read()
|
||||
self._offset = fh.tell()
|
||||
return data
|
||||
|
||||
|
||||
class _NullTailer(LogTailer):
|
||||
"""A tailer that yields nothing -- used in dry-run so scenarios need no live log."""
|
||||
|
||||
def __init__(self) -> None: # noqa: D401 - see base
|
||||
super().__init__(path="")
|
||||
|
||||
def mark(self) -> None:
|
||||
pass
|
||||
|
||||
def read_new(self) -> str:
|
||||
return ""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# UI helpers (all guarded)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def dump_ui(adb: Adb) -> Optional[uidump.UiNode]:
|
||||
"""Dump + parse the current UI; ``None`` in dry-run or on an unparseable dump."""
|
||||
if adb.dry_run:
|
||||
return None
|
||||
xml = adb.uiautomator_dump()
|
||||
if not xml or "<hierarchy" not in xml:
|
||||
return None
|
||||
try:
|
||||
return uidump.parse_dump(xml)
|
||||
except Exception: # pragma: no cover - malformed dump under load
|
||||
return None
|
||||
|
||||
|
||||
def ensure_awake(adb: Adb, log: Callable[[str], None]) -> None:
|
||||
"""Keep the screen on and awake for the run (restored by the caller afterwards)."""
|
||||
adb.stay_on(True)
|
||||
adb.wake()
|
||||
|
||||
|
||||
def guard_ready(adb: Adb, package: str, log: Callable[[str], None]) -> Optional[uidump.UiNode]:
|
||||
"""Return the current UI iff it is our app and not the keyguard; else try to recover.
|
||||
|
||||
Returns ``None`` if, after a wake attempt, the sample is still against the lockscreen or
|
||||
a foreign app -- the caller must skip that sample rather than measure garbage.
|
||||
"""
|
||||
root = dump_ui(adb)
|
||||
if root is None:
|
||||
return None
|
||||
if uidump.is_lockscreen(root):
|
||||
log("keyguard detected; waking and re-checking")
|
||||
adb.wake()
|
||||
time.sleep(SETTLE_S)
|
||||
root = dump_ui(adb)
|
||||
if root is None or uidump.is_lockscreen(root):
|
||||
log("still on keyguard after wake; skipping sample")
|
||||
return None
|
||||
if not uidump.is_app_foreground(root, package):
|
||||
log(f"foreground is {uidump.foreground_package(root)!r}, not {package!r}; skipping")
|
||||
return None
|
||||
return root
|
||||
|
||||
|
||||
def goto_mailbox(adb: Adb, package: str, log: Callable[[str], None]) -> Optional[uidump.UiNode]:
|
||||
"""Ensure the mailbox list is showing (press Back out of the reader if needed)."""
|
||||
root = guard_ready(adb, package, log)
|
||||
if root is None:
|
||||
return None
|
||||
if uidump.is_reader(root, package):
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
time.sleep(SETTLE_S)
|
||||
root = guard_ready(adb, package, log)
|
||||
return root
|
||||
|
||||
|
||||
def wait_for_reader_ready(adb: Adb, package: str, log: Callable[[str], None]) -> bool:
|
||||
"""Poll until the reader has loaded its body (no spinner) or timeout. True if loaded."""
|
||||
deadline = time.monotonic() + OPEN_TIMEOUT_S
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(POLL_INTERVAL_S)
|
||||
root = dump_ui(adb)
|
||||
if root is None:
|
||||
continue
|
||||
if uidump.is_lockscreen(root):
|
||||
log("keyguard appeared during open; sample is invalid")
|
||||
return False
|
||||
if uidump.is_reader(root, package) and not uidump.has_progress_bar(root):
|
||||
return True
|
||||
log("open timed out")
|
||||
return False
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 1: cold open
|
||||
# --------------------------------------------------------------------------- #
|
||||
def cold_open(
|
||||
adb: Adb,
|
||||
component: str,
|
||||
runs: int,
|
||||
log: Callable[[str], None],
|
||||
settle_s: float = 2.0,
|
||||
) -> List[ColdOpenSample]:
|
||||
"""Force-stop + clear cache + ``am start -W`` x ``runs``; parse TotalTime/WaitTime."""
|
||||
samples: List[ColdOpenSample] = []
|
||||
for run in range(1, runs + 1):
|
||||
adb.force_stop()
|
||||
adb.clear_cache()
|
||||
adb.settle(settle_s)
|
||||
out = adb.start_activity(component=component, wait=True).stdout
|
||||
fields = parse_am_start(out)
|
||||
note = "" if fields else ("dry-run" if adb.dry_run else "no am-start timing parsed")
|
||||
samples.append(
|
||||
ColdOpenSample(
|
||||
run=run,
|
||||
total_time_ms=fields.get("TotalTime"),
|
||||
wait_time_ms=fields.get("WaitTime"),
|
||||
note=note,
|
||||
)
|
||||
)
|
||||
log(f"cold-open run {run}: {fields or note}")
|
||||
adb.settle(settle_s)
|
||||
return samples
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 2: message open (uncached)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _row_from_events(index: int, label: str, events: List[breadcrumbs.Event]) -> ReaderOpenRow:
|
||||
"""Build a reader-open row from the breadcrumbs captured during one open."""
|
||||
opens = breadcrumbs.correlate_opens(events)
|
||||
if not opens:
|
||||
return ReaderOpenRow(index=index, label=label, skipped=True, reason="no breadcrumb")
|
||||
sample = opens[-1]
|
||||
return ReaderOpenRow.from_open_sample(sample, index=index, label=label)
|
||||
|
||||
|
||||
def open_one_message(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
tailer: LogTailer,
|
||||
index: int,
|
||||
already_opened: set,
|
||||
log: Callable[[str], None],
|
||||
) -> Optional[ReaderOpenRow]:
|
||||
"""Open the next not-yet-opened, uncached message; return its timed row (or ``None``)."""
|
||||
root = goto_mailbox(adb, package, log)
|
||||
if root is None:
|
||||
return ReaderOpenRow(index=index, label="?", skipped=True, reason="not on mailbox")
|
||||
|
||||
rows = uidump.find_message_rows(root, package)
|
||||
candidates = [r for r in rows if not r.cached and r.label not in already_opened]
|
||||
if not candidates:
|
||||
# Scroll to reveal more of the list, then re-scan once.
|
||||
adb.input_swipe(672, 2000, 672, 900, 400)
|
||||
time.sleep(SETTLE_S)
|
||||
root = guard_ready(adb, package, log)
|
||||
if root is None:
|
||||
return None
|
||||
rows = uidump.find_message_rows(root, package)
|
||||
candidates = [r for r in rows if not r.cached and r.label not in already_opened]
|
||||
if not candidates:
|
||||
log("no more uncached messages visible")
|
||||
return None
|
||||
|
||||
target = candidates[0]
|
||||
already_opened.add(target.label)
|
||||
log(f"opening row#{target.index} {target.label!r} at {target.center}")
|
||||
|
||||
tailer.mark()
|
||||
adb.input_tap(*target.center)
|
||||
loaded = wait_for_reader_ready(adb, package, log)
|
||||
events = list(breadcrumbs.iter_events(tailer.read_new().splitlines()))
|
||||
row = _row_from_events(index, target.label, events)
|
||||
if not loaded and not row.skipped:
|
||||
row.reason = "ui ready-signal timed out (breadcrumb used)"
|
||||
elif not loaded:
|
||||
row.skipped = True
|
||||
row.reason = "open timed out"
|
||||
# Return to the mailbox for the next sample.
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
time.sleep(SETTLE_S)
|
||||
return row
|
||||
|
||||
|
||||
def message_open(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
tailer: LogTailer,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> List[ReaderOpenRow]:
|
||||
"""Open up to ``count`` distinct uncached messages one at a time."""
|
||||
if adb.dry_run:
|
||||
_dry_run_open_demo(adb, log)
|
||||
return [ReaderOpenRow(index=1, label="<dry-run>", skipped=True, reason="dry-run")]
|
||||
|
||||
rows: List[ReaderOpenRow] = []
|
||||
opened: set = set()
|
||||
for i in range(1, count + 1):
|
||||
row = open_one_message(adb, package, tailer, i, opened, log)
|
||||
if row is None:
|
||||
break
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
|
||||
def _dry_run_open_demo(adb: Adb, log: Callable[[str], None]) -> None:
|
||||
"""Issue the canonical message-open command shape once (dry-run only)."""
|
||||
adb.uiautomator_dump()
|
||||
adb.input_tap(672, 504) # tap a representative message row centre
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 3: back navigation
|
||||
# --------------------------------------------------------------------------- #
|
||||
def back_nav(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> List[BackNavSample]:
|
||||
"""Time reader -> mailbox back transitions (dump-latency-bound; see report caveat)."""
|
||||
if adb.dry_run:
|
||||
adb.uiautomator_dump()
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
return [BackNavSample(index=1, back_ms=None, note="dry-run")]
|
||||
|
||||
samples: List[BackNavSample] = []
|
||||
opened: set = set()
|
||||
for i in range(1, count + 1):
|
||||
# Get into the reader by opening any message.
|
||||
root = goto_mailbox(adb, package, log)
|
||||
if root is None:
|
||||
samples.append(BackNavSample(index=i, back_ms=None, note="not on mailbox"))
|
||||
continue
|
||||
rows = uidump.find_message_rows(root, package)
|
||||
if not rows:
|
||||
samples.append(BackNavSample(index=i, back_ms=None, note="no rows"))
|
||||
continue
|
||||
adb.input_tap(*rows[0].center)
|
||||
wait_for_reader_ready(adb, package, log)
|
||||
# Now time the back transition to the mailbox.
|
||||
start = time.monotonic()
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
reached = _wait_until(
|
||||
lambda: _is_mailbox(adb, package), timeout_s=30.0
|
||||
)
|
||||
elapsed_ms = round((time.monotonic() - start) * 1000)
|
||||
samples.append(
|
||||
BackNavSample(
|
||||
index=i,
|
||||
back_ms=elapsed_ms if reached else None,
|
||||
note="" if reached else "did not reach mailbox",
|
||||
)
|
||||
)
|
||||
return samples
|
||||
|
||||
|
||||
def _is_mailbox(adb: Adb, package: str) -> bool:
|
||||
root = dump_ui(adb)
|
||||
return root is not None and uidump.is_app_foreground(root, package) and not uidump.is_reader(
|
||||
root, package
|
||||
)
|
||||
|
||||
|
||||
def _wait_until(predicate: Callable[[], bool], timeout_s: float) -> bool:
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while time.monotonic() < deadline:
|
||||
if predicate():
|
||||
return True
|
||||
time.sleep(0.2)
|
||||
return False
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 4: prefetch A/B (fetch policy toggle)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def set_fetch_policy(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
option_label: str,
|
||||
log: Callable[[str], None],
|
||||
) -> bool:
|
||||
"""Navigate Settings and select a fetch-policy option by its visible label.
|
||||
|
||||
NOTE: no uiautomator dump of the settings screen was captured in the manual run, so the
|
||||
settings-screen navigation is by on-screen text and needs live verification (documented
|
||||
in the README). Returns True if the option was found and tapped.
|
||||
"""
|
||||
if adb.dry_run:
|
||||
adb.uiautomator_dump()
|
||||
adb.input_tap(1014, 2728) # bottom-nav "Settings"
|
||||
adb.uiautomator_dump()
|
||||
adb.input_tap(672, 1400) # representative fetch-policy option
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
return True
|
||||
|
||||
root = goto_mailbox(adb, package, log)
|
||||
if root is None:
|
||||
return False
|
||||
settings_label = uidump.find_by_text(root, "Settings")
|
||||
if settings_label is None:
|
||||
log("no Settings entry found in bottom nav")
|
||||
return False
|
||||
anchor = settings_label.first_clickable_ancestor() or settings_label
|
||||
if anchor.center:
|
||||
adb.input_tap(*anchor.center)
|
||||
time.sleep(SETTLE_S)
|
||||
|
||||
# Find the fetch-policy option, scrolling the settings list if necessary.
|
||||
for _ in range(6):
|
||||
sroot = guard_ready(adb, package, log)
|
||||
if sroot is None:
|
||||
return False
|
||||
option = uidump.find_by_text(sroot, option_label)
|
||||
if option is not None:
|
||||
anchor = option.first_clickable_ancestor() or option
|
||||
if anchor.center:
|
||||
adb.input_tap(*anchor.center)
|
||||
log(f"selected fetch policy {option_label!r}")
|
||||
time.sleep(SETTLE_S)
|
||||
adb.input_keyevent("KEYCODE_BACK") # back to mailbox
|
||||
time.sleep(SETTLE_S)
|
||||
return True
|
||||
adb.input_swipe(672, 2000, 672, 900, 400)
|
||||
time.sleep(SETTLE_S)
|
||||
log(f"fetch-policy option {option_label!r} not found")
|
||||
return False
|
||||
|
||||
|
||||
def prefetch_ab(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
component: str,
|
||||
tailer: LogTailer,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> dict:
|
||||
"""Run message-open under prefetch ON (Wi-Fi) vs OFF (on-demand), cache cleared between."""
|
||||
result = {"conditionA": [], "conditionB": []}
|
||||
|
||||
log("Condition A: fetch policy = Fetch all on Wi-Fi (prefetch ON)")
|
||||
set_fetch_policy(adb, package, FETCH_WIFI_LABEL, log)
|
||||
_reset_for_uncached(adb, component, log)
|
||||
result["conditionA"] = message_open(adb, package, tailer, count, log)
|
||||
|
||||
log("Condition B: fetch policy = Always on-demand (prefetch OFF)")
|
||||
set_fetch_policy(adb, package, FETCH_ON_DEMAND_LABEL, log)
|
||||
_reset_for_uncached(adb, component, log)
|
||||
result["conditionB"] = message_open(adb, package, tailer, count, log)
|
||||
return result
|
||||
|
||||
|
||||
def _reset_for_uncached(adb: Adb, component: str, log: Callable[[str], None]) -> None:
|
||||
"""Clear the cache (and restart) so subsequent opens are uncached network fetches."""
|
||||
adb.force_stop()
|
||||
adb.clear_cache()
|
||||
adb.settle(1.0)
|
||||
adb.start_activity(component=component, wait=True)
|
||||
adb.settle(3.0)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 5: cross-provider (optional)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def cross_provider(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
tailer: LogTailer,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> dict:
|
||||
"""Open ``count`` messages from the (unified) inbox and bucket the rows by account_ref.
|
||||
|
||||
The breadcrumbs carry the account reference (``imap:...`` vs ``outlook:...``), so a
|
||||
single mixed-inbox pass yields the per-provider comparison without account switching.
|
||||
"""
|
||||
rows = message_open(adb, package, tailer, count, log)
|
||||
buckets: dict = {}
|
||||
for row in rows:
|
||||
key = row.account_ref or "unknown"
|
||||
buckets.setdefault(key, []).append(row)
|
||||
return buckets
|
||||
@@ -0,0 +1,2 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Test package for the LibreMail device-testing perf harness."""
|
||||
@@ -0,0 +1,17 @@
|
||||
07-05 17:55:38.343 10261 10423 D ImapPerf: prefetch-body connect=857ms work=2150ms live=3
|
||||
07-05 17:55:54.352 10261 10425 D ImapPerf: backfill-page connect=4366ms work=71989ms live=2
|
||||
07-05 17:57:38.981 10261 10425 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=false took=21ms
|
||||
07-05 17:57:39.041 10261 10261 D Reader : reader ready took=82ms html=true inline=0
|
||||
07-05 18:04:37.001 10261 10408 I MailBackfiller: backfill slice done: moreWork=true
|
||||
07-05 18:04:37.002 10261 10408 I MailBackfiller: backfill slice: maxBatches=20
|
||||
07-05 18:07:16.094 10261 15306 D ImapPerf: body-fetch select=2566ms body=14253ms flag=2356ms rfc822=60457B chars=51610 att=0
|
||||
07-05 18:07:18.435 10261 15306 D ImapPerf: body-fetch connect=2624ms work=26211ms live=3
|
||||
07-05 18:07:20.817 10261 15306 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=true took=31227ms
|
||||
07-05 18:07:21.206 10261 10261 D Reader : reader ready took=31617ms html=true inline=0
|
||||
07-05 18:07:32.607 10261 10414 D ImapPerf: backfill-page connect=656ms work=3038ms live=3
|
||||
07-05 18:36:37.959 19255 21059 D MailBackfiller: backfill outlook:6b54d6 folder=INBOX pages=20 complete=false
|
||||
07-05 18:36:54.941 19255 19314 D ImapPerf: body-fetch select=185ms body=1355ms flag=78ms rfc822=189086B chars=82986 att=0
|
||||
07-05 18:36:55.002 19255 19314 D ImapPerf: body-fetch connect=772ms work=2018ms live=2
|
||||
07-05 18:36:55.140 19255 21060 I MailReader: openMessage outlook:6b54d6 folder=INBOX fetchedBody=true took=2934ms
|
||||
07-05 18:36:55.144 19255 19255 D Reader : reader ready took=2939ms html=true inline=0
|
||||
07-05 18:37:11.704 19255 19314 D ImapPerf: imap connect=3156ms work=8032ms live=2
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><hierarchy rotation="0"><node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="0" hint=""><node index="0" text="" resource-id="" class="android.widget.LinearLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="2" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/action_bar_root" class="android.widget.LinearLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="android:id/content" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="2" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/content" class="android.view.ViewGroup" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/digital_clock" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1047][1290,1586]" drawing-order="18" hint=""><node index="0" text="18:00" resource-id="com.google.android.deskclock:id/auto_sizing_text_clock" class="android.widget.TextView" package="com.google.android.deskclock" content-desc="18:00" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1047][1290,1586]" drawing-order="1" hint="" /></node><node index="1" text="Bae check in" resource-id="com.google.android.deskclock:id/title" class="android.widget.TextView" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1646][1290,1960]" drawing-order="19" hint="" /><node index="2" text="Snooze" resource-id="com.google.android.deskclock:id/button_tap_snooze" class="android.widget.Button" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[126,2274][669,2562]" drawing-order="39" hint="" /><node index="3" text="Stop" resource-id="com.google.android.deskclock:id/button_tap_dismiss" class="android.widget.Button" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[675,2274][1218,2562]" drawing-order="40" hint="" /></node></node></node></node></node></node></hierarchy>
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><hierarchy rotation="0"><node index="0" text="" resource-id="com.android.systemui:id/legacy_window_root" class="android.widget.FrameLayout" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[0,0][1344,2992]"><node index="0" text="" resource-id="com.android.systemui:id/keyguard_root_view" class="android.view.ViewGroup" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[0,0][1344,2992]"><node index="0" text="" resource-id="com.android.systemui:id/accessibility_actions_view" class="android.view.View" package="com.android.systemui" content-desc="Lock screen" clickable="false" enabled="true" long-clickable="true" scrollable="false" bounds="[0,216][1344,2992]" /><node index="1" text="18 02" resource-id="" class="android.widget.TextView" package="com.android.systemui" content-desc="18:02" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[100,454][583,620]" /><node index="2" text="" resource-id="com.android.systemui:id/device_entry_icon_fg" class="android.widget.ImageView" package="com.android.systemui" content-desc="Fingerprint sensor" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[561,2126][791,2356]" /><node index="3" text="Charged" resource-id="com.android.systemui:id/keyguard_indication_text" class="android.widget.TextView" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[16,2860][1344,2925]" /></node></node></hierarchy>
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,160 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Tests for the adb safety guardrails -- the harness must refuse dangerous commands."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import adb # noqa: E402
|
||||
from adb import Adb, AdbSafetyError, assert_safe # noqa: E402
|
||||
|
||||
PKG = "org.libremail.app"
|
||||
|
||||
|
||||
class TestAllowedCommands(unittest.TestCase):
|
||||
def _ok(self, args):
|
||||
try:
|
||||
assert_safe(PKG, args)
|
||||
except AdbSafetyError as exc: # pragma: no cover - failure path
|
||||
self.fail(f"assert_safe wrongly rejected {args}: {exc}")
|
||||
|
||||
def test_lifecycle_and_input_allowed(self):
|
||||
self._ok(["shell", "am", "force-stop", PKG])
|
||||
self._ok(["shell", "am", "start", "-W", "-n", f"{PKG}/org.libremail.MainActivity"])
|
||||
self._ok(["shell", "input", "tap", "672", "504"])
|
||||
self._ok(["shell", "input", "swipe", "672", "2000", "672", "900", "400"])
|
||||
self._ok(["shell", "input", "keyevent", "KEYCODE_BACK"])
|
||||
self._ok(["shell", "input", "keyevent", "KEYCODE_WAKEUP"])
|
||||
self._ok(["shell", "uiautomator", "dump", "/dev/tty"])
|
||||
self._ok(["shell", "svc", "power", "stayon", "true"])
|
||||
self._ok(["shell", "dumpsys", "gfxinfo", PKG])
|
||||
self._ok(["shell", "settings", "get", "global", "airplane_mode_on"])
|
||||
self._ok(["shell", "logcat", "-c"])
|
||||
self._ok(["logcat", "-b", "all", "-v", "threadtime"])
|
||||
self._ok(["devices"])
|
||||
self._ok(["get-state"])
|
||||
self._ok(["install", "-r", "app-debug.apk"])
|
||||
|
||||
def test_sanctioned_cache_clear_allowed(self):
|
||||
self._ok(["shell", "run-as", PKG, "sh", "-c", "rm -rf cache/*"])
|
||||
self._ok(["shell", "run-as", PKG, "sh", "-c", "rm -rf cache"])
|
||||
self._ok(["shell", "run-as", PKG, "ls", "-la", "cache"])
|
||||
|
||||
def test_input_swipe_not_confused_with_wipe(self):
|
||||
# 'swipe' contains the substring 'wipe'; it must still be allowed.
|
||||
self._ok(["shell", "input", "swipe", "0", "0", "0", "500", "300"])
|
||||
|
||||
|
||||
class TestDeniedCommands(unittest.TestCase):
|
||||
def _deny(self, args):
|
||||
with self.assertRaises(AdbSafetyError):
|
||||
assert_safe(PKG, args)
|
||||
|
||||
def test_top_level_allowlist(self):
|
||||
self._deny(["uninstall", PKG])
|
||||
self._deny(["root"])
|
||||
self._deny(["remount"])
|
||||
self._deny(["reboot"])
|
||||
self._deny(["disable-verity"])
|
||||
self._deny(["emu", "kill"])
|
||||
self._deny(["push", "x", "/data"])
|
||||
self._deny(["pull", "/data/data/org.libremail.app/databases/x"])
|
||||
|
||||
def test_pm_clear_and_uninstall_denied(self):
|
||||
self._deny(["shell", "pm", "clear", PKG])
|
||||
self._deny(["shell", "pm", "uninstall", PKG])
|
||||
self._deny(["shell", "pm", "disable", PKG])
|
||||
|
||||
def test_reboot_and_root_via_shell_denied(self):
|
||||
self._deny(["shell", "reboot"])
|
||||
self._deny(["shell", "svc", "power", "reboot"])
|
||||
self._deny(["shell", "su", "-c", "reboot"])
|
||||
|
||||
def test_no_touching_private_dirs(self):
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf databases"])
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf shared_prefs"])
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf datastore"])
|
||||
self._deny(["shell", "run-as", PKG, "cat", "databases/libremail.db"])
|
||||
self._deny(["shell", "run-as", PKG, "ls", "files/datastore"])
|
||||
|
||||
def test_rm_outside_cache_denied(self):
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf /sdcard/x"])
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf cache/../databases"])
|
||||
self._deny(["shell", "rm", "-rf", "/data/local/tmp"])
|
||||
|
||||
def test_output_redirect_denied(self):
|
||||
self._deny(["shell", "echo", "x", ">", "/sdcard/y"])
|
||||
|
||||
def test_run_as_foreign_package_denied(self):
|
||||
self._deny(["shell", "run-as", "com.other.app", "ls", "cache"])
|
||||
|
||||
def test_am_foreign_targets_denied(self):
|
||||
self._deny(["shell", "am", "force-stop", "com.other.app"])
|
||||
self._deny(["shell", "am", "start", "-n", "com.other.app/.Main"])
|
||||
|
||||
def test_empty_command_denied(self):
|
||||
self._deny([])
|
||||
|
||||
|
||||
class TestAdbWrapperBuildsSafeArgv(unittest.TestCase):
|
||||
"""In dry-run the wrapper still validates and returns the argv it would run."""
|
||||
|
||||
def setUp(self):
|
||||
self.adb = Adb(serial="SERIAL123", package=PKG, dry_run=True)
|
||||
|
||||
def test_force_stop_argv(self):
|
||||
self.assertEqual(self.adb.force_stop().args, ["shell", "am", "force-stop", PKG])
|
||||
|
||||
def test_start_activity_argv(self):
|
||||
args = self.adb.start_activity().args
|
||||
self.assertEqual(args[:5], ["shell", "am", "start", "-W", "-n"])
|
||||
self.assertTrue(args[5].startswith(PKG + "/"))
|
||||
|
||||
def test_clear_cache_argv_is_sanctioned(self):
|
||||
self.assertEqual(
|
||||
self.adb.clear_cache().args,
|
||||
["shell", "run-as", PKG, "sh", "-c", "rm -rf cache/*"],
|
||||
)
|
||||
|
||||
def test_argv_includes_serial(self):
|
||||
full = self.adb._argv(["devices"])
|
||||
self.assertEqual(full, ["adb", "-s", "SERIAL123", "devices"])
|
||||
|
||||
def test_wrapper_rejects_unsafe_low_level_call(self):
|
||||
with self.assertRaises(AdbSafetyError):
|
||||
self.adb.run(["shell", "pm", "clear", PKG])
|
||||
|
||||
|
||||
class TestPureParsers(unittest.TestCase):
|
||||
def test_parse_devices(self):
|
||||
out = (
|
||||
"List of devices attached\n"
|
||||
"5C310DLCQ000G3\tdevice\n"
|
||||
"emulator-5554\tdevice\n"
|
||||
"0123456789\toffline\n"
|
||||
)
|
||||
self.assertEqual(adb.parse_devices(out), ["5C310DLCQ000G3", "emulator-5554"])
|
||||
|
||||
def test_parse_devices_empty(self):
|
||||
self.assertEqual(adb.parse_devices("List of devices attached\n"), [])
|
||||
|
||||
def test_parse_am_start(self):
|
||||
out = (
|
||||
"Starting: Intent { cmp=org.libremail.app/org.libremail.MainActivity }\n"
|
||||
"Status: ok\n"
|
||||
"LaunchState: COLD\n"
|
||||
"TotalTime: 384\n"
|
||||
"WaitTime: 386\n"
|
||||
)
|
||||
fields = adb.parse_am_start(out)
|
||||
self.assertEqual(fields["TotalTime"], 384)
|
||||
self.assertEqual(fields["WaitTime"], 386)
|
||||
|
||||
def test_parse_am_start_missing(self):
|
||||
self.assertEqual(adb.parse_am_start("Status: ok\n"), {})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,222 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Unit tests for the breadcrumb parser -- validated against the manual run's real captures.
|
||||
|
||||
The fixture ``fixtures/perf-extract-sample.log`` is a verbatim slice of the manual run's
|
||||
``perf-extract-ALL.log`` (plus the Outlook control), so these tests assert that the parser
|
||||
reproduces the exact figures in the hand-written ``timing-tables.md``.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import breadcrumbs as bc # noqa: E402
|
||||
|
||||
FIXTURE = os.path.join(os.path.dirname(__file__), "fixtures", "perf-extract-sample.log")
|
||||
|
||||
|
||||
def _load_fixture_lines():
|
||||
with open(FIXTURE, encoding="utf-8") as fh:
|
||||
return fh.read().splitlines()
|
||||
|
||||
|
||||
class TestLogcatLine(unittest.TestCase):
|
||||
def test_parses_threadtime_fields(self):
|
||||
line = "07-05 18:07:20.817 10261 15306 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=true took=31227ms"
|
||||
parsed = bc.parse_logcat_line(line)
|
||||
self.assertIsNotNone(parsed)
|
||||
self.assertEqual(parsed.date, "07-05")
|
||||
self.assertEqual(parsed.time, "18:07:20.817")
|
||||
self.assertEqual(parsed.pid, 10261)
|
||||
self.assertEqual(parsed.tid, 15306)
|
||||
self.assertEqual(parsed.level, "I")
|
||||
self.assertEqual(parsed.tag, "MailReader")
|
||||
self.assertTrue(parsed.message.startswith("openMessage imap:94058a"))
|
||||
self.assertEqual(parsed.timestamp, "07-05 18:07:20.817")
|
||||
|
||||
def test_tolerates_space_padded_tag(self):
|
||||
# logcat pads the tag column, e.g. "Reader :".
|
||||
line = "07-05 17:57:39.041 10261 10261 D Reader : reader ready took=82ms html=true inline=0"
|
||||
parsed = bc.parse_logcat_line(line)
|
||||
self.assertEqual(parsed.tag, "Reader")
|
||||
self.assertEqual(parsed.message, "reader ready took=82ms html=true inline=0")
|
||||
|
||||
def test_non_threadtime_returns_none(self):
|
||||
self.assertIsNone(bc.parse_logcat_line("--------- beginning of main"))
|
||||
self.assertIsNone(bc.parse_logcat_line(""))
|
||||
|
||||
def test_is_perf_line(self):
|
||||
self.assertTrue(
|
||||
bc.is_perf_line("07-05 17:55:38.343 10261 10423 D ImapPerf: prefetch-body connect=857ms work=2150ms live=3")
|
||||
)
|
||||
self.assertFalse(
|
||||
bc.is_perf_line("07-05 15:00:06.896 15192 15217 I MailSyncer: sync all: 1 accounts")
|
||||
)
|
||||
self.assertFalse(bc.is_perf_line("garbage"))
|
||||
|
||||
|
||||
class TestMessageParsers(unittest.TestCase):
|
||||
def test_imap_perf_generic_op(self):
|
||||
ev = bc.parse_imap_perf("prefetch-body connect=857ms work=2150ms live=3")
|
||||
self.assertIsInstance(ev, bc.ImapPerfOp)
|
||||
self.assertEqual((ev.op, ev.connect_ms, ev.work_ms, ev.live), ("prefetch-body", 857, 2150, 3))
|
||||
|
||||
def test_imap_perf_backfill_page(self):
|
||||
ev = bc.parse_imap_perf("backfill-page connect=4366ms work=71989ms live=2")
|
||||
self.assertEqual((ev.op, ev.connect_ms, ev.work_ms, ev.live), ("backfill-page", 4366, 71989, 2))
|
||||
|
||||
def test_imap_perf_body_fetch_op(self):
|
||||
ev = bc.parse_imap_perf("body-fetch connect=2624ms work=26211ms live=3")
|
||||
self.assertIsInstance(ev, bc.ImapPerfOp)
|
||||
self.assertEqual(ev.op, "body-fetch")
|
||||
|
||||
def test_body_fetch_detail(self):
|
||||
ev = bc.parse_imap_perf(
|
||||
"body-fetch select=2566ms body=14253ms flag=2356ms rfc822=60457B chars=51610 att=0"
|
||||
)
|
||||
self.assertIsInstance(ev, bc.BodyFetch)
|
||||
self.assertEqual(ev.select_ms, 2566)
|
||||
self.assertEqual(ev.body_ms, 14253)
|
||||
self.assertEqual(ev.flag_ms, 2356)
|
||||
self.assertEqual(ev.rfc822_bytes, 60457)
|
||||
self.assertEqual(ev.chars, 51610)
|
||||
self.assertEqual(ev.att, 0)
|
||||
|
||||
def test_body_fetch_throughput_matches_manual_table(self):
|
||||
# timing-tables.md reports 4.2 KB/s for A1 and 139.5 KB/s for O1.
|
||||
a1 = bc.parse_imap_perf(
|
||||
"body-fetch select=2566ms body=14253ms flag=2356ms rfc822=60457B chars=51610 att=0"
|
||||
)
|
||||
self.assertAlmostEqual(a1.body_kb_per_s, 4.24, places=2)
|
||||
o1 = bc.parse_imap_perf(
|
||||
"body-fetch select=185ms body=1355ms flag=78ms rfc822=189086B chars=82986 att=0"
|
||||
)
|
||||
self.assertAlmostEqual(o1.body_kb_per_s, 139.5, places=1)
|
||||
|
||||
def test_open_message_uncached(self):
|
||||
ev = bc.parse_open_message("openMessage imap:94058a folder=INBOX fetchedBody=true took=31227ms")
|
||||
self.assertEqual(ev.account_ref, "imap:94058a")
|
||||
self.assertEqual(ev.folder, "INBOX")
|
||||
self.assertTrue(ev.fetched_body)
|
||||
self.assertEqual(ev.took_ms, 31227)
|
||||
|
||||
def test_open_message_cached(self):
|
||||
ev = bc.parse_open_message("openMessage outlook:6b54d6 folder=INBOX fetchedBody=false took=4ms")
|
||||
self.assertEqual(ev.account_ref, "outlook:6b54d6")
|
||||
self.assertFalse(ev.fetched_body)
|
||||
self.assertEqual(ev.took_ms, 4)
|
||||
|
||||
def test_reader_ready(self):
|
||||
ev = bc.parse_reader_ready("reader ready took=31617ms html=true inline=0")
|
||||
self.assertEqual(ev.took_ms, 31617)
|
||||
self.assertTrue(ev.html)
|
||||
self.assertEqual(ev.inline, 0)
|
||||
|
||||
def test_backfill_progress(self):
|
||||
ev = bc.parse_backfill("backfill outlook:6b54d6 folder=INBOX pages=20 complete=false")
|
||||
self.assertIsInstance(ev, bc.BackfillProgress)
|
||||
self.assertEqual(ev.account_ref, "outlook:6b54d6")
|
||||
self.assertEqual(ev.pages, 20)
|
||||
self.assertFalse(ev.complete)
|
||||
|
||||
def test_backfill_slice_start_and_done(self):
|
||||
start = bc.parse_backfill("backfill slice: maxBatches=20")
|
||||
self.assertIsInstance(start, bc.BackfillSliceStart)
|
||||
self.assertEqual(start.max_batches, 20)
|
||||
done = bc.parse_backfill("backfill slice done: moreWork=true")
|
||||
self.assertIsInstance(done, bc.BackfillSliceDone)
|
||||
self.assertTrue(done.more_work)
|
||||
|
||||
def test_unparseable_messages_return_none(self):
|
||||
self.assertIsNone(bc.parse_imap_perf("nonsense here"))
|
||||
self.assertIsNone(bc.parse_open_message("openMessage missing fields"))
|
||||
self.assertIsNone(bc.parse_reader_ready("reader not ready"))
|
||||
self.assertIsNone(bc.parse_backfill("backfill mystery"))
|
||||
|
||||
|
||||
class TestParseBreadcrumbDispatch(unittest.TestCase):
|
||||
def test_dispatch_attaches_source_line(self):
|
||||
line = "07-05 18:07:18.435 10261 15306 D ImapPerf: body-fetch connect=2624ms work=26211ms live=3"
|
||||
ev = bc.parse_breadcrumb(line)
|
||||
self.assertIsInstance(ev, bc.ImapPerfOp)
|
||||
self.assertIsNotNone(ev.line)
|
||||
self.assertEqual(ev.line.pid, 10261)
|
||||
self.assertEqual(ev.timestamp, "07-05 18:07:18.435")
|
||||
|
||||
def test_dispatch_ignores_non_perf_tags(self):
|
||||
self.assertIsNone(
|
||||
bc.parse_breadcrumb("07-05 15:00:06.896 15192 15217 I MailSyncer: sync all: 1 accounts")
|
||||
)
|
||||
|
||||
|
||||
class TestFixtureStream(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.lines = _load_fixture_lines()
|
||||
self.events = list(bc.iter_events(self.lines))
|
||||
|
||||
def test_all_fixture_lines_are_perf_lines(self):
|
||||
self.assertEqual(sum(1 for line in self.lines if bc.is_perf_line(line)), len(self.lines))
|
||||
|
||||
def test_every_line_parses(self):
|
||||
self.assertEqual(len(self.events), len(self.lines))
|
||||
|
||||
def test_event_type_mix(self):
|
||||
kinds = [type(e).__name__ for e in self.events]
|
||||
# 3 prefetch/backfill-page ops + 2 body-fetch ops + 1 imap op = 6.
|
||||
self.assertEqual(kinds.count("ImapPerfOp"), 6)
|
||||
self.assertEqual(kinds.count("BodyFetch"), 2)
|
||||
self.assertEqual(kinds.count("OpenMessage"), 3)
|
||||
self.assertEqual(kinds.count("ReaderReady"), 3)
|
||||
self.assertEqual(kinds.count("BackfillProgress"), 1)
|
||||
self.assertEqual(kinds.count("BackfillSliceStart"), 1)
|
||||
self.assertEqual(kinds.count("BackfillSliceDone"), 1)
|
||||
|
||||
|
||||
class TestCorrelateOpens(unittest.TestCase):
|
||||
"""correlate_opens must reproduce the manual timing-tables rows exactly."""
|
||||
|
||||
def setUp(self):
|
||||
events = list(bc.iter_events(_load_fixture_lines()))
|
||||
self.samples = bc.correlate_opens(events)
|
||||
|
||||
def test_three_opens_detected(self):
|
||||
self.assertEqual(len(self.samples), 3)
|
||||
|
||||
def test_cached_gmail_open(self):
|
||||
s = self.samples[0]
|
||||
self.assertEqual(s.account_ref, "imap:94058a")
|
||||
self.assertTrue(s.cached)
|
||||
self.assertEqual(s.took_ms, 21)
|
||||
self.assertIsNone(s.body_fetch)
|
||||
self.assertEqual(s.reader_ready.took_ms, 82)
|
||||
|
||||
def test_gmail_A1_row_matches_table1(self):
|
||||
s = self.samples[1]
|
||||
self.assertEqual(s.account_ref, "imap:94058a")
|
||||
self.assertFalse(s.cached)
|
||||
self.assertEqual(s.took_ms, 31227)
|
||||
self.assertEqual(s.reader_ready.took_ms, 31617)
|
||||
self.assertEqual(s.rfc822_bytes, 60457)
|
||||
self.assertEqual(s.body_fetch.select_ms, 2566)
|
||||
self.assertEqual(s.body_fetch.body_ms, 14253)
|
||||
self.assertEqual(s.body_fetch.flag_ms, 2356)
|
||||
self.assertEqual(s.body_fetch_op.connect_ms, 2624)
|
||||
self.assertEqual(s.body_fetch_op.work_ms, 26211)
|
||||
self.assertEqual(s.body_fetch_op.live, 3)
|
||||
self.assertAlmostEqual(s.body_kb_per_s, 4.24, places=2)
|
||||
|
||||
def test_outlook_O1_row_matches_table2(self):
|
||||
s = self.samples[2]
|
||||
self.assertEqual(s.account_ref, "outlook:6b54d6")
|
||||
self.assertFalse(s.cached)
|
||||
self.assertEqual(s.took_ms, 2934)
|
||||
self.assertEqual(s.reader_ready.took_ms, 2939)
|
||||
self.assertEqual(s.rfc822_bytes, 189086)
|
||||
self.assertEqual(s.body_fetch_op.connect_ms, 772)
|
||||
self.assertAlmostEqual(s.body_kb_per_s, 139.5, places=1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,109 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Tests for aggregation and the timing-tables markdown renderers."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import breadcrumbs as bc # noqa: E402
|
||||
import report # noqa: E402
|
||||
|
||||
FIXTURE = os.path.join(os.path.dirname(__file__), "fixtures", "perf-extract-sample.log")
|
||||
|
||||
|
||||
class TestAggregate(unittest.TestCase):
|
||||
def test_basic_stats(self):
|
||||
agg = report.aggregate([10, 20, 30])
|
||||
self.assertEqual(agg.n, 3)
|
||||
self.assertEqual(agg.mean, 20)
|
||||
self.assertEqual(agg.median, 20)
|
||||
self.assertEqual((agg.minimum, agg.maximum), (10, 30))
|
||||
|
||||
def test_reproduces_manual_gmail_median(self):
|
||||
# timing-tables.md Condition A openMessage values -> median ~32.1 s.
|
||||
agg = report.aggregate([31227, 32059, 31721, 32148, 32949, 39728])
|
||||
self.assertEqual(agg.median, 32103.5)
|
||||
|
||||
def test_empty(self):
|
||||
agg = report.aggregate([])
|
||||
self.assertEqual(agg.n, 0)
|
||||
self.assertIsNone(agg.mean)
|
||||
self.assertIsNone(agg.median)
|
||||
|
||||
|
||||
class TestMdTable(unittest.TestCase):
|
||||
def test_table_shape(self):
|
||||
table = report.md_table(["a", "b"], [[1, 2], [3, None]])
|
||||
lines = table.splitlines()
|
||||
self.assertEqual(lines[0], "| a | b |")
|
||||
self.assertEqual(lines[1], "| --- | --- |")
|
||||
self.assertEqual(lines[2], "| 1 | 2 |")
|
||||
self.assertEqual(lines[3], "| 3 | |") # None renders empty
|
||||
|
||||
|
||||
class TestColdOpen(unittest.TestCase):
|
||||
def test_render_includes_mean_row(self):
|
||||
samples = [
|
||||
report.ColdOpenSample(run=1, total_time_ms=523, wait_time_ms=526),
|
||||
report.ColdOpenSample(run=2, total_time_ms=384, wait_time_ms=386),
|
||||
]
|
||||
md = report.render_cold_open(samples)
|
||||
self.assertIn("523", md)
|
||||
self.assertIn("**mean**", md)
|
||||
self.assertIn("454", md) # mean total = 453.5 -> 454
|
||||
|
||||
|
||||
class TestMessageOpen(unittest.TestCase):
|
||||
def setUp(self):
|
||||
with open(FIXTURE, encoding="utf-8") as fh:
|
||||
events = list(bc.iter_events(fh.read().splitlines()))
|
||||
self.samples = bc.correlate_opens(events)
|
||||
|
||||
def test_from_open_sample_maps_fields(self):
|
||||
row = report.ReaderOpenRow.from_open_sample(self.samples[1], index=1, label="A1")
|
||||
self.assertEqual(row.took_ms, 31227)
|
||||
self.assertEqual(row.reader_ready_ms, 31617)
|
||||
self.assertEqual(row.rfc822_bytes, 60457)
|
||||
self.assertEqual(row.connect_ms, 2624)
|
||||
self.assertEqual(row.select_ms, 2566)
|
||||
self.assertFalse(row.cached)
|
||||
|
||||
def test_render_message_open_table_and_aggregate(self):
|
||||
rows = [
|
||||
report.ReaderOpenRow.from_open_sample(s, index=i, label=f"m{i}")
|
||||
for i, s in enumerate(self.samples, start=1)
|
||||
]
|
||||
md = report.render_message_open("Message open (uncached)", rows)
|
||||
self.assertIn("31227 ms", md)
|
||||
self.assertIn("2934 ms", md)
|
||||
self.assertIn("60457", md)
|
||||
self.assertIn("4.2", md) # Gmail throughput
|
||||
self.assertIn("139.5", md) # Outlook throughput
|
||||
self.assertIn("n=2", md) # two uncached opens among the three
|
||||
|
||||
def test_skipped_row_renders(self):
|
||||
rows = [report.ReaderOpenRow(index=1, label="x", skipped=True, reason="keyguard")]
|
||||
md = report.render_message_open("t", rows)
|
||||
self.assertIn("SKIPPED: keyguard", md)
|
||||
|
||||
|
||||
class TestBackNav(unittest.TestCase):
|
||||
def test_caveat_present(self):
|
||||
samples = [report.BackNavSample(index=1, back_ms=3484)]
|
||||
md = report.render_back_nav(samples)
|
||||
self.assertIn("Caveat", md)
|
||||
self.assertIn("3484 ms", md)
|
||||
|
||||
|
||||
class TestDocument(unittest.TestCase):
|
||||
def test_build_document(self):
|
||||
doc = report.build_document({"scenario": "cold-open"}, ["## Section\n\nbody\n"])
|
||||
self.assertIn("# LibreMail device perf run", doc)
|
||||
self.assertIn("**scenario:** cold-open", doc)
|
||||
self.assertIn("## Section", doc)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,108 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Tests for the uiautomator-dump parser and screen/keyguard recognition.
|
||||
|
||||
Fixtures are cleaned copies of the manual run's real dumps: the reader (``ui_reader.xml``),
|
||||
a deskclock alarm (``ui_alarm.xml``) and the keyguard (``ui_lockscreen.xml``) are the
|
||||
foreign-screen negatives the harness must skip; ``ui_mailbox.xml`` is a compact, structurally
|
||||
faithful mailbox list.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import uidump # noqa: E402
|
||||
|
||||
FIX = os.path.join(os.path.dirname(__file__), "fixtures")
|
||||
APP = "org.libremail.app"
|
||||
|
||||
|
||||
def load(name):
|
||||
with open(os.path.join(FIX, name), encoding="utf-8") as fh:
|
||||
return uidump.parse_dump(fh.read())
|
||||
|
||||
|
||||
class TestBounds(unittest.TestCase):
|
||||
def test_parse_bounds(self):
|
||||
self.assertEqual(uidump.parse_bounds("[0,392][1344,617]"), (0, 392, 1344, 617))
|
||||
|
||||
def test_parse_bounds_none(self):
|
||||
self.assertIsNone(uidump.parse_bounds(""))
|
||||
self.assertIsNone(uidump.parse_bounds("not-bounds"))
|
||||
|
||||
|
||||
class TestMailbox(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.root = load("ui_mailbox.xml")
|
||||
|
||||
def test_recognised_as_app_not_lockscreen(self):
|
||||
self.assertEqual(uidump.foreground_package(self.root), APP)
|
||||
self.assertFalse(uidump.is_lockscreen(self.root))
|
||||
self.assertTrue(uidump.is_app_foreground(self.root, APP))
|
||||
self.assertFalse(uidump.is_reader(self.root, APP))
|
||||
|
||||
def test_finds_three_message_rows(self):
|
||||
rows = uidump.find_message_rows(self.root, APP)
|
||||
self.assertEqual(len(rows), 3)
|
||||
|
||||
def test_row_centers_and_cached_flags(self):
|
||||
rows = uidump.find_message_rows(self.root, APP)
|
||||
self.assertEqual(rows[0].center, (672, 504))
|
||||
self.assertFalse(rows[0].cached)
|
||||
self.assertEqual(rows[1].center, (672, 732))
|
||||
self.assertFalse(rows[1].cached)
|
||||
# Row 2 carries the "Available offline" content-desc -> cached.
|
||||
self.assertEqual(rows[2].center, (672, 988))
|
||||
self.assertTrue(rows[2].cached)
|
||||
|
||||
def test_row_label_drops_monogram(self):
|
||||
rows = uidump.find_message_rows(self.root, APP)
|
||||
self.assertIn("github-actions[bot]", rows[0].label)
|
||||
self.assertNotIn(" G ", rows[0].label) # single-letter avatar filtered out
|
||||
|
||||
def test_settings_entry_tappable(self):
|
||||
node = uidump.find_by_text(self.root, "Settings")
|
||||
self.assertIsNotNone(node)
|
||||
anchor = node.first_clickable_ancestor()
|
||||
self.assertIsNotNone(anchor)
|
||||
self.assertEqual(anchor.center, (1014, 2728))
|
||||
|
||||
def test_top_bar_content_descs(self):
|
||||
self.assertIsNotNone(uidump.find_by_content_desc(self.root, "Show folders"))
|
||||
self.assertIsNotNone(uidump.find_by_content_desc(self.root, "Search"))
|
||||
|
||||
|
||||
class TestReader(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.root = load("ui_reader.xml")
|
||||
|
||||
def test_recognised_as_reader(self):
|
||||
self.assertTrue(uidump.is_app_foreground(self.root, APP))
|
||||
self.assertTrue(uidump.is_reader(self.root, APP))
|
||||
|
||||
def test_progress_bar_signals_loading(self):
|
||||
self.assertTrue(uidump.has_progress_bar(self.root))
|
||||
|
||||
def test_no_message_rows_in_reader(self):
|
||||
self.assertEqual(uidump.find_message_rows(self.root, APP), [])
|
||||
|
||||
|
||||
class TestForeignScreensAreGuarded(unittest.TestCase):
|
||||
def test_lockscreen_detected(self):
|
||||
root = load("ui_lockscreen.xml")
|
||||
self.assertTrue(uidump.is_lockscreen(root))
|
||||
self.assertFalse(uidump.is_app_foreground(root, APP))
|
||||
self.assertFalse(uidump.is_reader(root, APP))
|
||||
|
||||
def test_alarm_is_foreign_app(self):
|
||||
root = load("ui_alarm.xml")
|
||||
self.assertEqual(uidump.foreground_package(root), "com.google.android.deskclock")
|
||||
self.assertFalse(uidump.is_app_foreground(root, APP))
|
||||
self.assertFalse(uidump.is_lockscreen(root))
|
||||
self.assertFalse(uidump.is_reader(root, APP))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,230 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""uidump.py -- parse ``uiautomator dump`` XML and recognise LibreMail's screens.
|
||||
|
||||
Pure (no device dependency) so it can be unit-tested against the saved dumps from the
|
||||
manual run. The Compose UI exposes no stable ``resource-id``s, so screens and rows are
|
||||
recognised structurally -- by class, clickable flags, bounds, and descendant
|
||||
text/content-desc -- exactly as observed in ``ui_mailbox.xml`` / ``ui_reader.xml``.
|
||||
|
||||
It also provides the keyguard / foreign-app guards the harness needs: a uiautomator sample
|
||||
taken against the lockscreen (``ui-mbox.xml`` in the manual run was a keyguard capture) or
|
||||
another app (``ui-03.xml`` was a deskclock alarm) must be detected and skipped.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
from collections import Counter
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Callable, List, Optional, Tuple
|
||||
|
||||
SYSTEMUI_PACKAGE = "com.android.systemui"
|
||||
_BOUNDS_RE = re.compile(r"\[(-?\d+),(-?\d+)\]\[(-?\d+),(-?\d+)\]")
|
||||
|
||||
|
||||
def parse_bounds(text: str) -> Optional[Tuple[int, int, int, int]]:
|
||||
"""``"[x1,y1][x2,y2]"`` -> ``(x1, y1, x2, y2)`` (or ``None``)."""
|
||||
m = _BOUNDS_RE.search(text or "")
|
||||
if not m:
|
||||
return None
|
||||
return tuple(int(g) for g in m.groups()) # type: ignore[return-value]
|
||||
|
||||
|
||||
@dataclass
|
||||
class UiNode:
|
||||
"""One uiautomator node with parsed attributes and parent/child links."""
|
||||
|
||||
cls: str = ""
|
||||
text: str = ""
|
||||
resource_id: str = ""
|
||||
content_desc: str = ""
|
||||
package: str = ""
|
||||
clickable: bool = False
|
||||
long_clickable: bool = False
|
||||
scrollable: bool = False
|
||||
enabled: bool = False
|
||||
bounds: Optional[Tuple[int, int, int, int]] = None
|
||||
parent: Optional["UiNode"] = None
|
||||
children: List["UiNode"] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def center(self) -> Optional[Tuple[int, int]]:
|
||||
if not self.bounds:
|
||||
return None
|
||||
x1, y1, x2, y2 = self.bounds
|
||||
return ((x1 + x2) // 2, (y1 + y2) // 2)
|
||||
|
||||
@property
|
||||
def height(self) -> int:
|
||||
return (self.bounds[3] - self.bounds[1]) if self.bounds else 0
|
||||
|
||||
def walk(self):
|
||||
"""Depth-first iterate over this node and all descendants."""
|
||||
yield self
|
||||
for child in self.children:
|
||||
yield from child.walk()
|
||||
|
||||
def find_all(self, predicate: Callable[["UiNode"], bool]) -> List["UiNode"]:
|
||||
return [n for n in self.walk() if predicate(n)]
|
||||
|
||||
def first_clickable_ancestor(self) -> Optional["UiNode"]:
|
||||
node: Optional[UiNode] = self
|
||||
while node is not None:
|
||||
if node.clickable:
|
||||
return node
|
||||
node = node.parent
|
||||
return None
|
||||
|
||||
def descendant_texts(self) -> List[str]:
|
||||
"""Non-empty ``text`` values under this node, in document order."""
|
||||
return [n.text for n in self.walk() if n.text]
|
||||
|
||||
def descendant_descs(self) -> List[str]:
|
||||
return [n.content_desc for n in self.walk() if n.content_desc]
|
||||
|
||||
|
||||
def _to_bool(value: Optional[str]) -> bool:
|
||||
return value == "true"
|
||||
|
||||
|
||||
def _build(elem: ET.Element, parent: Optional[UiNode]) -> UiNode:
|
||||
node = UiNode(
|
||||
cls=elem.get("class", ""),
|
||||
text=elem.get("text", ""),
|
||||
resource_id=elem.get("resource-id", ""),
|
||||
content_desc=elem.get("content-desc", ""),
|
||||
package=elem.get("package", ""),
|
||||
clickable=_to_bool(elem.get("clickable")),
|
||||
long_clickable=_to_bool(elem.get("long-clickable")),
|
||||
scrollable=_to_bool(elem.get("scrollable")),
|
||||
enabled=_to_bool(elem.get("enabled")),
|
||||
bounds=parse_bounds(elem.get("bounds", "")),
|
||||
parent=parent,
|
||||
)
|
||||
for child_elem in list(elem):
|
||||
if child_elem.tag == "node":
|
||||
node.children.append(_build(child_elem, node))
|
||||
return node
|
||||
|
||||
|
||||
def parse_dump(xml_text: str) -> UiNode:
|
||||
"""Parse uiautomator XML into a synthetic root :class:`UiNode` (the ``<hierarchy>``)."""
|
||||
root_elem = ET.fromstring(xml_text.strip())
|
||||
root = UiNode(cls="hierarchy")
|
||||
for child_elem in list(root_elem):
|
||||
if child_elem.tag == "node":
|
||||
root.children.append(_build(child_elem, root))
|
||||
return root
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Screen recognition
|
||||
# --------------------------------------------------------------------------- #
|
||||
def foreground_package(root: UiNode) -> str:
|
||||
"""The package that owns most of the tree -- a robust "what's on screen" signal."""
|
||||
counts = Counter(n.package for n in root.walk() if n.package)
|
||||
if not counts:
|
||||
return ""
|
||||
return counts.most_common(1)[0][0]
|
||||
|
||||
|
||||
def is_lockscreen(root: UiNode) -> bool:
|
||||
"""True if the dump is the keyguard/lockscreen rather than an app."""
|
||||
if foreground_package(root) != SYSTEMUI_PACKAGE:
|
||||
return False
|
||||
for node in root.walk():
|
||||
if "keyguard" in node.resource_id or node.content_desc in (
|
||||
"Lock screen",
|
||||
"Fingerprint sensor",
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def is_app_foreground(root: UiNode, package: str) -> bool:
|
||||
"""True if ``package`` is the foreground app in this dump."""
|
||||
return foreground_package(root) == package
|
||||
|
||||
|
||||
def find_by_text(root: UiNode, text: str, exact: bool = True) -> Optional[UiNode]:
|
||||
for node in root.walk():
|
||||
if (node.text == text) if exact else (text in node.text):
|
||||
return node
|
||||
return None
|
||||
|
||||
|
||||
def find_by_content_desc(root: UiNode, desc: str, exact: bool = True) -> Optional[UiNode]:
|
||||
for node in root.walk():
|
||||
if (node.content_desc == desc) if exact else (desc in node.content_desc):
|
||||
return node
|
||||
return None
|
||||
|
||||
|
||||
def is_reader(root: UiNode, package: str) -> bool:
|
||||
"""The message reader: a Back affordance plus the 'Message' title, in our package."""
|
||||
if not is_app_foreground(root, package):
|
||||
return False
|
||||
has_back = find_by_content_desc(root, "Back") is not None
|
||||
has_title = find_by_text(root, "Message") is not None
|
||||
return has_back and has_title
|
||||
|
||||
|
||||
def has_progress_bar(root: UiNode) -> bool:
|
||||
"""True if a ProgressBar is present (reader still loading its body -- a fallback signal)."""
|
||||
return any(n.cls.endswith("ProgressBar") for n in root.walk())
|
||||
|
||||
|
||||
@dataclass
|
||||
class MessageRow:
|
||||
"""A tappable message row in the mailbox list."""
|
||||
|
||||
index: int
|
||||
bounds: Tuple[int, int, int, int]
|
||||
center: Tuple[int, int]
|
||||
texts: List[str]
|
||||
cached: bool
|
||||
|
||||
@property
|
||||
def label(self) -> str:
|
||||
"""A short, log-safe identifier -- sender + subject when available."""
|
||||
if not self.texts:
|
||||
return f"row#{self.index}"
|
||||
if len(self.texts) >= 2:
|
||||
return f"{self.texts[0]} / {self.texts[-1]}"
|
||||
return self.texts[0]
|
||||
|
||||
|
||||
def find_message_rows(root: UiNode, package: str) -> List[MessageRow]:
|
||||
"""Return the tappable message rows in the mailbox's scrollable list.
|
||||
|
||||
Rows are the ``clickable`` + ``long-clickable`` children of the scrollable list (as in
|
||||
``ui_mailbox.xml``). A row carrying the ``"Available offline"`` content-desc has its
|
||||
body cached already, so it is marked ``cached`` (callers pick uncached rows for the
|
||||
uncached-open scenarios).
|
||||
"""
|
||||
scrollables = root.find_all(lambda n: n.scrollable and n.package == package)
|
||||
rows: List[MessageRow] = []
|
||||
seen_bounds = set()
|
||||
for scroller in scrollables:
|
||||
for child in scroller.children:
|
||||
if not (child.clickable and child.long_clickable and child.bounds):
|
||||
continue
|
||||
if child.bounds in seen_bounds:
|
||||
continue
|
||||
seen_bounds.add(child.bounds)
|
||||
texts = child.descendant_texts()
|
||||
# Drop single-letter avatar monograms; keep sender/subject/time.
|
||||
texts = [t for t in texts if len(t) > 1]
|
||||
cached = any(d == "Available offline" for d in child.descendant_descs())
|
||||
rows.append(
|
||||
MessageRow(
|
||||
index=len(rows),
|
||||
bounds=child.bounds,
|
||||
center=child.center, # type: ignore[arg-type]
|
||||
texts=texts,
|
||||
cached=cached,
|
||||
)
|
||||
)
|
||||
return rows
|
||||
Reference in New Issue
Block a user