Merge pull request #370 from JMR-dev/feat-device-testing-harness
feat(scripts): device-testing perf harness
This commit was merged in pull request #370.
This commit is contained in:
Jason Ross
@@ -30,6 +30,8 @@ secrets.properties
|
||||
|
||||
# Log Files
|
||||
*.log
|
||||
# ...but keep the device-testing parser fixtures (verbatim logcat slices used as test inputs)
|
||||
!scripts/device-testing/tests/fixtures/*.log
|
||||
|
||||
# Android Studio / IntelliJ
|
||||
.idea/
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
|
||||
# LibreMail device-testing perf harness
|
||||
|
||||
A cross-platform, **standard-library-only** Python tool that replicates LibreMail's
|
||||
on-device performance-test scenarios and logging capture. It codifies the methodology that
|
||||
was first run by hand (2026-07-05, Pixel 10 Pro XL) and written up in
|
||||
`scratchpad/perf/perf_summary.md`, `causation-report.md`, and `ab-run/timing-tables.md`.
|
||||
|
||||
Everything runs against **LibreMail only** plus read-only system-log/settings/dumpsys
|
||||
collection, behind hard device-safety guardrails (see [Device safety](#device-safety)).
|
||||
|
||||
## Requirements
|
||||
|
||||
- **Python 3.8+** (standard library only — no `pip install`, no third-party deps).
|
||||
- **`adb`** on `PATH` (or pass `--adb /path/to/adb`).
|
||||
- A connected device with:
|
||||
- **LibreMail installed** as a **debuggable** build (the cache-clear uses `run-as`, which
|
||||
only works on debuggable APKs), and
|
||||
- **at least one account signed in** with some **not-yet-cached** messages in the inbox
|
||||
(message bodies are fetched on first open), and
|
||||
- the screen **unlocked** (the harness keeps it awake during a run and guards every step
|
||||
against the keyguard, but it cannot get you *past* a secure lock screen).
|
||||
|
||||
No build step. Run it straight from the repo.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python scripts/device-testing/perf_harness.py <scenario> [options]
|
||||
```
|
||||
|
||||
Scenarios (each independently selectable):
|
||||
|
||||
| Scenario | What it does |
|
||||
|------------------|--------------|
|
||||
| `cold-open` | Force-stop LibreMail, clear **only** its `cache/`, `am start -W` ×N, parse `TotalTime`/`WaitTime`. |
|
||||
| `message-open` | Open N distinct **uncached** messages one at a time; time spinner→content from the breadcrumbs. |
|
||||
| `back-nav` | Time reader→mailbox back transitions ×N (dump-latency-bound; see caveat in the report). |
|
||||
| `prefetch-ab` | Run `message-open` under **Fetch all on Wi-Fi** (prefetch ON) vs **Always on-demand** (prefetch OFF), cache cleared between conditions. |
|
||||
| `cross-provider` | Open N messages from the (unified) inbox and tabulate per provider (`imap:…` vs `outlook:…`) from the breadcrumb account refs. |
|
||||
|
||||
Common options:
|
||||
|
||||
| Option | Default | Meaning |
|
||||
|--------|---------|---------|
|
||||
| `--serial <id>` | auto (if exactly one device) | choose the device |
|
||||
| `-n, --count <N>` | per-scenario | samples / runs (per condition for `prefetch-ab`) |
|
||||
| `--out <dir>` | `./device-perf-runs` | output root; a timestamped subdir is created per run |
|
||||
| `--package <pkg>` | `org.libremail.app` | target package |
|
||||
| `--component <c>` | `org.libremail.app/org.libremail.MainActivity` | launcher component |
|
||||
| `--adb <path>` | `adb` | path to the adb executable |
|
||||
| `--dry-run` | off | **print the exact command plan without changing device state** |
|
||||
|
||||
**Always start with `--dry-run`** to review the command plan a scenario will issue:
|
||||
|
||||
```bash
|
||||
python scripts/device-testing/perf_harness.py prefetch-ab --dry-run
|
||||
python scripts/device-testing/perf_harness.py cold-open -n 5 --serial 5C310DLCQ000G3
|
||||
```
|
||||
|
||||
## What each scenario measures
|
||||
|
||||
Timing comes **primarily from the on-device breadcrumbs** (PII-free), with uiautomator used
|
||||
only as a "content is ready" signal so the driver knows when to move on:
|
||||
|
||||
- `MailReader: openMessage <acctRef> folder=<label> fetchedBody=<bool> took=<ms>ms`
|
||||
— end-to-end reader open. `fetchedBody=true` ⇒ a real network body fetch (uncached).
|
||||
- `Reader: reader ready took=<ms>ms html=<bool> inline=<n>` — spinner→content.
|
||||
- `ImapPerf: <op> connect=<ms>ms work=<ms>ms live=<N>` and
|
||||
`ImapPerf: body-fetch select=<ms>ms body=<ms>ms flag=<ms>ms rfc822=<n>B chars=<n> att=<n>`
|
||||
— connection + phase split; `body KB/s` is `rfc822 / body_ms`.
|
||||
- `MailBackfiller: backfill … pages=<n> complete=<bool>` / `backfill slice…` — backfill activity.
|
||||
|
||||
`cold-open` instead parses `am start -W`'s `TotalTime` / `WaitTime`.
|
||||
|
||||
## Output
|
||||
|
||||
Each run writes a timestamped directory under `--out`:
|
||||
|
||||
```
|
||||
device-perf-runs/20260706-131612-cold-open/
|
||||
├── timing-tables.md # per-scenario tables + aggregates (mirrors the manual timing-tables.md)
|
||||
├── session-raw.log # the full `adb logcat -b all -v threadtime` stream for the run
|
||||
├── perf-extract.log # the ImapPerf|MailReader|Reader|MailBackfiller subset of the raw log
|
||||
└── driver.log # what the harness did, step by step
|
||||
```
|
||||
|
||||
## Device safety
|
||||
|
||||
Every device call goes through a guarded `adb` wrapper (`adb.py`). Two independent layers
|
||||
mean a dangerous command **cannot be constructed**:
|
||||
|
||||
- **Allow-list** of adb subcommands: `devices`, `get-state`, `install`, `shell`, `logcat`,
|
||||
`wait-for-device`, `start-server`. Anything else (`uninstall`, `root`, `remount`,
|
||||
`reboot`, `disable-verity`, `emu`, `push`, `pull`, …) is refused.
|
||||
- **Deny-list + assertions** on every `shell` command: no `pm clear` / `pm uninstall`, no
|
||||
reboot/remount/root/verity/factory-reset, **no touching the app's `databases/` / `files/`
|
||||
/ `shared_prefs/` / `datastore/`**, no output redirects, and `run-as` / `am force-stop` /
|
||||
`am start` are constrained to the target package.
|
||||
|
||||
The **only** sanctioned mutation of app state is clearing LibreMail's own **`cache/`**
|
||||
(`run-as org.libremail.app sh -c 'rm -rf cache/*'`) — an exact-match allow-list; any other
|
||||
`rm`/`mv`/`dd`/… is refused. There is **no** `pm clear`, uninstall, or data wipe anywhere.
|
||||
|
||||
The screen is kept awake (`svc power stayon true` + `KEYCODE_WAKEUP`) for the run and
|
||||
restored afterwards, and every uiautomator/input step is guarded against the keyguard and
|
||||
against a foreign app being in the foreground — a sample taken against either is **skipped**,
|
||||
not measured (the manual run hit exactly these: a lock-screen dump and a deskclock alarm).
|
||||
|
||||
## Tests
|
||||
|
||||
Pure-logic modules (the breadcrumb parser, the uiautomator parser, the safety guardrails,
|
||||
the report renderers) are unit-tested with the standard-library `unittest` against the
|
||||
**real captures** from the manual run:
|
||||
|
||||
```bash
|
||||
python -m unittest discover -s scripts/device-testing/tests -p "test_*.py"
|
||||
```
|
||||
|
||||
`tests/fixtures/perf-extract-sample.log` is a verbatim slice of the manual run's
|
||||
`perf-extract-ALL.log`, so the parser tests assert the harness reproduces the exact figures
|
||||
in the hand-written `timing-tables.md` (e.g. Gmail A1: `took=31227 ms`, `rfc822=60457 B`,
|
||||
`4.2 KB/s`; Outlook O1: `took=2934 ms`, `139.5 KB/s`). The UI fixtures include the reader,
|
||||
plus the lockscreen and deskclock-alarm negatives the keyguard/foreground guards must catch.
|
||||
|
||||
## Validated vs. needs the live run
|
||||
|
||||
**Validated offline** (by the unit tests, no device):
|
||||
|
||||
- Breadcrumb parsing + open-correlation reproduce the manual `timing-tables.md` figures.
|
||||
- The safety guardrails accept the known-good commands and refuse every forbidden one.
|
||||
- Screen recognition (mailbox rows + cached flag, reader, lockscreen, foreign app).
|
||||
- The report renders the same tables/aggregates as the manual write-up.
|
||||
- `--dry-run` emits the correct command plan for all five scenarios.
|
||||
|
||||
**Needs the first monitored live run** (a device makes the state real):
|
||||
|
||||
- End-to-end timing capture on hardware (streamed logcat → per-sample breadcrumb tailing).
|
||||
- **Settings-screen navigation for `prefetch-ab`.** No uiautomator dump of the settings
|
||||
screen was captured in the manual run, so `set_fetch_policy` navigates by the on-screen
|
||||
option text (`"Fetch all on Wi-Fi"`, `"Always on-demand"`, from `res/values/strings.xml`)
|
||||
with a scroll fallback. The bottom-nav "Settings" tap target is confirmed from
|
||||
`ui_mailbox.xml`; the option rows themselves need one live confirmation.
|
||||
- Row selection under a live, scrolling list and the auto-lock recovery path.
|
||||
|
||||
## Notes / design decisions
|
||||
|
||||
- **Uncached opens.** Message bodies live in the Room DB (`libremail.db`), **not** in
|
||||
`cache/`, so bodies can't be force-uncached without touching `databases/` (forbidden).
|
||||
The harness therefore opens **naturally-uncached** messages and verifies each was a real
|
||||
network fetch via the `fetchedBody=true` breadcrumb — exactly as the manual run did.
|
||||
- **Cross-provider** uses the unified inbox: a single mixed pass yields both providers, and
|
||||
the harness buckets rows by the breadcrumb account ref (`imap:…` vs `outlook:…`) — no
|
||||
account switching required.
|
||||
- **Back-nav** timings are dominated by the ~2.5–3 s uiautomator-dump latency floor; the
|
||||
report labels them accordingly (true in-app back is sub-second and not resolvable via adb
|
||||
UI polling under load).
|
||||
- Dev-script convention: Python 3, standard library only, cross-platform (Windows-primary),
|
||||
matching `.claude/skills/preflight/*.py`.
|
||||
@@ -0,0 +1,370 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""adb.py -- a deliberately narrow, safety-guarded wrapper around ``adb``.
|
||||
|
||||
Every device interaction in the harness goes through :class:`Adb`. The wrapper enforces
|
||||
two independent layers so a dangerous command *cannot be constructed*, even by a buggy
|
||||
caller:
|
||||
|
||||
* a TOP-LEVEL ALLOW-LIST of adb subcommands (``devices``, ``get-state``, ``install``,
|
||||
``shell``, ``logcat``, ``wait-for-device``) -- anything else raises :class:`AdbSafetyError`;
|
||||
* a DENY-LIST + assertions applied to every ``shell`` command: no ``pm clear`` / ``pm
|
||||
uninstall``, no ``reboot`` / ``remount`` / ``root`` / ``disable-verity`` / factory reset,
|
||||
no touching the app's ``databases/`` / ``files/`` / ``shared_prefs/`` / ``datastore/``,
|
||||
no output redirects, and the ONLY sanctioned destructive operation is clearing the app's
|
||||
own ``cache/`` (an exact-match allow-list of ``run-as <pkg> sh -c 'rm -rf cache/*'``).
|
||||
|
||||
``am force-stop`` / ``am start`` and ``run-as`` are constrained to the target package
|
||||
(default ``org.libremail.app``). All checks run in ``--dry-run`` mode too, so the guardrails
|
||||
are unit-testable without a device.
|
||||
|
||||
Scope: this harness operates ONLY on LibreMail plus read-only system-log/settings/dumpsys
|
||||
collection. It never writes app data other than clearing the cache.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import List, Optional, Sequence
|
||||
|
||||
DEFAULT_PACKAGE = "org.libremail.app"
|
||||
DEFAULT_COMPONENT = "org.libremail.app/org.libremail.MainActivity"
|
||||
|
||||
# adb subcommands the harness is ever allowed to invoke.
|
||||
_ALLOWED_SUBCOMMANDS = frozenset(
|
||||
{"devices", "get-state", "install", "shell", "logcat", "wait-for-device", "start-server"}
|
||||
)
|
||||
|
||||
# Substrings that must never appear anywhere in a shell command line.
|
||||
_DENY_SUBSTRINGS = (
|
||||
"pm clear",
|
||||
"pm uninstall",
|
||||
"pm disable",
|
||||
"pm hide",
|
||||
"disable-verity",
|
||||
"set-verity",
|
||||
"remount",
|
||||
"reboot",
|
||||
"bootloader",
|
||||
"fastboot",
|
||||
"factory",
|
||||
"wipe-data",
|
||||
"wipe_data",
|
||||
"mkfs",
|
||||
"format ",
|
||||
# App-private dirs we must never read or write (cache/ is the sole exception, handled
|
||||
# separately). Banning the segment names outright is defence-in-depth for this harness,
|
||||
# which never has a legitimate reason to touch them.
|
||||
"databases",
|
||||
"shared_prefs",
|
||||
"datastore",
|
||||
"/files",
|
||||
"files/",
|
||||
)
|
||||
|
||||
# Verbs that mutate the filesystem. Detected as whole *words* anywhere in the shell command
|
||||
# (they can be buried inside a ``sh -c '<payload>'`` token); permitted ONLY for the exact
|
||||
# sanctioned cache-clear (see _is_sanctioned_cache_clear).
|
||||
_DESTRUCTIVE_VERBS = ("rm", "rmdir", "mv", "dd", "truncate", "shred", "unlink", "mkfs", "chmod", "chown")
|
||||
_DESTRUCTIVE_RE = re.compile(r"\b(?:" + "|".join(_DESTRUCTIVE_VERBS) + r")\b")
|
||||
|
||||
# The exact ``sh -c`` payloads permitted for the cache clear -- nothing else.
|
||||
_CACHE_CLEAR_PAYLOADS = ("rm -rf cache/*", "rm -rf cache")
|
||||
|
||||
|
||||
class AdbSafetyError(RuntimeError):
|
||||
"""Raised when a command would violate the harness's device-safety guardrails."""
|
||||
|
||||
|
||||
class AdbError(RuntimeError):
|
||||
"""Raised when an adb command fails (non-zero exit) and the caller wanted a check."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class AdbResult:
|
||||
"""Outcome of one adb invocation."""
|
||||
|
||||
args: List[str]
|
||||
returncode: int
|
||||
stdout: str
|
||||
stderr: str
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return self.returncode == 0
|
||||
|
||||
|
||||
def _shell_command_string(args: Sequence[str]) -> str:
|
||||
"""Join the tokens *after* ``shell`` into a single inspection string."""
|
||||
return " ".join(args[1:]) if len(args) > 1 else ""
|
||||
|
||||
|
||||
def _is_sanctioned_cache_clear(package: str, args: Sequence[str]) -> bool:
|
||||
"""True iff ``args`` is exactly ``shell run-as <package> sh -c '<cache-clear>'``."""
|
||||
return (
|
||||
len(args) == 6
|
||||
and args[0] == "shell"
|
||||
and args[1] == "run-as"
|
||||
and args[2] == package
|
||||
and args[3] == "sh"
|
||||
and args[4] == "-c"
|
||||
and args[5] in _CACHE_CLEAR_PAYLOADS
|
||||
)
|
||||
|
||||
|
||||
def assert_safe(package: str, args: Sequence[str]) -> None:
|
||||
"""Raise :class:`AdbSafetyError` if ``args`` (an adb argv, sans ``adb``/``-s``) is unsafe.
|
||||
|
||||
Pure and side-effect-free so the guardrails can be unit-tested exhaustively.
|
||||
"""
|
||||
if not args:
|
||||
raise AdbSafetyError("empty adb command")
|
||||
|
||||
subcommand = args[0]
|
||||
if subcommand not in _ALLOWED_SUBCOMMANDS:
|
||||
raise AdbSafetyError(f"adb subcommand {subcommand!r} is not on the allow-list")
|
||||
|
||||
# 'install' may only take known-safe flags plus a path; it is never 'uninstall'.
|
||||
if subcommand == "install":
|
||||
for tok in args[1:]:
|
||||
if tok.startswith("-") and tok not in ("-r", "-t", "-g", "-d", "-i"):
|
||||
raise AdbSafetyError(f"install flag {tok!r} is not allowed")
|
||||
return
|
||||
|
||||
if subcommand != "shell":
|
||||
# devices / get-state / logcat / wait-for-device / start-server take no risky args.
|
||||
return
|
||||
|
||||
sanctioned_cache_clear = _is_sanctioned_cache_clear(package, args)
|
||||
sh = _shell_command_string(args)
|
||||
lowered = sh.lower()
|
||||
|
||||
for bad in _DENY_SUBSTRINGS:
|
||||
if bad in lowered:
|
||||
raise AdbSafetyError(f"shell command contains forbidden text {bad!r}: {sh!r}")
|
||||
|
||||
if ">" in sh:
|
||||
raise AdbSafetyError(f"shell output redirection is not allowed: {sh!r}")
|
||||
|
||||
tokens = list(args[1:])
|
||||
if _DESTRUCTIVE_RE.search(sh) and not sanctioned_cache_clear:
|
||||
raise AdbSafetyError(
|
||||
"the only permitted destructive shell command is clearing the app cache; "
|
||||
f"refusing: {sh!r}"
|
||||
)
|
||||
|
||||
# run-as must target our package and (unless the sanctioned cache clear) be a read.
|
||||
if "run-as" in tokens:
|
||||
idx = tokens.index("run-as")
|
||||
target = tokens[idx + 1] if idx + 1 < len(tokens) else None
|
||||
if target != package:
|
||||
raise AdbSafetyError(f"run-as may only target {package!r}, got {target!r}: {sh!r}")
|
||||
|
||||
# am force-stop / am start may only reference our package/component.
|
||||
if "am" in tokens:
|
||||
if "force-stop" in tokens:
|
||||
i = tokens.index("force-stop")
|
||||
target = tokens[i + 1] if i + 1 < len(tokens) else None
|
||||
if target != package:
|
||||
raise AdbSafetyError(f"am force-stop must target {package!r}: {sh!r}")
|
||||
if "-n" in tokens: # component: <package>/<activity>
|
||||
i = tokens.index("-n")
|
||||
component = tokens[i + 1] if i + 1 < len(tokens) else ""
|
||||
comp_pkg = component.split("/", 1)[0]
|
||||
if comp_pkg != package:
|
||||
raise AdbSafetyError(f"am start component must be in {package!r}: {sh!r}")
|
||||
for pkg_flag in ("-p", "--package"):
|
||||
if pkg_flag in tokens:
|
||||
i = tokens.index(pkg_flag)
|
||||
target = tokens[i + 1] if i + 1 < len(tokens) else None
|
||||
if target != package:
|
||||
raise AdbSafetyError(f"am {pkg_flag} must be {package!r}: {sh!r}")
|
||||
|
||||
|
||||
class Adb:
|
||||
"""Guarded adb wrapper bound to a single device serial and target package."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
serial: Optional[str] = None,
|
||||
package: str = DEFAULT_PACKAGE,
|
||||
adb_path: str = "adb",
|
||||
dry_run: bool = False,
|
||||
default_timeout: float = 60.0,
|
||||
logger=None,
|
||||
) -> None:
|
||||
self.serial = serial
|
||||
self.package = package
|
||||
self.adb_path = adb_path
|
||||
self.dry_run = dry_run
|
||||
self.default_timeout = default_timeout
|
||||
self._log = logger or (lambda msg: None)
|
||||
|
||||
# -- core ---------------------------------------------------------------- #
|
||||
def _argv(self, args: Sequence[str]) -> List[str]:
|
||||
prefix = [self.adb_path]
|
||||
if self.serial:
|
||||
prefix += ["-s", self.serial]
|
||||
return prefix + list(args)
|
||||
|
||||
def run(
|
||||
self,
|
||||
args: Sequence[str],
|
||||
check: bool = False,
|
||||
timeout: Optional[float] = None,
|
||||
) -> AdbResult:
|
||||
"""Validate and execute an adb command (``args`` excludes ``adb`` and ``-s <serial>``)."""
|
||||
assert_safe(self.package, args)
|
||||
argv = self._argv(args)
|
||||
printable = " ".join(argv)
|
||||
if self.dry_run:
|
||||
self._log(f"[dry-run] {printable}")
|
||||
return AdbResult(list(args), 0, "", "")
|
||||
self._log(f"$ {printable}")
|
||||
completed = subprocess.run(
|
||||
argv,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout if timeout is not None else self.default_timeout,
|
||||
)
|
||||
result = AdbResult(list(args), completed.returncode, completed.stdout, completed.stderr)
|
||||
if check and not result.ok:
|
||||
raise AdbError(f"adb failed ({result.returncode}): {printable}\n{result.stderr}")
|
||||
return result
|
||||
|
||||
# -- device discovery ---------------------------------------------------- #
|
||||
def devices(self) -> List[str]:
|
||||
"""Return the serials of attached, ready devices."""
|
||||
out = self.run(["devices"]).stdout
|
||||
return parse_devices(out)
|
||||
|
||||
def get_state(self) -> str:
|
||||
return self.run(["get-state"]).stdout.strip()
|
||||
|
||||
# -- lifecycle (LibreMail only) ------------------------------------------ #
|
||||
def force_stop(self) -> AdbResult:
|
||||
return self.run(["shell", "am", "force-stop", self.package])
|
||||
|
||||
def start_activity(
|
||||
self, component: str = DEFAULT_COMPONENT, wait: bool = True
|
||||
) -> AdbResult:
|
||||
args = ["shell", "am", "start"]
|
||||
if wait:
|
||||
args.append("-W")
|
||||
args += ["-n", component]
|
||||
return self.run(args)
|
||||
|
||||
def clear_cache(self) -> AdbResult:
|
||||
"""Clear ONLY the app's ``cache/`` via run-as. The sole sanctioned mutation."""
|
||||
return self.run(["shell", "run-as", self.package, "sh", "-c", "rm -rf cache/*"])
|
||||
|
||||
def list_cache(self) -> AdbResult:
|
||||
"""Read-only listing of the app cache dir (for verification/logging)."""
|
||||
return self.run(["shell", "run-as", self.package, "ls", "-la", "cache"])
|
||||
|
||||
# -- input / UI ---------------------------------------------------------- #
|
||||
def input_tap(self, x: int, y: int) -> AdbResult:
|
||||
return self.run(["shell", "input", "tap", str(x), str(y)])
|
||||
|
||||
def input_swipe(self, x1: int, y1: int, x2: int, y2: int, ms: int = 300) -> AdbResult:
|
||||
return self.run(
|
||||
["shell", "input", "swipe", str(x1), str(y1), str(x2), str(y2), str(ms)]
|
||||
)
|
||||
|
||||
def input_keyevent(self, keycode: str) -> AdbResult:
|
||||
return self.run(["shell", "input", "keyevent", str(keycode)])
|
||||
|
||||
def uiautomator_dump(self) -> str:
|
||||
"""Return the current window's uiautomator XML (via ``dump /dev/tty``)."""
|
||||
out = self.run(["shell", "uiautomator", "dump", "/dev/tty"], timeout=90).stdout
|
||||
return _extract_xml(out)
|
||||
|
||||
# -- screen / keyguard --------------------------------------------------- #
|
||||
def stay_on(self, on: bool = True) -> AdbResult:
|
||||
return self.run(["shell", "svc", "power", "stayon", "true" if on else "false"])
|
||||
|
||||
def wake(self) -> AdbResult:
|
||||
return self.input_keyevent("KEYCODE_WAKEUP")
|
||||
|
||||
def dumpsys(self, service: str, *extra: str) -> AdbResult:
|
||||
return self.run(["shell", "dumpsys", service, *extra])
|
||||
|
||||
def settings_get(self, namespace: str, key: str) -> str:
|
||||
return self.run(["shell", "settings", "get", namespace, key]).stdout.strip()
|
||||
|
||||
# -- logcat -------------------------------------------------------------- #
|
||||
def clear_logcat(self) -> AdbResult:
|
||||
return self.run(["shell", "logcat", "-c"])
|
||||
|
||||
def start_logcat(self, out_file) -> Optional[subprocess.Popen]:
|
||||
"""Start streaming the full logcat to an already-open file handle.
|
||||
|
||||
Returns the :class:`subprocess.Popen` (or ``None`` in dry-run). Stop it with
|
||||
:meth:`stop_logcat`.
|
||||
"""
|
||||
args = ["logcat", "-b", "all", "-v", "threadtime"]
|
||||
assert_safe(self.package, args)
|
||||
argv = self._argv(args)
|
||||
if self.dry_run:
|
||||
self._log(f"[dry-run] {' '.join(argv)} > <session-raw.log>")
|
||||
return None
|
||||
self._log(f"$ {' '.join(argv)} > <session-raw.log>")
|
||||
return subprocess.Popen(argv, stdout=out_file, stderr=subprocess.DEVNULL)
|
||||
|
||||
def settle(self, seconds: float) -> None:
|
||||
"""Sleep to let device state settle -- a no-op in dry-run so previews are instant."""
|
||||
if not self.dry_run and seconds > 0:
|
||||
time.sleep(seconds)
|
||||
|
||||
@staticmethod
|
||||
def stop_logcat(proc: Optional[subprocess.Popen]) -> None:
|
||||
if proc is None:
|
||||
return
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Pure parsers / helpers (unit-tested)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def parse_devices(output: str) -> List[str]:
|
||||
"""Parse ``adb devices`` output into a list of ready serials (state ``device``)."""
|
||||
serials: List[str] = []
|
||||
for line in output.splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("List of devices"):
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) >= 2 and parts[1] == "device":
|
||||
serials.append(parts[0])
|
||||
return serials
|
||||
|
||||
|
||||
_AM_FIELD_RE = re.compile(r"^\s*(TotalTime|WaitTime|ThisTime):\s*(\d+)\s*$")
|
||||
|
||||
|
||||
def parse_am_start(output: str) -> dict:
|
||||
"""Extract ``TotalTime`` / ``WaitTime`` / ``ThisTime`` (ms) from ``am start -W`` output."""
|
||||
result: dict = {}
|
||||
for line in output.splitlines():
|
||||
m = _AM_FIELD_RE.match(line)
|
||||
if m:
|
||||
result[m.group(1)] = int(m.group(2))
|
||||
return result
|
||||
|
||||
|
||||
def _extract_xml(raw: str) -> str:
|
||||
"""Pull the ``<?xml ...</hierarchy>`` payload out of ``uiautomator dump /dev/tty`` output."""
|
||||
start = raw.find("<?xml")
|
||||
if start == -1:
|
||||
start = raw.find("<hierarchy")
|
||||
end = raw.rfind("</hierarchy>")
|
||||
if start == -1 or end == -1:
|
||||
return raw.strip()
|
||||
return raw[start : end + len("</hierarchy>")]
|
||||
@@ -0,0 +1,422 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""breadcrumbs.py -- pure parser for LibreMail on-device perf breadcrumbs.
|
||||
|
||||
This module is the *core* of the device-testing harness and is intentionally free of any
|
||||
I/O, ``adb``, or device dependency so it can be unit-tested against the saved logcat
|
||||
captures from the manual perf run (see ``tests/test_breadcrumbs.py`` and
|
||||
``tests/fixtures/perf-extract-sample.log``).
|
||||
|
||||
It parses two things:
|
||||
|
||||
1. A raw ``adb logcat -v threadtime`` line into its fields (:func:`parse_logcat_line`).
|
||||
2. The message payload of the four perf tags into typed events
|
||||
(:func:`parse_breadcrumb` and the per-tag helpers).
|
||||
|
||||
Breadcrumb formats (verbatim from the app source, all PII-free):
|
||||
|
||||
* ``ImapPerf: <op> connect=<ms>ms work=<ms>ms live=<N>``
|
||||
-- ``ImapClient`` generic per-operation timing; ``op`` is a short label
|
||||
(``prefetch-body``, ``backfill-page``, ``body-fetch``, ``imap`` ...).
|
||||
* ``ImapPerf: body-fetch select=<ms>ms body=<ms>ms flag=<ms>ms rfc822=<n>B chars=<n> att=<n>``
|
||||
-- ``ImapClient`` detailed body-fetch phase split (emitted just before the generic
|
||||
``body-fetch connect=.. work=.. live=..`` line for the same fetch).
|
||||
* ``MailReader: openMessage <acctRef> folder=<label> fetchedBody=<bool> took=<ms>ms``
|
||||
-- ``MailRepositoryImpl`` reader-open; ``fetchedBody=true`` means a real network body
|
||||
fetch happened (i.e. the message was *not* cached).
|
||||
* ``Reader: reader ready took=<ms>ms html=<bool> inline=<n>``
|
||||
-- ``ReaderViewModel`` spinner-to-content time.
|
||||
* ``MailBackfiller: backfill <acctRef> folder=<label> pages=<n> complete=<bool>``
|
||||
* ``MailBackfiller: backfill slice: maxBatches=<n>``
|
||||
* ``MailBackfiller: backfill slice done: moreWork=<bool>``
|
||||
|
||||
The threadtime tag column is space-padded by logcat (e.g. ``Reader :``); the parser
|
||||
tolerates that and reports the trimmed tag (``Reader``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Iterable, Iterator, List, Optional, Union
|
||||
|
||||
# Tags that make up the filtered perf extract (mirrors the manual
|
||||
# `grep -E 'ImapPerf|MailReader|Reader|MailBackfiller'`). Matched on the *tag* column so a
|
||||
# stray mention of one of these words inside another tag's message is not miscounted.
|
||||
PERF_TAGS = ("ImapPerf", "MailReader", "Reader", "MailBackfiller")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Raw logcat line (``-v threadtime``)
|
||||
# --------------------------------------------------------------------------- #
|
||||
_THREADTIME_RE = re.compile(
|
||||
r"^(?P<date>\d{2}-\d{2})\s+"
|
||||
r"(?P<time>\d{2}:\d{2}:\d{2}\.\d{3})\s+"
|
||||
r"(?P<pid>\d+)\s+"
|
||||
r"(?P<tid>\d+)\s+"
|
||||
r"(?P<level>[VDIWEFS])\s+"
|
||||
r"(?P<tag>[^:]+?)\s*:\s?"
|
||||
r"(?P<message>.*)$"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LogLine:
|
||||
"""A single ``adb logcat -v threadtime`` line, split into fields."""
|
||||
|
||||
date: str
|
||||
time: str
|
||||
pid: int
|
||||
tid: int
|
||||
level: str
|
||||
tag: str
|
||||
message: str
|
||||
raw: str
|
||||
|
||||
@property
|
||||
def timestamp(self) -> str:
|
||||
"""`MM-DD HH:MM:SS.mmm` -- the wall-clock stamp (no year in logcat)."""
|
||||
return f"{self.date} {self.time}"
|
||||
|
||||
|
||||
def parse_logcat_line(line: str) -> Optional[LogLine]:
|
||||
"""Parse one threadtime logcat line; return ``None`` if it is not one."""
|
||||
m = _THREADTIME_RE.match(line.rstrip("\r\n"))
|
||||
if not m:
|
||||
return None
|
||||
return LogLine(
|
||||
date=m.group("date"),
|
||||
time=m.group("time"),
|
||||
pid=int(m.group("pid")),
|
||||
tid=int(m.group("tid")),
|
||||
level=m.group("level"),
|
||||
tag=m.group("tag").strip(),
|
||||
message=m.group("message").strip(),
|
||||
raw=line.rstrip("\r\n"),
|
||||
)
|
||||
|
||||
|
||||
def is_perf_line(line: str) -> bool:
|
||||
"""True if ``line`` is a threadtime line whose tag is one of :data:`PERF_TAGS`."""
|
||||
parsed = parse_logcat_line(line)
|
||||
return parsed is not None and parsed.tag in PERF_TAGS
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Typed breadcrumb events
|
||||
# --------------------------------------------------------------------------- #
|
||||
class Breadcrumb:
|
||||
"""Base type: every event optionally carries its source :class:`LogLine`.
|
||||
|
||||
A plain (non-dataclass) base so each subclass can declare its own fields with ``line``
|
||||
last -- this keeps the generated ``__init__`` valid on Python 3.7+ without the
|
||||
3.10-only ``kw_only`` field option (a base dataclass field with a default would force a
|
||||
"non-default argument follows default argument" error in the subclasses).
|
||||
"""
|
||||
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
@property
|
||||
def timestamp(self) -> Optional[str]:
|
||||
return self.line.timestamp if self.line else None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ImapPerfOp(Breadcrumb):
|
||||
"""``ImapPerf: <op> connect=<ms>ms work=<ms>ms live=<N>``."""
|
||||
|
||||
op: str
|
||||
connect_ms: int
|
||||
work_ms: int
|
||||
live: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BodyFetch(Breadcrumb):
|
||||
"""``ImapPerf: body-fetch select=.. body=.. flag=.. rfc822=..B chars=.. att=..``."""
|
||||
|
||||
select_ms: int
|
||||
body_ms: int
|
||||
flag_ms: int
|
||||
rfc822_bytes: int
|
||||
chars: int
|
||||
att: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
@property
|
||||
def body_kb_per_s(self) -> Optional[float]:
|
||||
"""Body-download throughput in decimal KB/s; ``None`` if body_ms==0.
|
||||
|
||||
``bytes/ms`` is already KB/s (``bytes/ms * 1000 ms/s / 1000 B/KB``), so this mirrors
|
||||
the manual ``timing-tables.md`` "body KB/s" column exactly (e.g. 60457/14253 = 4.2).
|
||||
"""
|
||||
if self.body_ms <= 0:
|
||||
return None
|
||||
return self.rfc822_bytes / self.body_ms
|
||||
|
||||
|
||||
@dataclass
|
||||
class OpenMessage(Breadcrumb):
|
||||
"""``MailReader: openMessage <ref> folder=<label> fetchedBody=<bool> took=<ms>ms``."""
|
||||
|
||||
account_ref: str
|
||||
folder: str
|
||||
fetched_body: bool
|
||||
took_ms: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ReaderReady(Breadcrumb):
|
||||
"""``Reader: reader ready took=<ms>ms html=<bool> inline=<n>``."""
|
||||
|
||||
took_ms: int
|
||||
html: bool
|
||||
inline: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackfillProgress(Breadcrumb):
|
||||
"""``MailBackfiller: backfill <ref> folder=<label> pages=<n> complete=<bool>``."""
|
||||
|
||||
account_ref: str
|
||||
folder: str
|
||||
pages: int
|
||||
complete: bool
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackfillSliceStart(Breadcrumb):
|
||||
"""``MailBackfiller: backfill slice: maxBatches=<n>``."""
|
||||
|
||||
max_batches: int
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class BackfillSliceDone(Breadcrumb):
|
||||
"""``MailBackfiller: backfill slice done: moreWork=<bool>``."""
|
||||
|
||||
more_work: bool
|
||||
line: Optional[LogLine] = None
|
||||
|
||||
|
||||
Event = Union[
|
||||
ImapPerfOp,
|
||||
BodyFetch,
|
||||
OpenMessage,
|
||||
ReaderReady,
|
||||
BackfillProgress,
|
||||
BackfillSliceStart,
|
||||
BackfillSliceDone,
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Message-payload parsers (pure; operate on the trimmed message string)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _to_bool(text: str) -> bool:
|
||||
return text == "true"
|
||||
|
||||
|
||||
_IMAP_OP_RE = re.compile(
|
||||
r"^(?P<op>\S+)\s+connect=(?P<connect>\d+)ms\s+work=(?P<work>\d+)ms\s+live=(?P<live>\d+)$"
|
||||
)
|
||||
_BODY_FETCH_RE = re.compile(
|
||||
r"^body-fetch\s+select=(?P<select>\d+)ms\s+body=(?P<body>\d+)ms\s+flag=(?P<flag>\d+)ms\s+"
|
||||
r"rfc822=(?P<bytes>\d+)B\s+chars=(?P<chars>\d+)\s+att=(?P<att>\d+)$"
|
||||
)
|
||||
_OPEN_MSG_RE = re.compile(
|
||||
r"^openMessage\s+(?P<ref>\S+)\s+folder=(?P<folder>\S+)\s+"
|
||||
r"fetchedBody=(?P<fetched>true|false)\s+took=(?P<took>\d+)ms$"
|
||||
)
|
||||
_READER_READY_RE = re.compile(
|
||||
r"^reader ready took=(?P<took>\d+)ms\s+html=(?P<html>true|false)\s+inline=(?P<inline>\d+)$"
|
||||
)
|
||||
_BACKFILL_PROGRESS_RE = re.compile(
|
||||
r"^backfill\s+(?P<ref>\S+)\s+folder=(?P<folder>\S+)\s+pages=(?P<pages>\d+)\s+"
|
||||
r"complete=(?P<complete>true|false)$"
|
||||
)
|
||||
_BACKFILL_SLICE_START_RE = re.compile(r"^backfill slice:\s+maxBatches=(?P<n>\d+)$")
|
||||
_BACKFILL_SLICE_DONE_RE = re.compile(r"^backfill slice done:\s+moreWork=(?P<more>true|false)$")
|
||||
|
||||
|
||||
def parse_imap_perf(message: str) -> Optional[Union[ImapPerfOp, BodyFetch]]:
|
||||
"""Parse an ``ImapPerf`` message (either the detailed body-fetch or the generic op)."""
|
||||
m = _BODY_FETCH_RE.match(message)
|
||||
if m:
|
||||
return BodyFetch(
|
||||
select_ms=int(m.group("select")),
|
||||
body_ms=int(m.group("body")),
|
||||
flag_ms=int(m.group("flag")),
|
||||
rfc822_bytes=int(m.group("bytes")),
|
||||
chars=int(m.group("chars")),
|
||||
att=int(m.group("att")),
|
||||
)
|
||||
m = _IMAP_OP_RE.match(message)
|
||||
if m:
|
||||
return ImapPerfOp(
|
||||
op=m.group("op"),
|
||||
connect_ms=int(m.group("connect")),
|
||||
work_ms=int(m.group("work")),
|
||||
live=int(m.group("live")),
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def parse_open_message(message: str) -> Optional[OpenMessage]:
|
||||
"""Parse a ``MailReader`` ``openMessage`` message."""
|
||||
m = _OPEN_MSG_RE.match(message)
|
||||
if not m:
|
||||
return None
|
||||
return OpenMessage(
|
||||
account_ref=m.group("ref"),
|
||||
folder=m.group("folder"),
|
||||
fetched_body=_to_bool(m.group("fetched")),
|
||||
took_ms=int(m.group("took")),
|
||||
)
|
||||
|
||||
|
||||
def parse_reader_ready(message: str) -> Optional[ReaderReady]:
|
||||
"""Parse a ``Reader`` ``reader ready`` message."""
|
||||
m = _READER_READY_RE.match(message)
|
||||
if not m:
|
||||
return None
|
||||
return ReaderReady(
|
||||
took_ms=int(m.group("took")),
|
||||
html=_to_bool(m.group("html")),
|
||||
inline=int(m.group("inline")),
|
||||
)
|
||||
|
||||
|
||||
def parse_backfill(
|
||||
message: str,
|
||||
) -> Optional[Union[BackfillProgress, BackfillSliceStart, BackfillSliceDone]]:
|
||||
"""Parse any of the three ``MailBackfiller`` messages."""
|
||||
m = _BACKFILL_PROGRESS_RE.match(message)
|
||||
if m:
|
||||
return BackfillProgress(
|
||||
account_ref=m.group("ref"),
|
||||
folder=m.group("folder"),
|
||||
pages=int(m.group("pages")),
|
||||
complete=_to_bool(m.group("complete")),
|
||||
)
|
||||
m = _BACKFILL_SLICE_START_RE.match(message)
|
||||
if m:
|
||||
return BackfillSliceStart(max_batches=int(m.group("n")))
|
||||
m = _BACKFILL_SLICE_DONE_RE.match(message)
|
||||
if m:
|
||||
return BackfillSliceDone(more_work=_to_bool(m.group("more")))
|
||||
return None
|
||||
|
||||
|
||||
# Dispatch table keyed by the trimmed logcat tag.
|
||||
_TAG_PARSERS = {
|
||||
"ImapPerf": parse_imap_perf,
|
||||
"MailReader": parse_open_message,
|
||||
"Reader": parse_reader_ready,
|
||||
"MailBackfiller": parse_backfill,
|
||||
}
|
||||
|
||||
|
||||
def parse_breadcrumb(line: str) -> Optional[Event]:
|
||||
"""Parse a full threadtime logcat line into a typed :data:`Event` (or ``None``).
|
||||
|
||||
The returned event carries its source :class:`LogLine` on ``.line`` so callers can
|
||||
order and time-box events. Lines that are not perf breadcrumbs return ``None``.
|
||||
"""
|
||||
log = parse_logcat_line(line)
|
||||
if log is None:
|
||||
return None
|
||||
parser = _TAG_PARSERS.get(log.tag)
|
||||
if parser is None:
|
||||
return None
|
||||
event = parser(log.message)
|
||||
if event is None:
|
||||
return None
|
||||
event.line = log
|
||||
return event
|
||||
|
||||
|
||||
def iter_events(lines: Iterable[str]) -> Iterator[Event]:
|
||||
"""Yield every parseable breadcrumb :data:`Event` from an iterable of logcat lines."""
|
||||
for line in lines:
|
||||
event = parse_breadcrumb(line)
|
||||
if event is not None:
|
||||
yield event
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Correlation -- group the breadcrumbs of one message-open together
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class OpenSample:
|
||||
"""One reader-open, correlating the breadcrumbs the app emits for a single open.
|
||||
|
||||
Mirrors a row of the manual ``timing-tables.md``: the ``openMessage`` marker plus the
|
||||
``body-fetch`` detail/op that preceded it and the ``reader ready`` that followed.
|
||||
"""
|
||||
|
||||
open_message: OpenMessage
|
||||
body_fetch: Optional[BodyFetch] = None
|
||||
body_fetch_op: Optional[ImapPerfOp] = None
|
||||
reader_ready: Optional[ReaderReady] = None
|
||||
|
||||
@property
|
||||
def account_ref(self) -> str:
|
||||
return self.open_message.account_ref
|
||||
|
||||
@property
|
||||
def cached(self) -> bool:
|
||||
"""A cached open did not fetch the body over the network."""
|
||||
return not self.open_message.fetched_body
|
||||
|
||||
@property
|
||||
def took_ms(self) -> int:
|
||||
return self.open_message.took_ms
|
||||
|
||||
@property
|
||||
def rfc822_bytes(self) -> Optional[int]:
|
||||
return self.body_fetch.rfc822_bytes if self.body_fetch else None
|
||||
|
||||
@property
|
||||
def body_kb_per_s(self) -> Optional[float]:
|
||||
return self.body_fetch.body_kb_per_s if self.body_fetch else None
|
||||
|
||||
|
||||
def correlate_opens(events: Iterable[Event]) -> List[OpenSample]:
|
||||
"""Group a flat, time-ordered event stream into :class:`OpenSample` records.
|
||||
|
||||
Strategy (matches how the breadcrumbs interleave in a real capture): keep the most
|
||||
recent ``body-fetch`` detail and generic ``body-fetch`` op seen; when an
|
||||
``openMessage`` marker arrives, attach those (consuming them so they are not reused);
|
||||
the first ``reader ready`` after the marker attaches to it.
|
||||
"""
|
||||
samples: List[OpenSample] = []
|
||||
pending_detail: Optional[BodyFetch] = None
|
||||
pending_op: Optional[ImapPerfOp] = None
|
||||
open_awaiting_reader: Optional[OpenSample] = None
|
||||
|
||||
for event in events:
|
||||
if isinstance(event, BodyFetch):
|
||||
pending_detail = event
|
||||
elif isinstance(event, ImapPerfOp) and event.op == "body-fetch":
|
||||
pending_op = event
|
||||
elif isinstance(event, OpenMessage):
|
||||
sample = OpenSample(open_message=event)
|
||||
if event.fetched_body:
|
||||
sample.body_fetch = pending_detail
|
||||
sample.body_fetch_op = pending_op
|
||||
pending_detail = None
|
||||
pending_op = None
|
||||
samples.append(sample)
|
||||
open_awaiting_reader = sample
|
||||
elif isinstance(event, ReaderReady):
|
||||
if open_awaiting_reader is not None and open_awaiting_reader.reader_ready is None:
|
||||
open_awaiting_reader.reader_ready = event
|
||||
open_awaiting_reader = None
|
||||
|
||||
return samples
|
||||
@@ -0,0 +1,258 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""perf_harness.py -- LibreMail on-device performance-test harness (CLI entry point).
|
||||
|
||||
Replicates the manual on-device perf methodology (2026-07-05, Pixel 10 Pro XL) as a
|
||||
repeatable, cross-platform, standard-library-only tool. Run one scenario at a time:
|
||||
|
||||
python scripts/device-testing/perf_harness.py cold-open [opts]
|
||||
python scripts/device-testing/perf_harness.py message-open [opts]
|
||||
python scripts/device-testing/perf_harness.py back-nav [opts]
|
||||
python scripts/device-testing/perf_harness.py prefetch-ab [opts]
|
||||
python scripts/device-testing/perf_harness.py cross-provider [opts]
|
||||
|
||||
Common options: ``--serial`` (auto-detected if exactly one device), ``--count/-n``,
|
||||
``--out``, ``--package``, ``--component``, ``--adb``, and ``--dry-run`` (print the exact
|
||||
command plan without touching device state -- use this to review a run before it happens).
|
||||
|
||||
Device safety is enforced by :mod:`adb` (allow-list of adb subcommands + deny-list on shell
|
||||
commands; the only sanctioned mutation is clearing LibreMail's own ``cache/``). See the
|
||||
README for the full guarantees and the required device state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as _dt
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from typing import List, Optional
|
||||
|
||||
# Flat-layout imports: this file's directory is on sys.path[0] when run as a script, and the
|
||||
# tests inject it explicitly. (The dir name contains a hyphen, so it is not an importable
|
||||
# package -- hence flat modules rather than `python -m`.)
|
||||
import breadcrumbs
|
||||
import report
|
||||
import scenarios
|
||||
from adb import Adb, DEFAULT_COMPONENT, DEFAULT_PACKAGE
|
||||
|
||||
SCENARIOS = ("cold-open", "message-open", "back-nav", "prefetch-ab", "cross-provider")
|
||||
_DEFAULT_COUNTS = {
|
||||
"cold-open": 5,
|
||||
"message-open": 6,
|
||||
"back-nav": 6,
|
||||
"prefetch-ab": 3,
|
||||
"cross-provider": 8,
|
||||
}
|
||||
|
||||
|
||||
def _make_logger(log_path: Optional[str]):
|
||||
handle = open(log_path, "a", encoding="utf-8") if log_path else None
|
||||
|
||||
def log(msg: str) -> None:
|
||||
stamp = _dt.datetime.now().strftime("%H:%M:%S")
|
||||
line = f"[{stamp}] {msg}"
|
||||
print(line, flush=True)
|
||||
if handle:
|
||||
handle.write(line + "\n")
|
||||
handle.flush()
|
||||
|
||||
return log, handle
|
||||
|
||||
|
||||
def resolve_serial(adb_path: str, requested: Optional[str]) -> Optional[str]:
|
||||
"""Return the serial to use, auto-detecting when exactly one device is attached."""
|
||||
probe = Adb(serial=None, adb_path=adb_path)
|
||||
serials = probe.devices()
|
||||
if requested:
|
||||
if serials and requested not in serials:
|
||||
print(
|
||||
f"warning: requested serial {requested!r} not in attached devices {serials}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return requested
|
||||
if len(serials) == 1:
|
||||
return serials[0]
|
||||
if not serials:
|
||||
raise SystemExit("no devices attached; connect one or pass --serial")
|
||||
raise SystemExit(f"multiple devices attached {serials}; pass --serial to choose one")
|
||||
|
||||
|
||||
def _run_dir(out_root: str, scenario: str) -> str:
|
||||
stamp = _dt.datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||
path = os.path.join(out_root, f"{stamp}-{scenario}")
|
||||
os.makedirs(path, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def _write_filtered_extract(raw_path: str, extract_path: str) -> int:
|
||||
"""Write the ImapPerf/MailReader/Reader/MailBackfiller subset of the raw log; return count."""
|
||||
if not os.path.exists(raw_path):
|
||||
return 0
|
||||
count = 0
|
||||
with open(raw_path, "r", encoding="utf-8", errors="replace") as src, open(
|
||||
extract_path, "w", encoding="utf-8", newline="\n"
|
||||
) as dst:
|
||||
for line in src:
|
||||
if breadcrumbs.is_perf_line(line):
|
||||
dst.write(line if line.endswith("\n") else line + "\n")
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def _render_sections(scenario: str, results, adb: Adb, count: int) -> List[str]:
|
||||
if scenario == "cold-open":
|
||||
return [report.render_cold_open(results)]
|
||||
if scenario == "message-open":
|
||||
return [report.render_message_open("Message open (uncached)", results)]
|
||||
if scenario == "back-nav":
|
||||
return [report.render_back_nav(results)]
|
||||
if scenario == "prefetch-ab":
|
||||
return [
|
||||
report.render_message_open(
|
||||
"Condition A - Fetch all on Wi-Fi (prefetch ON)", results["conditionA"]
|
||||
),
|
||||
report.render_message_open(
|
||||
"Condition B - Always on-demand (prefetch OFF)", results["conditionB"]
|
||||
),
|
||||
_ab_comparison(results),
|
||||
]
|
||||
if scenario == "cross-provider":
|
||||
sections = []
|
||||
for ref, rows in sorted(results.items()):
|
||||
sections.append(report.render_message_open(f"Provider {ref}", rows))
|
||||
if not sections:
|
||||
sections.append("## Cross-provider\n\nNo opens captured.\n")
|
||||
return sections
|
||||
return []
|
||||
|
||||
|
||||
def _ab_comparison(results: dict) -> str:
|
||||
def median_took(rows):
|
||||
vals = [r.took_ms for r in rows if not r.skipped and not r.cached and r.took_ms]
|
||||
return report.aggregate(vals).median
|
||||
|
||||
a = median_took(results["conditionA"])
|
||||
b = median_took(results["conditionB"])
|
||||
return (
|
||||
"## Prefetch A/B comparison\n\n"
|
||||
f"- Condition A (prefetch ON) median openMessage: {report._ms(a)} ms\n"
|
||||
f"- Condition B (prefetch OFF) median openMessage: {report._ms(b)} ms\n"
|
||||
)
|
||||
|
||||
|
||||
def _run_scenario(scenario: str, adb: Adb, tailer, args, log):
|
||||
package, component, count = args.package, args.component, args.count
|
||||
if scenario == "cold-open":
|
||||
return scenarios.cold_open(adb, component, count, log)
|
||||
if scenario == "message-open":
|
||||
return scenarios.message_open(adb, package, tailer, count, log)
|
||||
if scenario == "back-nav":
|
||||
return scenarios.back_nav(adb, package, count, log)
|
||||
if scenario == "prefetch-ab":
|
||||
return scenarios.prefetch_ab(adb, package, component, tailer, count, log)
|
||||
if scenario == "cross-provider":
|
||||
return scenarios.cross_provider(adb, package, tailer, count, log)
|
||||
raise SystemExit(f"unknown scenario {scenario!r}")
|
||||
|
||||
|
||||
def build_arg_parser() -> argparse.ArgumentParser:
|
||||
p = argparse.ArgumentParser(
|
||||
prog="perf_harness.py",
|
||||
description="LibreMail on-device performance-test harness (stdlib-only).",
|
||||
)
|
||||
p.add_argument("scenario", choices=SCENARIOS, help="which scenario to run")
|
||||
p.add_argument("--serial", help="device serial (auto-detected if exactly one attached)")
|
||||
p.add_argument("--package", default=DEFAULT_PACKAGE, help="target app package")
|
||||
p.add_argument("--component", default=DEFAULT_COMPONENT, help="launcher component")
|
||||
p.add_argument("--adb", default="adb", dest="adb_path", help="path to the adb executable")
|
||||
p.add_argument(
|
||||
"-n", "--count", type=int, default=None,
|
||||
help="samples/runs (per condition for prefetch-ab); scenario-specific default",
|
||||
)
|
||||
p.add_argument(
|
||||
"--out", default=os.path.join(os.getcwd(), "device-perf-runs"),
|
||||
help="output root; a timestamped subdir is created per run",
|
||||
)
|
||||
p.add_argument(
|
||||
"--dry-run", action="store_true",
|
||||
help="print the command plan without changing device state",
|
||||
)
|
||||
return p
|
||||
|
||||
|
||||
def main(argv: Optional[List[str]] = None) -> int:
|
||||
args = build_arg_parser().parse_args(argv)
|
||||
if args.count is None:
|
||||
args.count = _DEFAULT_COUNTS[args.scenario]
|
||||
|
||||
serial = None if args.dry_run else resolve_serial(args.adb_path, args.serial)
|
||||
if args.dry_run and args.serial:
|
||||
serial = args.serial
|
||||
|
||||
run_dir = _run_dir(args.out, args.scenario)
|
||||
driver_log_path = os.path.join(run_dir, "driver.log")
|
||||
log, log_handle = _make_logger(driver_log_path)
|
||||
raw_path = os.path.join(run_dir, "session-raw.log")
|
||||
|
||||
log(f"scenario={args.scenario} serial={serial} count={args.count} dry_run={args.dry_run}")
|
||||
log(f"run dir: {run_dir}")
|
||||
|
||||
adb = Adb(
|
||||
serial=serial,
|
||||
package=args.package,
|
||||
adb_path=args.adb_path,
|
||||
dry_run=args.dry_run,
|
||||
logger=log,
|
||||
)
|
||||
|
||||
logcat_proc = None
|
||||
raw_handle = None
|
||||
tailer: scenarios.LogTailer = scenarios._NullTailer()
|
||||
try:
|
||||
if not args.dry_run:
|
||||
adb.clear_logcat()
|
||||
# Binary handle: the logcat child writes raw bytes to this fd.
|
||||
raw_handle = open(raw_path, "wb")
|
||||
logcat_proc = adb.start_logcat(raw_handle)
|
||||
tailer = scenarios.LogTailer(raw_path)
|
||||
scenarios.ensure_awake(adb, log)
|
||||
time.sleep(1.0)
|
||||
|
||||
results = _run_scenario(args.scenario, adb, tailer, args, log)
|
||||
sections = _render_sections(args.scenario, results, adb, args.count)
|
||||
finally:
|
||||
Adb.stop_logcat(logcat_proc)
|
||||
if raw_handle:
|
||||
raw_handle.close()
|
||||
if not args.dry_run:
|
||||
adb.stay_on(False) # restore default screen-timeout behaviour
|
||||
if log_handle:
|
||||
log_handle.flush()
|
||||
|
||||
metadata = {
|
||||
"scenario": args.scenario,
|
||||
"timestamp": _dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"serial": serial or "(dry-run)",
|
||||
"package": args.package,
|
||||
"count": args.count,
|
||||
}
|
||||
document = report.build_document(metadata, sections)
|
||||
tables_path = os.path.join(run_dir, "timing-tables.md")
|
||||
with open(tables_path, "w", encoding="utf-8", newline="\n") as fh:
|
||||
fh.write(document)
|
||||
|
||||
extract_count = 0
|
||||
if not args.dry_run:
|
||||
extract_count = _write_filtered_extract(
|
||||
raw_path, os.path.join(run_dir, "perf-extract.log")
|
||||
)
|
||||
log(f"wrote {tables_path} (filtered {extract_count} breadcrumb lines)")
|
||||
if log_handle:
|
||||
log_handle.close()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,253 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""report.py -- aggregate scenario samples and render ``timing-tables.md``.
|
||||
|
||||
Pure and testable: given the samples a scenario collected, it produces the same
|
||||
per-scenario markdown tables + aggregates as the hand-written ``timing-tables.md`` from the
|
||||
manual run. No device or I/O dependency beyond writing the final file.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import statistics
|
||||
from dataclasses import dataclass
|
||||
from typing import List, Optional, Sequence
|
||||
|
||||
from breadcrumbs import OpenSample
|
||||
|
||||
# The floor below which adb uiautomator-dump timings cannot resolve an in-app transition
|
||||
# (see perf_summary.md: nav/back were dominated by the ~2.5-3 s dump latency).
|
||||
UIAUTOMATOR_LATENCY_FLOOR_MS = 3000
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Stats + markdown helpers
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class Aggregate:
|
||||
n: int
|
||||
mean: Optional[float]
|
||||
median: Optional[float]
|
||||
minimum: Optional[int]
|
||||
maximum: Optional[int]
|
||||
|
||||
|
||||
def aggregate(values: Sequence[float]) -> Aggregate:
|
||||
"""Summarise a list of numbers; safe on an empty list."""
|
||||
vals = [v for v in values if v is not None]
|
||||
if not vals:
|
||||
return Aggregate(0, None, None, None, None)
|
||||
return Aggregate(
|
||||
n=len(vals),
|
||||
mean=statistics.fmean(vals),
|
||||
median=statistics.median(vals),
|
||||
minimum=min(vals),
|
||||
maximum=max(vals),
|
||||
)
|
||||
|
||||
|
||||
def md_table(headers: Sequence[str], rows: Sequence[Sequence[object]]) -> str:
|
||||
"""Render a GitHub-flavoured markdown table."""
|
||||
head = "| " + " | ".join(str(h) for h in headers) + " |"
|
||||
sep = "| " + " | ".join("---" for _ in headers) + " |"
|
||||
body = [
|
||||
"| " + " | ".join("" if c is None else str(c) for c in row) + " |" for row in rows
|
||||
]
|
||||
return "\n".join([head, sep, *body])
|
||||
|
||||
|
||||
def _ms(value: Optional[float]) -> str:
|
||||
return "" if value is None else f"{round(value)}"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Cold open
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class ColdOpenSample:
|
||||
run: int
|
||||
total_time_ms: Optional[int]
|
||||
wait_time_ms: Optional[int]
|
||||
note: str = ""
|
||||
|
||||
|
||||
def render_cold_open(samples: Sequence[ColdOpenSample]) -> str:
|
||||
rows: List[Sequence[object]] = []
|
||||
for s in samples:
|
||||
rows.append([s.run, _ms(s.total_time_ms), _ms(s.wait_time_ms), s.note])
|
||||
total_agg = aggregate([s.total_time_ms for s in samples if s.total_time_ms is not None])
|
||||
wait_agg = aggregate([s.wait_time_ms for s in samples if s.wait_time_ms is not None])
|
||||
rows.append(
|
||||
[
|
||||
"**mean**",
|
||||
f"**{_ms(total_agg.mean)}**",
|
||||
f"**{_ms(wait_agg.mean)}**",
|
||||
f"median {_ms(total_agg.median)} / {_ms(wait_agg.median)}",
|
||||
]
|
||||
)
|
||||
table = md_table(["Run", "TotalTime (ms)", "WaitTime (ms)", "note"], rows)
|
||||
return (
|
||||
"## Cold open (am start -W, cache cleared each run)\n\n"
|
||||
+ table
|
||||
+ "\n\nColdest run is the first post-clear launch (class-load/JIT); steady state is lower.\n"
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Message open (reader body-load)
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class ReaderOpenRow:
|
||||
"""A flattened, render-ready reader-open row (mirrors a timing-tables.md line)."""
|
||||
|
||||
index: int
|
||||
label: str
|
||||
account_ref: str = ""
|
||||
cached: bool = False
|
||||
took_ms: Optional[int] = None
|
||||
reader_ready_ms: Optional[int] = None
|
||||
connect_ms: Optional[int] = None
|
||||
work_ms: Optional[int] = None
|
||||
select_ms: Optional[int] = None
|
||||
body_ms: Optional[int] = None
|
||||
flag_ms: Optional[int] = None
|
||||
live: Optional[int] = None
|
||||
rfc822_bytes: Optional[int] = None
|
||||
body_kb_per_s: Optional[float] = None
|
||||
skipped: bool = False
|
||||
reason: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_open_sample(
|
||||
cls, sample: OpenSample, index: int, label: str
|
||||
) -> "ReaderOpenRow":
|
||||
bf = sample.body_fetch
|
||||
op = sample.body_fetch_op
|
||||
return cls(
|
||||
index=index,
|
||||
label=label,
|
||||
account_ref=sample.account_ref,
|
||||
cached=sample.cached,
|
||||
took_ms=sample.took_ms,
|
||||
reader_ready_ms=sample.reader_ready.took_ms if sample.reader_ready else None,
|
||||
connect_ms=op.connect_ms if op else None,
|
||||
work_ms=op.work_ms if op else None,
|
||||
select_ms=bf.select_ms if bf else None,
|
||||
body_ms=bf.body_ms if bf else None,
|
||||
flag_ms=bf.flag_ms if bf else None,
|
||||
live=op.live if op else None,
|
||||
rfc822_bytes=bf.rfc822_bytes if bf else None,
|
||||
body_kb_per_s=sample.body_kb_per_s,
|
||||
)
|
||||
|
||||
|
||||
_OPEN_HEADERS = [
|
||||
"#",
|
||||
"message",
|
||||
"cached",
|
||||
"rfc822 B",
|
||||
"openMessage took",
|
||||
"reader ready",
|
||||
"connect",
|
||||
"work",
|
||||
"select",
|
||||
"body-dl",
|
||||
"flag",
|
||||
"live",
|
||||
"body KB/s",
|
||||
]
|
||||
|
||||
|
||||
def _open_row_cells(row: ReaderOpenRow) -> Sequence[object]:
|
||||
if row.skipped:
|
||||
return [
|
||||
row.index,
|
||||
row.label,
|
||||
"-",
|
||||
f"SKIPPED: {row.reason}",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
"",
|
||||
]
|
||||
kbps = "" if row.body_kb_per_s is None else f"{row.body_kb_per_s:.1f}"
|
||||
return [
|
||||
row.index,
|
||||
row.label,
|
||||
"yes" if row.cached else "no",
|
||||
row.rfc822_bytes if row.rfc822_bytes is not None else "",
|
||||
_fmt_ms(row.took_ms),
|
||||
_fmt_ms(row.reader_ready_ms),
|
||||
_ms(row.connect_ms),
|
||||
_ms(row.work_ms),
|
||||
_ms(row.select_ms),
|
||||
_ms(row.body_ms),
|
||||
_ms(row.flag_ms),
|
||||
_ms(row.live),
|
||||
kbps,
|
||||
]
|
||||
|
||||
|
||||
def _fmt_ms(value: Optional[int]) -> str:
|
||||
return "" if value is None else f"{value} ms"
|
||||
|
||||
|
||||
def render_message_open(title: str, rows: Sequence[ReaderOpenRow]) -> str:
|
||||
table = md_table(_OPEN_HEADERS, [_open_row_cells(r) for r in rows])
|
||||
uncached = [r for r in rows if not r.skipped and not r.cached and r.took_ms is not None]
|
||||
agg = aggregate([r.took_ms for r in uncached])
|
||||
lines = [f"## {title}", "", table, ""]
|
||||
if agg.n:
|
||||
lines.append(
|
||||
f"Uncached opens: n={agg.n}, openMessage median "
|
||||
f"{_ms(agg.median)} ms, mean {_ms(agg.mean)} ms, "
|
||||
f"range {agg.minimum}-{agg.maximum} ms."
|
||||
)
|
||||
skipped = [r for r in rows if r.skipped]
|
||||
if skipped:
|
||||
lines.append(f"Skipped samples: {len(skipped)} (see rows above).")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Back navigation
|
||||
# --------------------------------------------------------------------------- #
|
||||
@dataclass
|
||||
class BackNavSample:
|
||||
index: int
|
||||
back_ms: Optional[int]
|
||||
note: str = ""
|
||||
|
||||
|
||||
def render_back_nav(samples: Sequence[BackNavSample]) -> str:
|
||||
rows = [[s.index, _fmt_ms(s.back_ms), s.note] for s in samples]
|
||||
agg = aggregate([s.back_ms for s in samples if s.back_ms is not None])
|
||||
table = md_table(["#", "back (reader->mailbox)", "note"], rows)
|
||||
caveat = (
|
||||
f"\n\n**Caveat:** these are dominated by the ~{UIAUTOMATOR_LATENCY_FLOOR_MS} ms "
|
||||
"uiautomator-dump latency floor; true in-app back is sub-second and not precisely "
|
||||
"measurable via adb UI polling under load (see perf_summary.md)."
|
||||
)
|
||||
summary = "" if not agg.n else f"\n\nBack: n={agg.n}, mean {_ms(agg.mean)} ms, median {_ms(agg.median)} ms."
|
||||
return "## Reader -> mailbox (back)\n\n" + table + summary + caveat + "\n"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Document assembly
|
||||
# --------------------------------------------------------------------------- #
|
||||
def render_header(metadata: dict) -> str:
|
||||
lines = ["# LibreMail device perf run", ""]
|
||||
for key, value in metadata.items():
|
||||
lines.append(f"- **{key}:** {value}")
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def build_document(metadata: dict, sections: Sequence[str]) -> str:
|
||||
parts = [render_header(metadata), *sections]
|
||||
return "\n".join(parts).rstrip() + "\n"
|
||||
@@ -0,0 +1,445 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""scenarios.py -- the on-device performance scenarios.
|
||||
|
||||
Each scenario drives LibreMail through the guarded :class:`adb.Adb` wrapper, times the
|
||||
behaviour primarily from the on-device breadcrumbs (with uiautomator as a fallback ready
|
||||
signal), and returns render-ready samples for :mod:`report`.
|
||||
|
||||
Timing philosophy (from the manual run): message-open and reader-ready times come from the
|
||||
``MailReader`` / ``Reader`` / ``ImapPerf`` breadcrumbs -- adb UI polling only tells us *when*
|
||||
the content is ready so we can move on, and back-nav is explicitly dump-latency-bound.
|
||||
|
||||
Safety: every device call goes through :class:`adb.Adb`, so its allow/deny guardrails apply.
|
||||
Every uiautomator/input step is guarded against the keyguard and against a foreign app being
|
||||
in the foreground; a sample taken against either is skipped, not measured.
|
||||
|
||||
``--dry-run`` executes one representative pass per scenario -- issuing the canonical command
|
||||
sequence (so the plan is auditable) without looping on a live UI.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from typing import Callable, List, Optional
|
||||
|
||||
import breadcrumbs
|
||||
import uidump
|
||||
from adb import Adb, parse_am_start
|
||||
from report import BackNavSample, ColdOpenSample, ReaderOpenRow
|
||||
|
||||
# Message-open can stall 30-75 s behind the spinner (Gmail throttle), so allow generous
|
||||
# headroom before giving up on a single open.
|
||||
OPEN_TIMEOUT_S = 150.0
|
||||
POLL_INTERVAL_S = 2.0
|
||||
SETTLE_S = 1.5
|
||||
|
||||
# Fetch-policy option labels (from res/values/strings.xml) used to drive the A/B toggle.
|
||||
FETCH_WIFI_LABEL = "Fetch all on Wi-Fi" # prefetch ON (Condition A / WIFI_ONLY)
|
||||
FETCH_ON_DEMAND_LABEL = "Always on-demand" # prefetch OFF (Condition B / ON_DEMAND)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Live-logcat tailing (per-sample breadcrumb extraction)
|
||||
# --------------------------------------------------------------------------- #
|
||||
class LogTailer:
|
||||
"""Reads newly-appended text from the growing session logcat file since the last mark."""
|
||||
|
||||
def __init__(self, path: str) -> None:
|
||||
self.path = path
|
||||
self._offset = 0
|
||||
|
||||
def mark(self) -> None:
|
||||
"""Set the read cursor to the current end of file."""
|
||||
self._offset = os.path.getsize(self.path) if os.path.exists(self.path) else 0
|
||||
|
||||
def read_new(self) -> str:
|
||||
if not os.path.exists(self.path):
|
||||
return ""
|
||||
with open(self.path, "r", encoding="utf-8", errors="replace") as fh:
|
||||
fh.seek(self._offset)
|
||||
data = fh.read()
|
||||
self._offset = fh.tell()
|
||||
return data
|
||||
|
||||
|
||||
class _NullTailer(LogTailer):
|
||||
"""A tailer that yields nothing -- used in dry-run so scenarios need no live log."""
|
||||
|
||||
def __init__(self) -> None: # noqa: D401 - see base
|
||||
super().__init__(path="")
|
||||
|
||||
def mark(self) -> None:
|
||||
pass
|
||||
|
||||
def read_new(self) -> str:
|
||||
return ""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# UI helpers (all guarded)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def dump_ui(adb: Adb) -> Optional[uidump.UiNode]:
|
||||
"""Dump + parse the current UI; ``None`` in dry-run or on an unparseable dump."""
|
||||
if adb.dry_run:
|
||||
return None
|
||||
xml = adb.uiautomator_dump()
|
||||
if not xml or "<hierarchy" not in xml:
|
||||
return None
|
||||
try:
|
||||
return uidump.parse_dump(xml)
|
||||
except Exception: # pragma: no cover - malformed dump under load
|
||||
return None
|
||||
|
||||
|
||||
def ensure_awake(adb: Adb, log: Callable[[str], None]) -> None:
|
||||
"""Keep the screen on and awake for the run (restored by the caller afterwards)."""
|
||||
adb.stay_on(True)
|
||||
adb.wake()
|
||||
|
||||
|
||||
def guard_ready(adb: Adb, package: str, log: Callable[[str], None]) -> Optional[uidump.UiNode]:
|
||||
"""Return the current UI iff it is our app and not the keyguard; else try to recover.
|
||||
|
||||
Returns ``None`` if, after a wake attempt, the sample is still against the lockscreen or
|
||||
a foreign app -- the caller must skip that sample rather than measure garbage.
|
||||
"""
|
||||
root = dump_ui(adb)
|
||||
if root is None:
|
||||
return None
|
||||
if uidump.is_lockscreen(root):
|
||||
log("keyguard detected; waking and re-checking")
|
||||
adb.wake()
|
||||
time.sleep(SETTLE_S)
|
||||
root = dump_ui(adb)
|
||||
if root is None or uidump.is_lockscreen(root):
|
||||
log("still on keyguard after wake; skipping sample")
|
||||
return None
|
||||
if not uidump.is_app_foreground(root, package):
|
||||
log(f"foreground is {uidump.foreground_package(root)!r}, not {package!r}; skipping")
|
||||
return None
|
||||
return root
|
||||
|
||||
|
||||
def goto_mailbox(adb: Adb, package: str, log: Callable[[str], None]) -> Optional[uidump.UiNode]:
|
||||
"""Ensure the mailbox list is showing (press Back out of the reader if needed)."""
|
||||
root = guard_ready(adb, package, log)
|
||||
if root is None:
|
||||
return None
|
||||
if uidump.is_reader(root, package):
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
time.sleep(SETTLE_S)
|
||||
root = guard_ready(adb, package, log)
|
||||
return root
|
||||
|
||||
|
||||
def wait_for_reader_ready(adb: Adb, package: str, log: Callable[[str], None]) -> bool:
|
||||
"""Poll until the reader has loaded its body (no spinner) or timeout. True if loaded."""
|
||||
deadline = time.monotonic() + OPEN_TIMEOUT_S
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(POLL_INTERVAL_S)
|
||||
root = dump_ui(adb)
|
||||
if root is None:
|
||||
continue
|
||||
if uidump.is_lockscreen(root):
|
||||
log("keyguard appeared during open; sample is invalid")
|
||||
return False
|
||||
if uidump.is_reader(root, package) and not uidump.has_progress_bar(root):
|
||||
return True
|
||||
log("open timed out")
|
||||
return False
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 1: cold open
|
||||
# --------------------------------------------------------------------------- #
|
||||
def cold_open(
|
||||
adb: Adb,
|
||||
component: str,
|
||||
runs: int,
|
||||
log: Callable[[str], None],
|
||||
settle_s: float = 2.0,
|
||||
) -> List[ColdOpenSample]:
|
||||
"""Force-stop + clear cache + ``am start -W`` x ``runs``; parse TotalTime/WaitTime."""
|
||||
samples: List[ColdOpenSample] = []
|
||||
for run in range(1, runs + 1):
|
||||
adb.force_stop()
|
||||
adb.clear_cache()
|
||||
adb.settle(settle_s)
|
||||
out = adb.start_activity(component=component, wait=True).stdout
|
||||
fields = parse_am_start(out)
|
||||
note = "" if fields else ("dry-run" if adb.dry_run else "no am-start timing parsed")
|
||||
samples.append(
|
||||
ColdOpenSample(
|
||||
run=run,
|
||||
total_time_ms=fields.get("TotalTime"),
|
||||
wait_time_ms=fields.get("WaitTime"),
|
||||
note=note,
|
||||
)
|
||||
)
|
||||
log(f"cold-open run {run}: {fields or note}")
|
||||
adb.settle(settle_s)
|
||||
return samples
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 2: message open (uncached)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _row_from_events(index: int, label: str, events: List[breadcrumbs.Event]) -> ReaderOpenRow:
|
||||
"""Build a reader-open row from the breadcrumbs captured during one open."""
|
||||
opens = breadcrumbs.correlate_opens(events)
|
||||
if not opens:
|
||||
return ReaderOpenRow(index=index, label=label, skipped=True, reason="no breadcrumb")
|
||||
sample = opens[-1]
|
||||
return ReaderOpenRow.from_open_sample(sample, index=index, label=label)
|
||||
|
||||
|
||||
def open_one_message(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
tailer: LogTailer,
|
||||
index: int,
|
||||
already_opened: set,
|
||||
log: Callable[[str], None],
|
||||
) -> Optional[ReaderOpenRow]:
|
||||
"""Open the next not-yet-opened, uncached message; return its timed row (or ``None``)."""
|
||||
root = goto_mailbox(adb, package, log)
|
||||
if root is None:
|
||||
return ReaderOpenRow(index=index, label="?", skipped=True, reason="not on mailbox")
|
||||
|
||||
rows = uidump.find_message_rows(root, package)
|
||||
candidates = [r for r in rows if not r.cached and r.label not in already_opened]
|
||||
if not candidates:
|
||||
# Scroll to reveal more of the list, then re-scan once.
|
||||
adb.input_swipe(672, 2000, 672, 900, 400)
|
||||
time.sleep(SETTLE_S)
|
||||
root = guard_ready(adb, package, log)
|
||||
if root is None:
|
||||
return None
|
||||
rows = uidump.find_message_rows(root, package)
|
||||
candidates = [r for r in rows if not r.cached and r.label not in already_opened]
|
||||
if not candidates:
|
||||
log("no more uncached messages visible")
|
||||
return None
|
||||
|
||||
target = candidates[0]
|
||||
already_opened.add(target.label)
|
||||
log(f"opening row#{target.index} {target.label!r} at {target.center}")
|
||||
|
||||
tailer.mark()
|
||||
adb.input_tap(*target.center)
|
||||
loaded = wait_for_reader_ready(adb, package, log)
|
||||
events = list(breadcrumbs.iter_events(tailer.read_new().splitlines()))
|
||||
row = _row_from_events(index, target.label, events)
|
||||
if not loaded and not row.skipped:
|
||||
row.reason = "ui ready-signal timed out (breadcrumb used)"
|
||||
elif not loaded:
|
||||
row.skipped = True
|
||||
row.reason = "open timed out"
|
||||
# Return to the mailbox for the next sample.
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
time.sleep(SETTLE_S)
|
||||
return row
|
||||
|
||||
|
||||
def message_open(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
tailer: LogTailer,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> List[ReaderOpenRow]:
|
||||
"""Open up to ``count`` distinct uncached messages one at a time."""
|
||||
if adb.dry_run:
|
||||
_dry_run_open_demo(adb, log)
|
||||
return [ReaderOpenRow(index=1, label="<dry-run>", skipped=True, reason="dry-run")]
|
||||
|
||||
rows: List[ReaderOpenRow] = []
|
||||
opened: set = set()
|
||||
for i in range(1, count + 1):
|
||||
row = open_one_message(adb, package, tailer, i, opened, log)
|
||||
if row is None:
|
||||
break
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
|
||||
def _dry_run_open_demo(adb: Adb, log: Callable[[str], None]) -> None:
|
||||
"""Issue the canonical message-open command shape once (dry-run only)."""
|
||||
adb.uiautomator_dump()
|
||||
adb.input_tap(672, 504) # tap a representative message row centre
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 3: back navigation
|
||||
# --------------------------------------------------------------------------- #
|
||||
def back_nav(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> List[BackNavSample]:
|
||||
"""Time reader -> mailbox back transitions (dump-latency-bound; see report caveat)."""
|
||||
if adb.dry_run:
|
||||
adb.uiautomator_dump()
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
return [BackNavSample(index=1, back_ms=None, note="dry-run")]
|
||||
|
||||
samples: List[BackNavSample] = []
|
||||
opened: set = set()
|
||||
for i in range(1, count + 1):
|
||||
# Get into the reader by opening any message.
|
||||
root = goto_mailbox(adb, package, log)
|
||||
if root is None:
|
||||
samples.append(BackNavSample(index=i, back_ms=None, note="not on mailbox"))
|
||||
continue
|
||||
rows = uidump.find_message_rows(root, package)
|
||||
if not rows:
|
||||
samples.append(BackNavSample(index=i, back_ms=None, note="no rows"))
|
||||
continue
|
||||
adb.input_tap(*rows[0].center)
|
||||
wait_for_reader_ready(adb, package, log)
|
||||
# Now time the back transition to the mailbox.
|
||||
start = time.monotonic()
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
reached = _wait_until(
|
||||
lambda: _is_mailbox(adb, package), timeout_s=30.0
|
||||
)
|
||||
elapsed_ms = round((time.monotonic() - start) * 1000)
|
||||
samples.append(
|
||||
BackNavSample(
|
||||
index=i,
|
||||
back_ms=elapsed_ms if reached else None,
|
||||
note="" if reached else "did not reach mailbox",
|
||||
)
|
||||
)
|
||||
return samples
|
||||
|
||||
|
||||
def _is_mailbox(adb: Adb, package: str) -> bool:
|
||||
root = dump_ui(adb)
|
||||
return root is not None and uidump.is_app_foreground(root, package) and not uidump.is_reader(
|
||||
root, package
|
||||
)
|
||||
|
||||
|
||||
def _wait_until(predicate: Callable[[], bool], timeout_s: float) -> bool:
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while time.monotonic() < deadline:
|
||||
if predicate():
|
||||
return True
|
||||
time.sleep(0.2)
|
||||
return False
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 4: prefetch A/B (fetch policy toggle)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def set_fetch_policy(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
option_label: str,
|
||||
log: Callable[[str], None],
|
||||
) -> bool:
|
||||
"""Navigate Settings and select a fetch-policy option by its visible label.
|
||||
|
||||
NOTE: no uiautomator dump of the settings screen was captured in the manual run, so the
|
||||
settings-screen navigation is by on-screen text and needs live verification (documented
|
||||
in the README). Returns True if the option was found and tapped.
|
||||
"""
|
||||
if adb.dry_run:
|
||||
adb.uiautomator_dump()
|
||||
adb.input_tap(1014, 2728) # bottom-nav "Settings"
|
||||
adb.uiautomator_dump()
|
||||
adb.input_tap(672, 1400) # representative fetch-policy option
|
||||
adb.input_keyevent("KEYCODE_BACK")
|
||||
return True
|
||||
|
||||
root = goto_mailbox(adb, package, log)
|
||||
if root is None:
|
||||
return False
|
||||
settings_label = uidump.find_by_text(root, "Settings")
|
||||
if settings_label is None:
|
||||
log("no Settings entry found in bottom nav")
|
||||
return False
|
||||
anchor = settings_label.first_clickable_ancestor() or settings_label
|
||||
if anchor.center:
|
||||
adb.input_tap(*anchor.center)
|
||||
time.sleep(SETTLE_S)
|
||||
|
||||
# Find the fetch-policy option, scrolling the settings list if necessary.
|
||||
for _ in range(6):
|
||||
sroot = guard_ready(adb, package, log)
|
||||
if sroot is None:
|
||||
return False
|
||||
option = uidump.find_by_text(sroot, option_label)
|
||||
if option is not None:
|
||||
anchor = option.first_clickable_ancestor() or option
|
||||
if anchor.center:
|
||||
adb.input_tap(*anchor.center)
|
||||
log(f"selected fetch policy {option_label!r}")
|
||||
time.sleep(SETTLE_S)
|
||||
adb.input_keyevent("KEYCODE_BACK") # back to mailbox
|
||||
time.sleep(SETTLE_S)
|
||||
return True
|
||||
adb.input_swipe(672, 2000, 672, 900, 400)
|
||||
time.sleep(SETTLE_S)
|
||||
log(f"fetch-policy option {option_label!r} not found")
|
||||
return False
|
||||
|
||||
|
||||
def prefetch_ab(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
component: str,
|
||||
tailer: LogTailer,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> dict:
|
||||
"""Run message-open under prefetch ON (Wi-Fi) vs OFF (on-demand), cache cleared between."""
|
||||
result = {"conditionA": [], "conditionB": []}
|
||||
|
||||
log("Condition A: fetch policy = Fetch all on Wi-Fi (prefetch ON)")
|
||||
set_fetch_policy(adb, package, FETCH_WIFI_LABEL, log)
|
||||
_reset_for_uncached(adb, component, log)
|
||||
result["conditionA"] = message_open(adb, package, tailer, count, log)
|
||||
|
||||
log("Condition B: fetch policy = Always on-demand (prefetch OFF)")
|
||||
set_fetch_policy(adb, package, FETCH_ON_DEMAND_LABEL, log)
|
||||
_reset_for_uncached(adb, component, log)
|
||||
result["conditionB"] = message_open(adb, package, tailer, count, log)
|
||||
return result
|
||||
|
||||
|
||||
def _reset_for_uncached(adb: Adb, component: str, log: Callable[[str], None]) -> None:
|
||||
"""Clear the cache (and restart) so subsequent opens are uncached network fetches."""
|
||||
adb.force_stop()
|
||||
adb.clear_cache()
|
||||
adb.settle(1.0)
|
||||
adb.start_activity(component=component, wait=True)
|
||||
adb.settle(3.0)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Scenario 5: cross-provider (optional)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def cross_provider(
|
||||
adb: Adb,
|
||||
package: str,
|
||||
tailer: LogTailer,
|
||||
count: int,
|
||||
log: Callable[[str], None],
|
||||
) -> dict:
|
||||
"""Open ``count`` messages from the (unified) inbox and bucket the rows by account_ref.
|
||||
|
||||
The breadcrumbs carry the account reference (``imap:...`` vs ``outlook:...``), so a
|
||||
single mixed-inbox pass yields the per-provider comparison without account switching.
|
||||
"""
|
||||
rows = message_open(adb, package, tailer, count, log)
|
||||
buckets: dict = {}
|
||||
for row in rows:
|
||||
key = row.account_ref or "unknown"
|
||||
buckets.setdefault(key, []).append(row)
|
||||
return buckets
|
||||
@@ -0,0 +1,2 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Test package for the LibreMail device-testing perf harness."""
|
||||
@@ -0,0 +1,17 @@
|
||||
07-05 17:55:38.343 10261 10423 D ImapPerf: prefetch-body connect=857ms work=2150ms live=3
|
||||
07-05 17:55:54.352 10261 10425 D ImapPerf: backfill-page connect=4366ms work=71989ms live=2
|
||||
07-05 17:57:38.981 10261 10425 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=false took=21ms
|
||||
07-05 17:57:39.041 10261 10261 D Reader : reader ready took=82ms html=true inline=0
|
||||
07-05 18:04:37.001 10261 10408 I MailBackfiller: backfill slice done: moreWork=true
|
||||
07-05 18:04:37.002 10261 10408 I MailBackfiller: backfill slice: maxBatches=20
|
||||
07-05 18:07:16.094 10261 15306 D ImapPerf: body-fetch select=2566ms body=14253ms flag=2356ms rfc822=60457B chars=51610 att=0
|
||||
07-05 18:07:18.435 10261 15306 D ImapPerf: body-fetch connect=2624ms work=26211ms live=3
|
||||
07-05 18:07:20.817 10261 15306 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=true took=31227ms
|
||||
07-05 18:07:21.206 10261 10261 D Reader : reader ready took=31617ms html=true inline=0
|
||||
07-05 18:07:32.607 10261 10414 D ImapPerf: backfill-page connect=656ms work=3038ms live=3
|
||||
07-05 18:36:37.959 19255 21059 D MailBackfiller: backfill outlook:6b54d6 folder=INBOX pages=20 complete=false
|
||||
07-05 18:36:54.941 19255 19314 D ImapPerf: body-fetch select=185ms body=1355ms flag=78ms rfc822=189086B chars=82986 att=0
|
||||
07-05 18:36:55.002 19255 19314 D ImapPerf: body-fetch connect=772ms work=2018ms live=2
|
||||
07-05 18:36:55.140 19255 21060 I MailReader: openMessage outlook:6b54d6 folder=INBOX fetchedBody=true took=2934ms
|
||||
07-05 18:36:55.144 19255 19255 D Reader : reader ready took=2939ms html=true inline=0
|
||||
07-05 18:37:11.704 19255 19314 D ImapPerf: imap connect=3156ms work=8032ms live=2
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><hierarchy rotation="0"><node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="0" hint=""><node index="0" text="" resource-id="" class="android.widget.LinearLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="2" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/action_bar_root" class="android.widget.LinearLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="android:id/content" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="2" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/content" class="android.view.ViewGroup" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/digital_clock" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1047][1290,1586]" drawing-order="18" hint=""><node index="0" text="18:00" resource-id="com.google.android.deskclock:id/auto_sizing_text_clock" class="android.widget.TextView" package="com.google.android.deskclock" content-desc="18:00" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1047][1290,1586]" drawing-order="1" hint="" /></node><node index="1" text="Bae check in" resource-id="com.google.android.deskclock:id/title" class="android.widget.TextView" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1646][1290,1960]" drawing-order="19" hint="" /><node index="2" text="Snooze" resource-id="com.google.android.deskclock:id/button_tap_snooze" class="android.widget.Button" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[126,2274][669,2562]" drawing-order="39" hint="" /><node index="3" text="Stop" resource-id="com.google.android.deskclock:id/button_tap_dismiss" class="android.widget.Button" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[675,2274][1218,2562]" drawing-order="40" hint="" /></node></node></node></node></node></node></hierarchy>
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><hierarchy rotation="0"><node index="0" text="" resource-id="com.android.systemui:id/legacy_window_root" class="android.widget.FrameLayout" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[0,0][1344,2992]"><node index="0" text="" resource-id="com.android.systemui:id/keyguard_root_view" class="android.view.ViewGroup" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[0,0][1344,2992]"><node index="0" text="" resource-id="com.android.systemui:id/accessibility_actions_view" class="android.view.View" package="com.android.systemui" content-desc="Lock screen" clickable="false" enabled="true" long-clickable="true" scrollable="false" bounds="[0,216][1344,2992]" /><node index="1" text="18 02" resource-id="" class="android.widget.TextView" package="com.android.systemui" content-desc="18:02" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[100,454][583,620]" /><node index="2" text="" resource-id="com.android.systemui:id/device_entry_icon_fg" class="android.widget.ImageView" package="com.android.systemui" content-desc="Fingerprint sensor" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[561,2126][791,2356]" /><node index="3" text="Charged" resource-id="com.android.systemui:id/keyguard_indication_text" class="android.widget.TextView" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[16,2860][1344,2925]" /></node></node></hierarchy>
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,160 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Tests for the adb safety guardrails -- the harness must refuse dangerous commands."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import adb # noqa: E402
|
||||
from adb import Adb, AdbSafetyError, assert_safe # noqa: E402
|
||||
|
||||
PKG = "org.libremail.app"
|
||||
|
||||
|
||||
class TestAllowedCommands(unittest.TestCase):
|
||||
def _ok(self, args):
|
||||
try:
|
||||
assert_safe(PKG, args)
|
||||
except AdbSafetyError as exc: # pragma: no cover - failure path
|
||||
self.fail(f"assert_safe wrongly rejected {args}: {exc}")
|
||||
|
||||
def test_lifecycle_and_input_allowed(self):
|
||||
self._ok(["shell", "am", "force-stop", PKG])
|
||||
self._ok(["shell", "am", "start", "-W", "-n", f"{PKG}/org.libremail.MainActivity"])
|
||||
self._ok(["shell", "input", "tap", "672", "504"])
|
||||
self._ok(["shell", "input", "swipe", "672", "2000", "672", "900", "400"])
|
||||
self._ok(["shell", "input", "keyevent", "KEYCODE_BACK"])
|
||||
self._ok(["shell", "input", "keyevent", "KEYCODE_WAKEUP"])
|
||||
self._ok(["shell", "uiautomator", "dump", "/dev/tty"])
|
||||
self._ok(["shell", "svc", "power", "stayon", "true"])
|
||||
self._ok(["shell", "dumpsys", "gfxinfo", PKG])
|
||||
self._ok(["shell", "settings", "get", "global", "airplane_mode_on"])
|
||||
self._ok(["shell", "logcat", "-c"])
|
||||
self._ok(["logcat", "-b", "all", "-v", "threadtime"])
|
||||
self._ok(["devices"])
|
||||
self._ok(["get-state"])
|
||||
self._ok(["install", "-r", "app-debug.apk"])
|
||||
|
||||
def test_sanctioned_cache_clear_allowed(self):
|
||||
self._ok(["shell", "run-as", PKG, "sh", "-c", "rm -rf cache/*"])
|
||||
self._ok(["shell", "run-as", PKG, "sh", "-c", "rm -rf cache"])
|
||||
self._ok(["shell", "run-as", PKG, "ls", "-la", "cache"])
|
||||
|
||||
def test_input_swipe_not_confused_with_wipe(self):
|
||||
# 'swipe' contains the substring 'wipe'; it must still be allowed.
|
||||
self._ok(["shell", "input", "swipe", "0", "0", "0", "500", "300"])
|
||||
|
||||
|
||||
class TestDeniedCommands(unittest.TestCase):
|
||||
def _deny(self, args):
|
||||
with self.assertRaises(AdbSafetyError):
|
||||
assert_safe(PKG, args)
|
||||
|
||||
def test_top_level_allowlist(self):
|
||||
self._deny(["uninstall", PKG])
|
||||
self._deny(["root"])
|
||||
self._deny(["remount"])
|
||||
self._deny(["reboot"])
|
||||
self._deny(["disable-verity"])
|
||||
self._deny(["emu", "kill"])
|
||||
self._deny(["push", "x", "/data"])
|
||||
self._deny(["pull", "/data/data/org.libremail.app/databases/x"])
|
||||
|
||||
def test_pm_clear_and_uninstall_denied(self):
|
||||
self._deny(["shell", "pm", "clear", PKG])
|
||||
self._deny(["shell", "pm", "uninstall", PKG])
|
||||
self._deny(["shell", "pm", "disable", PKG])
|
||||
|
||||
def test_reboot_and_root_via_shell_denied(self):
|
||||
self._deny(["shell", "reboot"])
|
||||
self._deny(["shell", "svc", "power", "reboot"])
|
||||
self._deny(["shell", "su", "-c", "reboot"])
|
||||
|
||||
def test_no_touching_private_dirs(self):
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf databases"])
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf shared_prefs"])
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf datastore"])
|
||||
self._deny(["shell", "run-as", PKG, "cat", "databases/libremail.db"])
|
||||
self._deny(["shell", "run-as", PKG, "ls", "files/datastore"])
|
||||
|
||||
def test_rm_outside_cache_denied(self):
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf /sdcard/x"])
|
||||
self._deny(["shell", "run-as", PKG, "sh", "-c", "rm -rf cache/../databases"])
|
||||
self._deny(["shell", "rm", "-rf", "/data/local/tmp"])
|
||||
|
||||
def test_output_redirect_denied(self):
|
||||
self._deny(["shell", "echo", "x", ">", "/sdcard/y"])
|
||||
|
||||
def test_run_as_foreign_package_denied(self):
|
||||
self._deny(["shell", "run-as", "com.other.app", "ls", "cache"])
|
||||
|
||||
def test_am_foreign_targets_denied(self):
|
||||
self._deny(["shell", "am", "force-stop", "com.other.app"])
|
||||
self._deny(["shell", "am", "start", "-n", "com.other.app/.Main"])
|
||||
|
||||
def test_empty_command_denied(self):
|
||||
self._deny([])
|
||||
|
||||
|
||||
class TestAdbWrapperBuildsSafeArgv(unittest.TestCase):
|
||||
"""In dry-run the wrapper still validates and returns the argv it would run."""
|
||||
|
||||
def setUp(self):
|
||||
self.adb = Adb(serial="SERIAL123", package=PKG, dry_run=True)
|
||||
|
||||
def test_force_stop_argv(self):
|
||||
self.assertEqual(self.adb.force_stop().args, ["shell", "am", "force-stop", PKG])
|
||||
|
||||
def test_start_activity_argv(self):
|
||||
args = self.adb.start_activity().args
|
||||
self.assertEqual(args[:5], ["shell", "am", "start", "-W", "-n"])
|
||||
self.assertTrue(args[5].startswith(PKG + "/"))
|
||||
|
||||
def test_clear_cache_argv_is_sanctioned(self):
|
||||
self.assertEqual(
|
||||
self.adb.clear_cache().args,
|
||||
["shell", "run-as", PKG, "sh", "-c", "rm -rf cache/*"],
|
||||
)
|
||||
|
||||
def test_argv_includes_serial(self):
|
||||
full = self.adb._argv(["devices"])
|
||||
self.assertEqual(full, ["adb", "-s", "SERIAL123", "devices"])
|
||||
|
||||
def test_wrapper_rejects_unsafe_low_level_call(self):
|
||||
with self.assertRaises(AdbSafetyError):
|
||||
self.adb.run(["shell", "pm", "clear", PKG])
|
||||
|
||||
|
||||
class TestPureParsers(unittest.TestCase):
|
||||
def test_parse_devices(self):
|
||||
out = (
|
||||
"List of devices attached\n"
|
||||
"5C310DLCQ000G3\tdevice\n"
|
||||
"emulator-5554\tdevice\n"
|
||||
"0123456789\toffline\n"
|
||||
)
|
||||
self.assertEqual(adb.parse_devices(out), ["5C310DLCQ000G3", "emulator-5554"])
|
||||
|
||||
def test_parse_devices_empty(self):
|
||||
self.assertEqual(adb.parse_devices("List of devices attached\n"), [])
|
||||
|
||||
def test_parse_am_start(self):
|
||||
out = (
|
||||
"Starting: Intent { cmp=org.libremail.app/org.libremail.MainActivity }\n"
|
||||
"Status: ok\n"
|
||||
"LaunchState: COLD\n"
|
||||
"TotalTime: 384\n"
|
||||
"WaitTime: 386\n"
|
||||
)
|
||||
fields = adb.parse_am_start(out)
|
||||
self.assertEqual(fields["TotalTime"], 384)
|
||||
self.assertEqual(fields["WaitTime"], 386)
|
||||
|
||||
def test_parse_am_start_missing(self):
|
||||
self.assertEqual(adb.parse_am_start("Status: ok\n"), {})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,222 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Unit tests for the breadcrumb parser -- validated against the manual run's real captures.
|
||||
|
||||
The fixture ``fixtures/perf-extract-sample.log`` is a verbatim slice of the manual run's
|
||||
``perf-extract-ALL.log`` (plus the Outlook control), so these tests assert that the parser
|
||||
reproduces the exact figures in the hand-written ``timing-tables.md``.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import breadcrumbs as bc # noqa: E402
|
||||
|
||||
FIXTURE = os.path.join(os.path.dirname(__file__), "fixtures", "perf-extract-sample.log")
|
||||
|
||||
|
||||
def _load_fixture_lines():
|
||||
with open(FIXTURE, encoding="utf-8") as fh:
|
||||
return fh.read().splitlines()
|
||||
|
||||
|
||||
class TestLogcatLine(unittest.TestCase):
|
||||
def test_parses_threadtime_fields(self):
|
||||
line = "07-05 18:07:20.817 10261 15306 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=true took=31227ms"
|
||||
parsed = bc.parse_logcat_line(line)
|
||||
self.assertIsNotNone(parsed)
|
||||
self.assertEqual(parsed.date, "07-05")
|
||||
self.assertEqual(parsed.time, "18:07:20.817")
|
||||
self.assertEqual(parsed.pid, 10261)
|
||||
self.assertEqual(parsed.tid, 15306)
|
||||
self.assertEqual(parsed.level, "I")
|
||||
self.assertEqual(parsed.tag, "MailReader")
|
||||
self.assertTrue(parsed.message.startswith("openMessage imap:94058a"))
|
||||
self.assertEqual(parsed.timestamp, "07-05 18:07:20.817")
|
||||
|
||||
def test_tolerates_space_padded_tag(self):
|
||||
# logcat pads the tag column, e.g. "Reader :".
|
||||
line = "07-05 17:57:39.041 10261 10261 D Reader : reader ready took=82ms html=true inline=0"
|
||||
parsed = bc.parse_logcat_line(line)
|
||||
self.assertEqual(parsed.tag, "Reader")
|
||||
self.assertEqual(parsed.message, "reader ready took=82ms html=true inline=0")
|
||||
|
||||
def test_non_threadtime_returns_none(self):
|
||||
self.assertIsNone(bc.parse_logcat_line("--------- beginning of main"))
|
||||
self.assertIsNone(bc.parse_logcat_line(""))
|
||||
|
||||
def test_is_perf_line(self):
|
||||
self.assertTrue(
|
||||
bc.is_perf_line("07-05 17:55:38.343 10261 10423 D ImapPerf: prefetch-body connect=857ms work=2150ms live=3")
|
||||
)
|
||||
self.assertFalse(
|
||||
bc.is_perf_line("07-05 15:00:06.896 15192 15217 I MailSyncer: sync all: 1 accounts")
|
||||
)
|
||||
self.assertFalse(bc.is_perf_line("garbage"))
|
||||
|
||||
|
||||
class TestMessageParsers(unittest.TestCase):
|
||||
def test_imap_perf_generic_op(self):
|
||||
ev = bc.parse_imap_perf("prefetch-body connect=857ms work=2150ms live=3")
|
||||
self.assertIsInstance(ev, bc.ImapPerfOp)
|
||||
self.assertEqual((ev.op, ev.connect_ms, ev.work_ms, ev.live), ("prefetch-body", 857, 2150, 3))
|
||||
|
||||
def test_imap_perf_backfill_page(self):
|
||||
ev = bc.parse_imap_perf("backfill-page connect=4366ms work=71989ms live=2")
|
||||
self.assertEqual((ev.op, ev.connect_ms, ev.work_ms, ev.live), ("backfill-page", 4366, 71989, 2))
|
||||
|
||||
def test_imap_perf_body_fetch_op(self):
|
||||
ev = bc.parse_imap_perf("body-fetch connect=2624ms work=26211ms live=3")
|
||||
self.assertIsInstance(ev, bc.ImapPerfOp)
|
||||
self.assertEqual(ev.op, "body-fetch")
|
||||
|
||||
def test_body_fetch_detail(self):
|
||||
ev = bc.parse_imap_perf(
|
||||
"body-fetch select=2566ms body=14253ms flag=2356ms rfc822=60457B chars=51610 att=0"
|
||||
)
|
||||
self.assertIsInstance(ev, bc.BodyFetch)
|
||||
self.assertEqual(ev.select_ms, 2566)
|
||||
self.assertEqual(ev.body_ms, 14253)
|
||||
self.assertEqual(ev.flag_ms, 2356)
|
||||
self.assertEqual(ev.rfc822_bytes, 60457)
|
||||
self.assertEqual(ev.chars, 51610)
|
||||
self.assertEqual(ev.att, 0)
|
||||
|
||||
def test_body_fetch_throughput_matches_manual_table(self):
|
||||
# timing-tables.md reports 4.2 KB/s for A1 and 139.5 KB/s for O1.
|
||||
a1 = bc.parse_imap_perf(
|
||||
"body-fetch select=2566ms body=14253ms flag=2356ms rfc822=60457B chars=51610 att=0"
|
||||
)
|
||||
self.assertAlmostEqual(a1.body_kb_per_s, 4.24, places=2)
|
||||
o1 = bc.parse_imap_perf(
|
||||
"body-fetch select=185ms body=1355ms flag=78ms rfc822=189086B chars=82986 att=0"
|
||||
)
|
||||
self.assertAlmostEqual(o1.body_kb_per_s, 139.5, places=1)
|
||||
|
||||
def test_open_message_uncached(self):
|
||||
ev = bc.parse_open_message("openMessage imap:94058a folder=INBOX fetchedBody=true took=31227ms")
|
||||
self.assertEqual(ev.account_ref, "imap:94058a")
|
||||
self.assertEqual(ev.folder, "INBOX")
|
||||
self.assertTrue(ev.fetched_body)
|
||||
self.assertEqual(ev.took_ms, 31227)
|
||||
|
||||
def test_open_message_cached(self):
|
||||
ev = bc.parse_open_message("openMessage outlook:6b54d6 folder=INBOX fetchedBody=false took=4ms")
|
||||
self.assertEqual(ev.account_ref, "outlook:6b54d6")
|
||||
self.assertFalse(ev.fetched_body)
|
||||
self.assertEqual(ev.took_ms, 4)
|
||||
|
||||
def test_reader_ready(self):
|
||||
ev = bc.parse_reader_ready("reader ready took=31617ms html=true inline=0")
|
||||
self.assertEqual(ev.took_ms, 31617)
|
||||
self.assertTrue(ev.html)
|
||||
self.assertEqual(ev.inline, 0)
|
||||
|
||||
def test_backfill_progress(self):
|
||||
ev = bc.parse_backfill("backfill outlook:6b54d6 folder=INBOX pages=20 complete=false")
|
||||
self.assertIsInstance(ev, bc.BackfillProgress)
|
||||
self.assertEqual(ev.account_ref, "outlook:6b54d6")
|
||||
self.assertEqual(ev.pages, 20)
|
||||
self.assertFalse(ev.complete)
|
||||
|
||||
def test_backfill_slice_start_and_done(self):
|
||||
start = bc.parse_backfill("backfill slice: maxBatches=20")
|
||||
self.assertIsInstance(start, bc.BackfillSliceStart)
|
||||
self.assertEqual(start.max_batches, 20)
|
||||
done = bc.parse_backfill("backfill slice done: moreWork=true")
|
||||
self.assertIsInstance(done, bc.BackfillSliceDone)
|
||||
self.assertTrue(done.more_work)
|
||||
|
||||
def test_unparseable_messages_return_none(self):
|
||||
self.assertIsNone(bc.parse_imap_perf("nonsense here"))
|
||||
self.assertIsNone(bc.parse_open_message("openMessage missing fields"))
|
||||
self.assertIsNone(bc.parse_reader_ready("reader not ready"))
|
||||
self.assertIsNone(bc.parse_backfill("backfill mystery"))
|
||||
|
||||
|
||||
class TestParseBreadcrumbDispatch(unittest.TestCase):
|
||||
def test_dispatch_attaches_source_line(self):
|
||||
line = "07-05 18:07:18.435 10261 15306 D ImapPerf: body-fetch connect=2624ms work=26211ms live=3"
|
||||
ev = bc.parse_breadcrumb(line)
|
||||
self.assertIsInstance(ev, bc.ImapPerfOp)
|
||||
self.assertIsNotNone(ev.line)
|
||||
self.assertEqual(ev.line.pid, 10261)
|
||||
self.assertEqual(ev.timestamp, "07-05 18:07:18.435")
|
||||
|
||||
def test_dispatch_ignores_non_perf_tags(self):
|
||||
self.assertIsNone(
|
||||
bc.parse_breadcrumb("07-05 15:00:06.896 15192 15217 I MailSyncer: sync all: 1 accounts")
|
||||
)
|
||||
|
||||
|
||||
class TestFixtureStream(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.lines = _load_fixture_lines()
|
||||
self.events = list(bc.iter_events(self.lines))
|
||||
|
||||
def test_all_fixture_lines_are_perf_lines(self):
|
||||
self.assertEqual(sum(1 for line in self.lines if bc.is_perf_line(line)), len(self.lines))
|
||||
|
||||
def test_every_line_parses(self):
|
||||
self.assertEqual(len(self.events), len(self.lines))
|
||||
|
||||
def test_event_type_mix(self):
|
||||
kinds = [type(e).__name__ for e in self.events]
|
||||
# 3 prefetch/backfill-page ops + 2 body-fetch ops + 1 imap op = 6.
|
||||
self.assertEqual(kinds.count("ImapPerfOp"), 6)
|
||||
self.assertEqual(kinds.count("BodyFetch"), 2)
|
||||
self.assertEqual(kinds.count("OpenMessage"), 3)
|
||||
self.assertEqual(kinds.count("ReaderReady"), 3)
|
||||
self.assertEqual(kinds.count("BackfillProgress"), 1)
|
||||
self.assertEqual(kinds.count("BackfillSliceStart"), 1)
|
||||
self.assertEqual(kinds.count("BackfillSliceDone"), 1)
|
||||
|
||||
|
||||
class TestCorrelateOpens(unittest.TestCase):
|
||||
"""correlate_opens must reproduce the manual timing-tables rows exactly."""
|
||||
|
||||
def setUp(self):
|
||||
events = list(bc.iter_events(_load_fixture_lines()))
|
||||
self.samples = bc.correlate_opens(events)
|
||||
|
||||
def test_three_opens_detected(self):
|
||||
self.assertEqual(len(self.samples), 3)
|
||||
|
||||
def test_cached_gmail_open(self):
|
||||
s = self.samples[0]
|
||||
self.assertEqual(s.account_ref, "imap:94058a")
|
||||
self.assertTrue(s.cached)
|
||||
self.assertEqual(s.took_ms, 21)
|
||||
self.assertIsNone(s.body_fetch)
|
||||
self.assertEqual(s.reader_ready.took_ms, 82)
|
||||
|
||||
def test_gmail_A1_row_matches_table1(self):
|
||||
s = self.samples[1]
|
||||
self.assertEqual(s.account_ref, "imap:94058a")
|
||||
self.assertFalse(s.cached)
|
||||
self.assertEqual(s.took_ms, 31227)
|
||||
self.assertEqual(s.reader_ready.took_ms, 31617)
|
||||
self.assertEqual(s.rfc822_bytes, 60457)
|
||||
self.assertEqual(s.body_fetch.select_ms, 2566)
|
||||
self.assertEqual(s.body_fetch.body_ms, 14253)
|
||||
self.assertEqual(s.body_fetch.flag_ms, 2356)
|
||||
self.assertEqual(s.body_fetch_op.connect_ms, 2624)
|
||||
self.assertEqual(s.body_fetch_op.work_ms, 26211)
|
||||
self.assertEqual(s.body_fetch_op.live, 3)
|
||||
self.assertAlmostEqual(s.body_kb_per_s, 4.24, places=2)
|
||||
|
||||
def test_outlook_O1_row_matches_table2(self):
|
||||
s = self.samples[2]
|
||||
self.assertEqual(s.account_ref, "outlook:6b54d6")
|
||||
self.assertFalse(s.cached)
|
||||
self.assertEqual(s.took_ms, 2934)
|
||||
self.assertEqual(s.reader_ready.took_ms, 2939)
|
||||
self.assertEqual(s.rfc822_bytes, 189086)
|
||||
self.assertEqual(s.body_fetch_op.connect_ms, 772)
|
||||
self.assertAlmostEqual(s.body_kb_per_s, 139.5, places=1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,109 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Tests for aggregation and the timing-tables markdown renderers."""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import breadcrumbs as bc # noqa: E402
|
||||
import report # noqa: E402
|
||||
|
||||
FIXTURE = os.path.join(os.path.dirname(__file__), "fixtures", "perf-extract-sample.log")
|
||||
|
||||
|
||||
class TestAggregate(unittest.TestCase):
|
||||
def test_basic_stats(self):
|
||||
agg = report.aggregate([10, 20, 30])
|
||||
self.assertEqual(agg.n, 3)
|
||||
self.assertEqual(agg.mean, 20)
|
||||
self.assertEqual(agg.median, 20)
|
||||
self.assertEqual((agg.minimum, agg.maximum), (10, 30))
|
||||
|
||||
def test_reproduces_manual_gmail_median(self):
|
||||
# timing-tables.md Condition A openMessage values -> median ~32.1 s.
|
||||
agg = report.aggregate([31227, 32059, 31721, 32148, 32949, 39728])
|
||||
self.assertEqual(agg.median, 32103.5)
|
||||
|
||||
def test_empty(self):
|
||||
agg = report.aggregate([])
|
||||
self.assertEqual(agg.n, 0)
|
||||
self.assertIsNone(agg.mean)
|
||||
self.assertIsNone(agg.median)
|
||||
|
||||
|
||||
class TestMdTable(unittest.TestCase):
|
||||
def test_table_shape(self):
|
||||
table = report.md_table(["a", "b"], [[1, 2], [3, None]])
|
||||
lines = table.splitlines()
|
||||
self.assertEqual(lines[0], "| a | b |")
|
||||
self.assertEqual(lines[1], "| --- | --- |")
|
||||
self.assertEqual(lines[2], "| 1 | 2 |")
|
||||
self.assertEqual(lines[3], "| 3 | |") # None renders empty
|
||||
|
||||
|
||||
class TestColdOpen(unittest.TestCase):
|
||||
def test_render_includes_mean_row(self):
|
||||
samples = [
|
||||
report.ColdOpenSample(run=1, total_time_ms=523, wait_time_ms=526),
|
||||
report.ColdOpenSample(run=2, total_time_ms=384, wait_time_ms=386),
|
||||
]
|
||||
md = report.render_cold_open(samples)
|
||||
self.assertIn("523", md)
|
||||
self.assertIn("**mean**", md)
|
||||
self.assertIn("454", md) # mean total = 453.5 -> 454
|
||||
|
||||
|
||||
class TestMessageOpen(unittest.TestCase):
|
||||
def setUp(self):
|
||||
with open(FIXTURE, encoding="utf-8") as fh:
|
||||
events = list(bc.iter_events(fh.read().splitlines()))
|
||||
self.samples = bc.correlate_opens(events)
|
||||
|
||||
def test_from_open_sample_maps_fields(self):
|
||||
row = report.ReaderOpenRow.from_open_sample(self.samples[1], index=1, label="A1")
|
||||
self.assertEqual(row.took_ms, 31227)
|
||||
self.assertEqual(row.reader_ready_ms, 31617)
|
||||
self.assertEqual(row.rfc822_bytes, 60457)
|
||||
self.assertEqual(row.connect_ms, 2624)
|
||||
self.assertEqual(row.select_ms, 2566)
|
||||
self.assertFalse(row.cached)
|
||||
|
||||
def test_render_message_open_table_and_aggregate(self):
|
||||
rows = [
|
||||
report.ReaderOpenRow.from_open_sample(s, index=i, label=f"m{i}")
|
||||
for i, s in enumerate(self.samples, start=1)
|
||||
]
|
||||
md = report.render_message_open("Message open (uncached)", rows)
|
||||
self.assertIn("31227 ms", md)
|
||||
self.assertIn("2934 ms", md)
|
||||
self.assertIn("60457", md)
|
||||
self.assertIn("4.2", md) # Gmail throughput
|
||||
self.assertIn("139.5", md) # Outlook throughput
|
||||
self.assertIn("n=2", md) # two uncached opens among the three
|
||||
|
||||
def test_skipped_row_renders(self):
|
||||
rows = [report.ReaderOpenRow(index=1, label="x", skipped=True, reason="keyguard")]
|
||||
md = report.render_message_open("t", rows)
|
||||
self.assertIn("SKIPPED: keyguard", md)
|
||||
|
||||
|
||||
class TestBackNav(unittest.TestCase):
|
||||
def test_caveat_present(self):
|
||||
samples = [report.BackNavSample(index=1, back_ms=3484)]
|
||||
md = report.render_back_nav(samples)
|
||||
self.assertIn("Caveat", md)
|
||||
self.assertIn("3484 ms", md)
|
||||
|
||||
|
||||
class TestDocument(unittest.TestCase):
|
||||
def test_build_document(self):
|
||||
doc = report.build_document({"scenario": "cold-open"}, ["## Section\n\nbody\n"])
|
||||
self.assertIn("# LibreMail device perf run", doc)
|
||||
self.assertIn("**scenario:** cold-open", doc)
|
||||
self.assertIn("## Section", doc)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,108 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Tests for the uiautomator-dump parser and screen/keyguard recognition.
|
||||
|
||||
Fixtures are cleaned copies of the manual run's real dumps: the reader (``ui_reader.xml``),
|
||||
a deskclock alarm (``ui_alarm.xml``) and the keyguard (``ui_lockscreen.xml``) are the
|
||||
foreign-screen negatives the harness must skip; ``ui_mailbox.xml`` is a compact, structurally
|
||||
faithful mailbox list.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import uidump # noqa: E402
|
||||
|
||||
FIX = os.path.join(os.path.dirname(__file__), "fixtures")
|
||||
APP = "org.libremail.app"
|
||||
|
||||
|
||||
def load(name):
|
||||
with open(os.path.join(FIX, name), encoding="utf-8") as fh:
|
||||
return uidump.parse_dump(fh.read())
|
||||
|
||||
|
||||
class TestBounds(unittest.TestCase):
|
||||
def test_parse_bounds(self):
|
||||
self.assertEqual(uidump.parse_bounds("[0,392][1344,617]"), (0, 392, 1344, 617))
|
||||
|
||||
def test_parse_bounds_none(self):
|
||||
self.assertIsNone(uidump.parse_bounds(""))
|
||||
self.assertIsNone(uidump.parse_bounds("not-bounds"))
|
||||
|
||||
|
||||
class TestMailbox(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.root = load("ui_mailbox.xml")
|
||||
|
||||
def test_recognised_as_app_not_lockscreen(self):
|
||||
self.assertEqual(uidump.foreground_package(self.root), APP)
|
||||
self.assertFalse(uidump.is_lockscreen(self.root))
|
||||
self.assertTrue(uidump.is_app_foreground(self.root, APP))
|
||||
self.assertFalse(uidump.is_reader(self.root, APP))
|
||||
|
||||
def test_finds_three_message_rows(self):
|
||||
rows = uidump.find_message_rows(self.root, APP)
|
||||
self.assertEqual(len(rows), 3)
|
||||
|
||||
def test_row_centers_and_cached_flags(self):
|
||||
rows = uidump.find_message_rows(self.root, APP)
|
||||
self.assertEqual(rows[0].center, (672, 504))
|
||||
self.assertFalse(rows[0].cached)
|
||||
self.assertEqual(rows[1].center, (672, 732))
|
||||
self.assertFalse(rows[1].cached)
|
||||
# Row 2 carries the "Available offline" content-desc -> cached.
|
||||
self.assertEqual(rows[2].center, (672, 988))
|
||||
self.assertTrue(rows[2].cached)
|
||||
|
||||
def test_row_label_drops_monogram(self):
|
||||
rows = uidump.find_message_rows(self.root, APP)
|
||||
self.assertIn("github-actions[bot]", rows[0].label)
|
||||
self.assertNotIn(" G ", rows[0].label) # single-letter avatar filtered out
|
||||
|
||||
def test_settings_entry_tappable(self):
|
||||
node = uidump.find_by_text(self.root, "Settings")
|
||||
self.assertIsNotNone(node)
|
||||
anchor = node.first_clickable_ancestor()
|
||||
self.assertIsNotNone(anchor)
|
||||
self.assertEqual(anchor.center, (1014, 2728))
|
||||
|
||||
def test_top_bar_content_descs(self):
|
||||
self.assertIsNotNone(uidump.find_by_content_desc(self.root, "Show folders"))
|
||||
self.assertIsNotNone(uidump.find_by_content_desc(self.root, "Search"))
|
||||
|
||||
|
||||
class TestReader(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.root = load("ui_reader.xml")
|
||||
|
||||
def test_recognised_as_reader(self):
|
||||
self.assertTrue(uidump.is_app_foreground(self.root, APP))
|
||||
self.assertTrue(uidump.is_reader(self.root, APP))
|
||||
|
||||
def test_progress_bar_signals_loading(self):
|
||||
self.assertTrue(uidump.has_progress_bar(self.root))
|
||||
|
||||
def test_no_message_rows_in_reader(self):
|
||||
self.assertEqual(uidump.find_message_rows(self.root, APP), [])
|
||||
|
||||
|
||||
class TestForeignScreensAreGuarded(unittest.TestCase):
|
||||
def test_lockscreen_detected(self):
|
||||
root = load("ui_lockscreen.xml")
|
||||
self.assertTrue(uidump.is_lockscreen(root))
|
||||
self.assertFalse(uidump.is_app_foreground(root, APP))
|
||||
self.assertFalse(uidump.is_reader(root, APP))
|
||||
|
||||
def test_alarm_is_foreign_app(self):
|
||||
root = load("ui_alarm.xml")
|
||||
self.assertEqual(uidump.foreground_package(root), "com.google.android.deskclock")
|
||||
self.assertFalse(uidump.is_app_foreground(root, APP))
|
||||
self.assertFalse(uidump.is_lockscreen(root))
|
||||
self.assertFalse(uidump.is_reader(root, APP))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,230 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""uidump.py -- parse ``uiautomator dump`` XML and recognise LibreMail's screens.
|
||||
|
||||
Pure (no device dependency) so it can be unit-tested against the saved dumps from the
|
||||
manual run. The Compose UI exposes no stable ``resource-id``s, so screens and rows are
|
||||
recognised structurally -- by class, clickable flags, bounds, and descendant
|
||||
text/content-desc -- exactly as observed in ``ui_mailbox.xml`` / ``ui_reader.xml``.
|
||||
|
||||
It also provides the keyguard / foreign-app guards the harness needs: a uiautomator sample
|
||||
taken against the lockscreen (``ui-mbox.xml`` in the manual run was a keyguard capture) or
|
||||
another app (``ui-03.xml`` was a deskclock alarm) must be detected and skipped.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
from collections import Counter
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Callable, List, Optional, Tuple
|
||||
|
||||
SYSTEMUI_PACKAGE = "com.android.systemui"
|
||||
_BOUNDS_RE = re.compile(r"\[(-?\d+),(-?\d+)\]\[(-?\d+),(-?\d+)\]")
|
||||
|
||||
|
||||
def parse_bounds(text: str) -> Optional[Tuple[int, int, int, int]]:
|
||||
"""``"[x1,y1][x2,y2]"`` -> ``(x1, y1, x2, y2)`` (or ``None``)."""
|
||||
m = _BOUNDS_RE.search(text or "")
|
||||
if not m:
|
||||
return None
|
||||
return tuple(int(g) for g in m.groups()) # type: ignore[return-value]
|
||||
|
||||
|
||||
@dataclass
|
||||
class UiNode:
|
||||
"""One uiautomator node with parsed attributes and parent/child links."""
|
||||
|
||||
cls: str = ""
|
||||
text: str = ""
|
||||
resource_id: str = ""
|
||||
content_desc: str = ""
|
||||
package: str = ""
|
||||
clickable: bool = False
|
||||
long_clickable: bool = False
|
||||
scrollable: bool = False
|
||||
enabled: bool = False
|
||||
bounds: Optional[Tuple[int, int, int, int]] = None
|
||||
parent: Optional["UiNode"] = None
|
||||
children: List["UiNode"] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def center(self) -> Optional[Tuple[int, int]]:
|
||||
if not self.bounds:
|
||||
return None
|
||||
x1, y1, x2, y2 = self.bounds
|
||||
return ((x1 + x2) // 2, (y1 + y2) // 2)
|
||||
|
||||
@property
|
||||
def height(self) -> int:
|
||||
return (self.bounds[3] - self.bounds[1]) if self.bounds else 0
|
||||
|
||||
def walk(self):
|
||||
"""Depth-first iterate over this node and all descendants."""
|
||||
yield self
|
||||
for child in self.children:
|
||||
yield from child.walk()
|
||||
|
||||
def find_all(self, predicate: Callable[["UiNode"], bool]) -> List["UiNode"]:
|
||||
return [n for n in self.walk() if predicate(n)]
|
||||
|
||||
def first_clickable_ancestor(self) -> Optional["UiNode"]:
|
||||
node: Optional[UiNode] = self
|
||||
while node is not None:
|
||||
if node.clickable:
|
||||
return node
|
||||
node = node.parent
|
||||
return None
|
||||
|
||||
def descendant_texts(self) -> List[str]:
|
||||
"""Non-empty ``text`` values under this node, in document order."""
|
||||
return [n.text for n in self.walk() if n.text]
|
||||
|
||||
def descendant_descs(self) -> List[str]:
|
||||
return [n.content_desc for n in self.walk() if n.content_desc]
|
||||
|
||||
|
||||
def _to_bool(value: Optional[str]) -> bool:
|
||||
return value == "true"
|
||||
|
||||
|
||||
def _build(elem: ET.Element, parent: Optional[UiNode]) -> UiNode:
|
||||
node = UiNode(
|
||||
cls=elem.get("class", ""),
|
||||
text=elem.get("text", ""),
|
||||
resource_id=elem.get("resource-id", ""),
|
||||
content_desc=elem.get("content-desc", ""),
|
||||
package=elem.get("package", ""),
|
||||
clickable=_to_bool(elem.get("clickable")),
|
||||
long_clickable=_to_bool(elem.get("long-clickable")),
|
||||
scrollable=_to_bool(elem.get("scrollable")),
|
||||
enabled=_to_bool(elem.get("enabled")),
|
||||
bounds=parse_bounds(elem.get("bounds", "")),
|
||||
parent=parent,
|
||||
)
|
||||
for child_elem in list(elem):
|
||||
if child_elem.tag == "node":
|
||||
node.children.append(_build(child_elem, node))
|
||||
return node
|
||||
|
||||
|
||||
def parse_dump(xml_text: str) -> UiNode:
|
||||
"""Parse uiautomator XML into a synthetic root :class:`UiNode` (the ``<hierarchy>``)."""
|
||||
root_elem = ET.fromstring(xml_text.strip())
|
||||
root = UiNode(cls="hierarchy")
|
||||
for child_elem in list(root_elem):
|
||||
if child_elem.tag == "node":
|
||||
root.children.append(_build(child_elem, root))
|
||||
return root
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Screen recognition
|
||||
# --------------------------------------------------------------------------- #
|
||||
def foreground_package(root: UiNode) -> str:
|
||||
"""The package that owns most of the tree -- a robust "what's on screen" signal."""
|
||||
counts = Counter(n.package for n in root.walk() if n.package)
|
||||
if not counts:
|
||||
return ""
|
||||
return counts.most_common(1)[0][0]
|
||||
|
||||
|
||||
def is_lockscreen(root: UiNode) -> bool:
|
||||
"""True if the dump is the keyguard/lockscreen rather than an app."""
|
||||
if foreground_package(root) != SYSTEMUI_PACKAGE:
|
||||
return False
|
||||
for node in root.walk():
|
||||
if "keyguard" in node.resource_id or node.content_desc in (
|
||||
"Lock screen",
|
||||
"Fingerprint sensor",
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def is_app_foreground(root: UiNode, package: str) -> bool:
|
||||
"""True if ``package`` is the foreground app in this dump."""
|
||||
return foreground_package(root) == package
|
||||
|
||||
|
||||
def find_by_text(root: UiNode, text: str, exact: bool = True) -> Optional[UiNode]:
|
||||
for node in root.walk():
|
||||
if (node.text == text) if exact else (text in node.text):
|
||||
return node
|
||||
return None
|
||||
|
||||
|
||||
def find_by_content_desc(root: UiNode, desc: str, exact: bool = True) -> Optional[UiNode]:
|
||||
for node in root.walk():
|
||||
if (node.content_desc == desc) if exact else (desc in node.content_desc):
|
||||
return node
|
||||
return None
|
||||
|
||||
|
||||
def is_reader(root: UiNode, package: str) -> bool:
|
||||
"""The message reader: a Back affordance plus the 'Message' title, in our package."""
|
||||
if not is_app_foreground(root, package):
|
||||
return False
|
||||
has_back = find_by_content_desc(root, "Back") is not None
|
||||
has_title = find_by_text(root, "Message") is not None
|
||||
return has_back and has_title
|
||||
|
||||
|
||||
def has_progress_bar(root: UiNode) -> bool:
|
||||
"""True if a ProgressBar is present (reader still loading its body -- a fallback signal)."""
|
||||
return any(n.cls.endswith("ProgressBar") for n in root.walk())
|
||||
|
||||
|
||||
@dataclass
|
||||
class MessageRow:
|
||||
"""A tappable message row in the mailbox list."""
|
||||
|
||||
index: int
|
||||
bounds: Tuple[int, int, int, int]
|
||||
center: Tuple[int, int]
|
||||
texts: List[str]
|
||||
cached: bool
|
||||
|
||||
@property
|
||||
def label(self) -> str:
|
||||
"""A short, log-safe identifier -- sender + subject when available."""
|
||||
if not self.texts:
|
||||
return f"row#{self.index}"
|
||||
if len(self.texts) >= 2:
|
||||
return f"{self.texts[0]} / {self.texts[-1]}"
|
||||
return self.texts[0]
|
||||
|
||||
|
||||
def find_message_rows(root: UiNode, package: str) -> List[MessageRow]:
|
||||
"""Return the tappable message rows in the mailbox's scrollable list.
|
||||
|
||||
Rows are the ``clickable`` + ``long-clickable`` children of the scrollable list (as in
|
||||
``ui_mailbox.xml``). A row carrying the ``"Available offline"`` content-desc has its
|
||||
body cached already, so it is marked ``cached`` (callers pick uncached rows for the
|
||||
uncached-open scenarios).
|
||||
"""
|
||||
scrollables = root.find_all(lambda n: n.scrollable and n.package == package)
|
||||
rows: List[MessageRow] = []
|
||||
seen_bounds = set()
|
||||
for scroller in scrollables:
|
||||
for child in scroller.children:
|
||||
if not (child.clickable and child.long_clickable and child.bounds):
|
||||
continue
|
||||
if child.bounds in seen_bounds:
|
||||
continue
|
||||
seen_bounds.add(child.bounds)
|
||||
texts = child.descendant_texts()
|
||||
# Drop single-letter avatar monograms; keep sender/subject/time.
|
||||
texts = [t for t in texts if len(t) > 1]
|
||||
cached = any(d == "Available offline" for d in child.descendant_descs())
|
||||
rows.append(
|
||||
MessageRow(
|
||||
index=len(rows),
|
||||
bounds=child.bounds,
|
||||
center=child.center, # type: ignore[arg-type]
|
||||
texts=texts,
|
||||
cached=cached,
|
||||
)
|
||||
)
|
||||
return rows
|
||||
Reference in New Issue
Block a user