feat(scripts): device-testing perf harness
Add a cross-platform, standard-library-only Python package under scripts/device-testing/ that replicates LibreMail's on-device performance-test scenarios and logging capture, codifying the methodology run by hand on 2026-07-05 (Pixel 10 Pro XL). Modules: - breadcrumbs.py: a pure, unit-tested parser for the ImapPerf / MailReader / Reader / MailBackfiller breadcrumbs, plus open-correlation that reproduces the manual timing-tables.md figures exactly. - adb.py: a safety-guarded adb wrapper -- an allow-list of adb subcommands and a deny-list + assertions on shell commands. The only sanctioned app-state mutation is clearing LibreMail's own cache/ (exact-match); no pm clear / uninstall / data wipe, and no touching databases/ files/ shared_prefs/ datastore/ can be constructed. - uidump.py: uiautomator XML parser + screen recognition (mailbox rows with the cached "Available offline" flag, reader, and the keyguard / foreign-app guards). - scenarios.py: cold-open, message-open (uncached), back-nav, prefetch A/B (fetch-policy toggle) and cross-provider, each keyguard-guarded and driven through the guarded wrapper. - report.py + perf_harness.py: aggregates, a timing-tables.md renderer mirroring the manual write-up, and the CLI (timestamped run dir with the raw logcat, a filtered breadcrumb extract, and the tables). --dry-run prints the exact command plan without touching device state. Tests (stdlib unittest, 68 cases) validate the parser, guardrails, UI recognition and report against the manual run's real captures (fixtures include a verbatim perf-extract slice and the reader/lockscreen/alarm dumps). Track the *.log fixture past the gitignore *.log rule via a scoped negation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,17 @@
|
||||
07-05 17:55:38.343 10261 10423 D ImapPerf: prefetch-body connect=857ms work=2150ms live=3
|
||||
07-05 17:55:54.352 10261 10425 D ImapPerf: backfill-page connect=4366ms work=71989ms live=2
|
||||
07-05 17:57:38.981 10261 10425 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=false took=21ms
|
||||
07-05 17:57:39.041 10261 10261 D Reader : reader ready took=82ms html=true inline=0
|
||||
07-05 18:04:37.001 10261 10408 I MailBackfiller: backfill slice done: moreWork=true
|
||||
07-05 18:04:37.002 10261 10408 I MailBackfiller: backfill slice: maxBatches=20
|
||||
07-05 18:07:16.094 10261 15306 D ImapPerf: body-fetch select=2566ms body=14253ms flag=2356ms rfc822=60457B chars=51610 att=0
|
||||
07-05 18:07:18.435 10261 15306 D ImapPerf: body-fetch connect=2624ms work=26211ms live=3
|
||||
07-05 18:07:20.817 10261 15306 I MailReader: openMessage imap:94058a folder=INBOX fetchedBody=true took=31227ms
|
||||
07-05 18:07:21.206 10261 10261 D Reader : reader ready took=31617ms html=true inline=0
|
||||
07-05 18:07:32.607 10261 10414 D ImapPerf: backfill-page connect=656ms work=3038ms live=3
|
||||
07-05 18:36:37.959 19255 21059 D MailBackfiller: backfill outlook:6b54d6 folder=INBOX pages=20 complete=false
|
||||
07-05 18:36:54.941 19255 19314 D ImapPerf: body-fetch select=185ms body=1355ms flag=78ms rfc822=189086B chars=82986 att=0
|
||||
07-05 18:36:55.002 19255 19314 D ImapPerf: body-fetch connect=772ms work=2018ms live=2
|
||||
07-05 18:36:55.140 19255 21060 I MailReader: openMessage outlook:6b54d6 folder=INBOX fetchedBody=true took=2934ms
|
||||
07-05 18:36:55.144 19255 19255 D Reader : reader ready took=2939ms html=true inline=0
|
||||
07-05 18:37:11.704 19255 19314 D ImapPerf: imap connect=3156ms work=8032ms live=2
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><hierarchy rotation="0"><node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="0" hint=""><node index="0" text="" resource-id="" class="android.widget.LinearLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="2" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/action_bar_root" class="android.widget.LinearLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="android:id/content" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="2" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/content" class="android.view.ViewGroup" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1344,2992]" drawing-order="1" hint=""><node index="0" text="" resource-id="com.google.android.deskclock:id/digital_clock" class="android.widget.FrameLayout" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1047][1290,1586]" drawing-order="18" hint=""><node index="0" text="18:00" resource-id="com.google.android.deskclock:id/auto_sizing_text_clock" class="android.widget.TextView" package="com.google.android.deskclock" content-desc="18:00" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1047][1290,1586]" drawing-order="1" hint="" /></node><node index="1" text="Bae check in" resource-id="com.google.android.deskclock:id/title" class="android.widget.TextView" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[54,1646][1290,1960]" drawing-order="19" hint="" /><node index="2" text="Snooze" resource-id="com.google.android.deskclock:id/button_tap_snooze" class="android.widget.Button" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[126,2274][669,2562]" drawing-order="39" hint="" /><node index="3" text="Stop" resource-id="com.google.android.deskclock:id/button_tap_dismiss" class="android.widget.Button" package="com.google.android.deskclock" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[675,2274][1218,2562]" drawing-order="40" hint="" /></node></node></node></node></node></node></hierarchy>
|
||||
@@ -0,0 +1 @@
|
||||
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><hierarchy rotation="0"><node index="0" text="" resource-id="com.android.systemui:id/legacy_window_root" class="android.widget.FrameLayout" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[0,0][1344,2992]"><node index="0" text="" resource-id="com.android.systemui:id/keyguard_root_view" class="android.view.ViewGroup" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[0,0][1344,2992]"><node index="0" text="" resource-id="com.android.systemui:id/accessibility_actions_view" class="android.view.View" package="com.android.systemui" content-desc="Lock screen" clickable="false" enabled="true" long-clickable="true" scrollable="false" bounds="[0,216][1344,2992]" /><node index="1" text="18 02" resource-id="" class="android.widget.TextView" package="com.android.systemui" content-desc="18:02" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[100,454][583,620]" /><node index="2" text="" resource-id="com.android.systemui:id/device_entry_icon_fg" class="android.widget.ImageView" package="com.android.systemui" content-desc="Fingerprint sensor" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[561,2126][791,2356]" /><node index="3" text="Charged" resource-id="com.android.systemui:id/keyguard_indication_text" class="android.widget.TextView" package="com.android.systemui" content-desc="" clickable="false" enabled="true" long-clickable="false" scrollable="false" bounds="[16,2860][1344,2925]" /></node></node></hierarchy>
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user