Compare commits

...
Author SHA1 Message Date
JMR-dev 72ff7adfcc Merge branch 'main' into docs/seven-run-counts 2026-08-25 20:25:06 -05:00
Jason Ross 3fd34c24a6 Merge pull request #109 from JMR-dev/test/release-permission-guard
Notice if the release job loses the permission that lets it publish
2026-08-25 15:59:04 -05:00
JMR-dev d01a46a708 Stop counting the run this page calls inconclusive
R29 found the discriminator claimed "exact across all seven" while r07 is recorded lower
down as "inconclusive rather than ruled out, because no evidence came back from it". A row
this page calls inconclusive cannot also be counted as evidence for the conclusion.

Checking it turned up a second instance of the same over-count, which R29 did not name. The
abort-cadence section said "Measured across the seven runs above" -- but the table records
r07's aborts as **not readable**, because adb wedged before a crash buffer could be taken.
Six runs contributed gaps, not seven.

Both now say six, and both say why. The discriminator paragraph also says what excluding r07
costs, which is nothing: it is a `host` row, so the discriminator predicts it would not boot,
and confirming a prediction with the one run whose evidence did not come back adds no
information in either direction. That is the point R29 made -- claiming six does not weaken
the conclusion -- and it is worth stating in the document rather than only in the ticket,
because the next reader will otherwise wonder whether a run was quietly dropped.

Deliberately left: "four of the seven runs show the directory creation itself is broken
during the loop". That is a count of how many runs showed something, not a claim that all
seven were readable for it, so it survives. Checked rather than assumed, and named here so
the next pass does not re-audit it.

R29's other half -- "state how r07's boot outcome was read" -- is not taken, because I do
not know and inventing a source would be worse than narrowing the claim. Narrowing is the
option R29 offered and the one that can be honest.

Closes #38.
2026-08-25 15:54:48 -05:00
JMR-dev 3806641cb2 Merge branch 'main' into test/release-permission-guard 2026-08-25 15:50:20 -05:00
Jason Ross 5f9498150c Merge pull request #106 from JMR-dev/ci/build-workflow-permissions
Declare build.yml's token reach in build.yml
2026-08-25 15:49:19 -05:00
JMR-dev 8d8703ab49 Merge branch 'main' into ci/build-workflow-permissions 2026-08-25 15:41:39 -05:00
Jason Ross 27b7654418 Merge pull request #105 from JMR-dev/docs/api37-point-release
Say 37.0 is the choice, not the only api-level that exists
2026-08-25 15:41:11 -05:00
JMR-dev 4a8e30099e Notice if the release job loses the permission that lets it publish
build.yml's `release` job declares `contents: write`, and nothing checked it. Deleting those
lines leaves actionlint clean and CodeQL silent -- a narrower permission is not an alert --
and the job is `if: startsWith(github.ref, 'refs/tags/v')`, so no pull request and no merge
can exercise it. Measured with the declaration removed: every gating check still passed. The
first thing that would notice is a release failing to publish, at the moment someone is
trying to cut one.

The deletion also looks like tidying. #106 has just put a top-level `permissions: contents:
read` directly above it, so a reader could reasonably take the job-level block for a
duplicate. It is an override, and a comment saying so is not a check.

BackupExclusionsTest is the precedent: configuration rather than code, load bearing, and
unguarded because nothing compiles it.

The part worth reading twice is the second commit-worth of work in here. The test passed,
and then the mutation that is supposed to redden it did not:

  BUILD SUCCESSFUL in 614ms

Gradle cannot infer that a test depends on a file outside the source set, so the task stayed
UP-TO-DATE and the test never ran. Under --rerun-tasks the same mutation failed it properly,
which is the tell: the assertion was right and the wiring was not. A guard that does not
re-run when its subject changes is not a guard -- it is a test that will be green on the day
it matters, which is worse than no test because it reads as cover.

Fixed by declaring the workflow as a task input. Verified the whole way round afterwards,
without --rerun-tasks: mutate the file and the task re-runs and fails; restore it and the
task re-runs and passes.

What this pins and what it does not: it asserts the declaration exists in the release job's
block. It cannot assert a release actually publishes -- that needs a tag push, which is the
thing no PR can do. A tripwire against silent removal, not proof the path works, and the
KDoc says so.

Closes #107.
2026-08-25 15:38:14 -05:00
JMR-dev e7d84cc69f Merge branch 'main' into docs/api37-point-release 2026-08-25 15:33:27 -05:00
Jason Ross a8b494b846 Merge pull request #104 from JMR-dev/docs/benchmark-populate-path
Stop telling people to stage the benchmark the one way it cannot be staged
2026-08-25 15:33:02 -05:00
JMR-dev 865a4a7c8e Merge branch 'main' into docs/benchmark-populate-path 2026-08-25 10:21:23 -05:00
Jason Ross e856679395 Merge pull request #103 from JMR-dev/fix/dead-assertion-probe-test
Delete an assertion that could never fail, and say what guards instead
2026-08-25 10:21:18 -05:00
JMR-dev 49c483d877 Declare build.yml's token reach in build.yml
CodeQL alert #1, the only open one on this repository:

  actions/missing-workflow-permissions, warning / medium, build.yml:23
  Actions job or workflow does not limit the permissions of the GITHUB_TOKEN.

Alerts 2, 3 and 4 were the same rule against status_check.yml and are fixed -- that file
has a top-level block. build.yml declares permissions in exactly one place, the release
job's `contents: write`, and has no top-level default, so the `test` job inherits the
repository setting.

**Nothing is over-privileged today.** The repository default is already `read`
(default_workflow_permissions: read, can_approve_pull_request_reviews: false, read from the
API rather than assumed), so the test job holds a read token now. Saying so matters: this
is hygiene, and a commit that implied it was closing a live hole would be overstating it.

What it buys is that the default CANNOT widen these jobs later without someone editing this
file. That is not invented for the occasion -- it is the argument status_check.yml already
makes, which even names this file:

  the token's reach should be readable here, and a default that widens later should not
  silently widen these jobs with it. build.yml's release job makes the opposite
  declaration for the same reason.

So the principle was decided, applied in two workflows and in one job of this one, and the
top level of build.yml was the gap.

Verified the thing that would actually break: the release job's `contents: write` still
wins. Top level is a default, not a ceiling -- parsed and printed both, test inherits
`contents: read`, release keeps `contents: write`.

Also ran the ticket's mutation, and it found something. Deleting the release job's
`contents: write` leaves actionlint green and CodeQL quiet -- a narrower permission is not
an alert -- so nothing would catch it until a tagged release failed to publish. That is a
separate gap and is filed rather than fixed here.

actionlint clean at the pinned digest. Comment and permissions only; no step, job or
trigger changes.

Closes #100.
2026-08-25 10:16:08 -05:00
JMR-dev 1b220856ab Say 37.0 is the choice, not the only api-level that exists
R19 raised two things about this comment. One resolved itself: it used to explain why the
matrix had no API 37 row at all, and #56 added the gating row, so that half is gone.

The other survived, and this is it. The comment read

  api-level must be "37.0". A bare 37 is not an SDK package and fails during setup

The second sentence is true and was measured -- it cost a run to find. The first overstates
it. What must be true is that the api-level is a POINT release; 37.0 is one of several.
api37-debug.yml's own input descriptions already say so:

  API level, as the SDK spells it. 37.0, 37.1, 37.2-beta3, 36 ...
  System image target. android-37.1 and 37.2-beta* ship ONLY as google_apis_ps16k

and docs/api-37-emulator-crash.md measures android-37.0 rev 6 and android-37.1 rev 8 side
by side, both aborting. So the repo already knows 37.1 exists and behaves the same; only
this comment implied otherwise.

That matters for the reader it is written for. Someone debugging this row and wondering
whether a newer image helps reads "must be 37.0" as a constraint and stops. The measured
answer is that it does not help, which is a better thing to learn than a rule that is not
one -- and the ps16k-only wrinkle above 37.0 is the detail that would actually bite them.

Comment only. No job, matrix, filter or gating behaviour changes. actionlint clean at the
pinned digest.

Closes #28.
2026-08-25 10:14:34 -05:00
JMR-dev d37c391c60 Stop telling people to stage the benchmark the one way it cannot be staged
RealMediaBenchmark's class KDoc said:

  Populate with:
    adb push <file>.mp4 /sdcard/Android/data/org.libremediaconverter/files/

Twelve lines below, the `samples` property KDoc -- on `get() = context.filesDir` -- says:

  Internal storage, not the external files dir. Files placed in the external dir by
  `adb push` or `adb shell cp` stay owned by the shell user, and the app then gets
  EACCES trying to read them -- which presents as an unparseable input rather than a
  permission problem.

Different directories, and the second exists specifically to explain why the first fails.
Anyone following the class KDoc stages files the benchmark cannot read, gets a skip, and
reads the skip as "not staged yet" -- the failure mode the property KDoc warns about, walked
into by the instruction in the same file.

The fix is not a corrected command. Restating the mechanism in a second place is what let
these drift, and a replacement command I have not executed would be the same defect with a
fresher date. The class KDoc now names [samples] as the single place that answers it.

Two things added that are checkable rather than remembered: the exact filenames the tests
look for, via [H264_SAMPLE] and [AV1_SAMPLE] -- the old text said `<file>.mp4`, so even the
right directory left you guessing -- and a note that the two skips every green E2E leg
reports are these.

Not claimed: that the benchmark misbehaves on CI. An earlier version of the ticket said so;
it was wrong, and measuring settled it -- both tests report SKIPPED on the gating legs, the
guards work, and "harmless in CI" is accurate. The failure that prompted the look is
Media3EngineTest, tracked as #102.

Closes #101.
2026-08-25 09:45:00 -05:00
6 changed files with 119 additions and 8 deletions
+11
View File
@@ -13,6 +13,17 @@ on:
# reference amounts to running whatever that repository contains tomorrow. This matters
# more here than on pull requests: these jobs sign nothing today, but they do publish
# the artifacts people install.
# Declared here rather than inherited, for the reason status_check.yml gives for its own
# block: the token's reach should be readable in the file that uses it, and a repository
# default that widens later should not silently widen these jobs with it. The repository
# default is `read` today, so this changes nothing about what runs -- it fixes what a
# reader can know without leaving the file, and it is what CodeQL alert #1 asked for.
#
# The `release` job below overrides this with `contents: write`, which is how job-level
# permissions work: this is a default, not a ceiling.
permissions:
contents: read
env:
GRADLE_CACHE_PATHS: |
~/.gradle/caches
+7 -2
View File
@@ -280,8 +280,13 @@ jobs:
# docs/api-37-emulator-crash.md has the per-method measurements, and the
# correction that produced them.
#
# api-level must be "37.0". A bare 37 is not an SDK package and fails
# during setup, which cost a run to discover.
# api-level must be a POINT release. A bare 37 is not an SDK package and
# fails during setup, which cost a run to discover. `37.0` is the choice
# here rather than the only option: `37.1` and `37.2-beta*` exist and
# abort the same way, and api37-debug.yml's inputs document both, with
# the wrinkle that above 37.0 they ship only as google_apis_ps16k.
# docs/api-37-emulator-crash.md measures 37.0 rev 6 and 37.1 rev 8 side
# by side, so pinning 37.0 is a decision, not a constraint.
#
# notAnnotation removes the three tests that do not pass on this image; they
# run in the advisory job below, off the same marker so they cannot end up
+11
View File
@@ -1,3 +1,4 @@
import org.gradle.api.tasks.PathSensitivity
import org.gradle.testing.jacoco.tasks.JacocoReport
plugins {
@@ -206,6 +207,16 @@ detekt {
// `excludes` is not optional. Without it JaCoCo walks JDK-internal classes that Robolectric has
// no location for either, and the test JVM dies rather than reporting a number.
tasks.withType<Test>().configureEach {
// ReleasePermissionTest reads .github/workflows/build.yml, and Gradle cannot infer that a
// test depends on a file outside the source set. Without this the task stays UP-TO-DATE
// when the workflow changes, so the guard goes stale exactly when it matters. Measured:
// deleting the release job's `contents: write` and re-running gave "BUILD SUCCESSFUL in
// 614ms" with the test never executing; the same mutation under --rerun-tasks failed it.
// A guard that does not re-run when its subject changes is not a guard.
inputs.file(rootProject.file(".github/workflows/build.yml"))
.withPropertyName("releaseWorkflow")
.withPathSensitivity(PathSensitivity.RELATIVE)
extensions.configure<JacocoTaskExtension> {
isIncludeNoLocationClasses = true
excludes = listOf("jdk.internal.*")
@@ -36,8 +36,20 @@ import java.io.File
* 1. that the hardware path is worth having a second engine for at all, and
* 2. that x264's CRF is worth the GPL licence the app carries for it.
*
* Skips itself when the sample files are absent, so it is harmless in CI. Populate with:
* adb push <file>.mp4 /sdcard/Android/data/org.libremediaconverter/files/
* Skips itself when the sample files are absent, so it is harmless in CI — every green E2E
* leg reports two skips, and these are they.
*
* The two files it looks for, by exact name:
*
* - [H264_SAMPLE] for [hardwareVersusSoftwareOnRealVideo]
* - [AV1_SAMPLE] for [av1InputRoutesAccordingToDeviceDecodeSupport]
*
* **Where they go, and how, is on [samples] — read it before staging anything.** This used to
* carry an `adb push` line naming the external files dir, which [samples] then explains cannot
* work: a pushed file stays owned by the shell user and the app reads EACCES, surfacing as an
* unparseable input rather than a permission error. The instruction and its own refutation sat
* twelve lines apart. It is named in one place now rather than restated here, because restating
* it is what let the two drift.
*/
@UnstableApi
@RunWith(AndroidJUnit4::class)
@@ -0,0 +1,67 @@
package org.libremediaconverter.ci
import org.junit.Assert.assertTrue
import org.junit.Test
import java.io.File
/**
* That the release job still holds the one permission it needs to publish.
*
* `build.yml`'s `release` job declares `contents: write`, and nothing was checking it. Deleting
* those two lines leaves actionlint clean and CodeQL silent — a *narrower* permission is not an
* alert — and the job is `if: startsWith(github.ref, 'refs/tags/v')`, so no pull request and no
* merge to `main` can exercise it. Measured: with the declaration removed, every gating check
* still passes. The first thing that would notice is a release failing to publish, at the moment
* someone is trying to cut one.
*
* The deletion also looks like tidying. A top-level `permissions: contents: read` now sits
* directly above it, so a reader could reasonably take the job-level block for a duplicate. It is
* an override, not a duplicate, and a comment saying so is not a check.
*
* `BackupExclusionsTest` is the precedent: a file that is configuration rather than code, load
* bearing, and unguarded because nothing compiles it.
*
* **What this pins, and what it does not.** It asserts the declaration exists in the `release`
* job's block. It cannot assert that a release actually publishes — that needs a tag push, which
* is the thing no PR can do. So this is a tripwire against silent removal, not proof the release
* path works.
*/
class ReleasePermissionTest {
@Test
fun `the release job declares the write permission it needs to publish`() {
val release = jobBlock("release")
assertTrue(
"build.yml's `release` job no longer declares `contents: write`. It is the only " +
"permission that lets the job create a release, the top-level block above it is " +
"`contents: read`, and nothing else in CI would catch this until a tag failed to " +
"publish. If the release moved elsewhere, delete this test deliberately.",
release.any { it.trimStart().startsWith("contents: write") },
)
}
/**
* The lines of one top-level job, from its ` <name>:` header to the next job at that indent.
*
* Line-based rather than parsed: the module has no YAML dependency, and adding one to read two
* lines would be a worse trade than a scan that fails loudly when the shape changes.
*/
private fun jobBlock(name: String): List<String> {
val lines = workflow.readLines()
val start = lines.indexOfFirst { it == " $name:" }
check(start >= 0) { "no ` $name:` job in ${workflow.path} — has the file been restructured?" }
val rest = lines.drop(start + 1)
val end = rest.indexOfFirst { it.matches(Regex("^ {2}[A-Za-z0-9_-]+:.*")) }
return if (end < 0) rest else rest.take(end)
}
/**
* Found by walking up rather than by a fixed relative path: Gradle's working directory for the
* unit tests is the module, but that is a default rather than a promise.
*/
private val workflow: File
get() = generateSequence(File(".").absoluteFile) { it.parentFile }
.map { File(it, ".github/workflows/build.yml") }
.firstOrNull { it.isFile }
?: error("could not find .github/workflows/build.yml above ${File(".").absolutePath}")
}
+9 -4
View File
@@ -32,8 +32,12 @@ reached* is not. Re-measured on 2026-08-22, seven runs, one variable at a time:
| r06 | `android-37.1` rev 8 | `swangle_indirect` | ANGLE | **yes, 285 s** | 23 |
| r07 | `android-37.0` rev 6 | `host` + `-feature -HostComposition` | host | **no**, wedged adb at 208 s | not readable |
The discriminator is exact across all seven: **a run boots if and only if the emulator log says
something other than `gles_mode_selected:host`.**
The discriminator is exact across the **six runs that reported**: a run boots if and only if the
emulator log says something other than `gles_mode_selected:host`. r07 is excluded on purpose — it
wedged adb at 208 s and is recorded below as inconclusive rather than ruled out, and a row this
page calls inconclusive cannot also be counted as evidence. Excluding it costs nothing: r07 is a
`host` row, so the discriminator predicts it would not boot, and confirming a prediction with the
one run whose evidence did not come back would add no information either way.
One caveat about how independent those rows are, because the table flatters itself. `-gpu
angle_indirect` (r05) and `-gpu swangle_indirect` (r03) both logged `gles_mode_selected:swangle`
@@ -509,8 +513,9 @@ API 37", not "is that codec broken".
`.github/workflows/api37-debug.yml` carried "roughly every 20 s" for the kill cycle in its own
comments. That number was the watchdog's **sampling** interval, not the cadence, and the two got
conflated. Measured across the seven runs above, gaps between successive `hasReadColorBufferDma`
aborts run **20 s to 90 s, median 60–70 s — three to five aborts in a four-minute window**.
conflated. Measured across the **six runs whose crash buffer could be read** — r07 wedged adb
before one could be taken, so it contributes no gaps — successive `hasReadColorBufferDma` aborts
run **20 s to 90 s, median 60–70 s — three to five aborts in a four-minute window**.
Slower than assumed, and still not slow enough: install, data-directory creation and
instrumentation start-up do not fit inside one gap.