Compare commits

..
Author SHA1 Message Date
JMR-dev 72ff7adfcc Merge branch 'main' into docs/seven-run-counts 2026-08-25 20:25:06 -05:00
Jason Ross 3fd34c24a6 Merge pull request #109 from JMR-dev/test/release-permission-guard
Notice if the release job loses the permission that lets it publish
2026-08-25 15:59:04 -05:00
JMR-dev d01a46a708 Stop counting the run this page calls inconclusive
R29 found the discriminator claimed "exact across all seven" while r07 is recorded lower
down as "inconclusive rather than ruled out, because no evidence came back from it". A row
this page calls inconclusive cannot also be counted as evidence for the conclusion.

Checking it turned up a second instance of the same over-count, which R29 did not name. The
abort-cadence section said "Measured across the seven runs above" -- but the table records
r07's aborts as **not readable**, because adb wedged before a crash buffer could be taken.
Six runs contributed gaps, not seven.

Both now say six, and both say why. The discriminator paragraph also says what excluding r07
costs, which is nothing: it is a `host` row, so the discriminator predicts it would not boot,
and confirming a prediction with the one run whose evidence did not come back adds no
information in either direction. That is the point R29 made -- claiming six does not weaken
the conclusion -- and it is worth stating in the document rather than only in the ticket,
because the next reader will otherwise wonder whether a run was quietly dropped.

Deliberately left: "four of the seven runs show the directory creation itself is broken
during the loop". That is a count of how many runs showed something, not a claim that all
seven were readable for it, so it survives. Checked rather than assumed, and named here so
the next pass does not re-audit it.

R29's other half -- "state how r07's boot outcome was read" -- is not taken, because I do
not know and inventing a source would be worse than narrowing the claim. Narrowing is the
option R29 offered and the one that can be honest.

Closes #38.
2026-08-25 15:54:48 -05:00
JMR-dev 3806641cb2 Merge branch 'main' into test/release-permission-guard 2026-08-25 15:50:20 -05:00
Jason Ross 5f9498150c Merge pull request #106 from JMR-dev/ci/build-workflow-permissions
Declare build.yml's token reach in build.yml
2026-08-25 15:49:19 -05:00
JMR-dev 4a8e30099e Notice if the release job loses the permission that lets it publish
build.yml's `release` job declares `contents: write`, and nothing checked it. Deleting those
lines leaves actionlint clean and CodeQL silent -- a narrower permission is not an alert --
and the job is `if: startsWith(github.ref, 'refs/tags/v')`, so no pull request and no merge
can exercise it. Measured with the declaration removed: every gating check still passed. The
first thing that would notice is a release failing to publish, at the moment someone is
trying to cut one.

The deletion also looks like tidying. #106 has just put a top-level `permissions: contents:
read` directly above it, so a reader could reasonably take the job-level block for a
duplicate. It is an override, and a comment saying so is not a check.

BackupExclusionsTest is the precedent: configuration rather than code, load bearing, and
unguarded because nothing compiles it.

The part worth reading twice is the second commit-worth of work in here. The test passed,
and then the mutation that is supposed to redden it did not:

  BUILD SUCCESSFUL in 614ms

Gradle cannot infer that a test depends on a file outside the source set, so the task stayed
UP-TO-DATE and the test never ran. Under --rerun-tasks the same mutation failed it properly,
which is the tell: the assertion was right and the wiring was not. A guard that does not
re-run when its subject changes is not a guard -- it is a test that will be green on the day
it matters, which is worse than no test because it reads as cover.

Fixed by declaring the workflow as a task input. Verified the whole way round afterwards,
without --rerun-tasks: mutate the file and the task re-runs and fails; restore it and the
task re-runs and passes.

What this pins and what it does not: it asserts the declaration exists in the release job's
block. It cannot assert a release actually publishes -- that needs a tag push, which is the
thing no PR can do. A tripwire against silent removal, not proof the path works, and the
KDoc says so.

Closes #107.
2026-08-25 15:38:14 -05:00
3 changed files with 87 additions and 4 deletions
+11
View File
@@ -1,3 +1,4 @@
import org.gradle.api.tasks.PathSensitivity
import org.gradle.testing.jacoco.tasks.JacocoReport
plugins {
@@ -206,6 +207,16 @@ detekt {
// `excludes` is not optional. Without it JaCoCo walks JDK-internal classes that Robolectric has
// no location for either, and the test JVM dies rather than reporting a number.
tasks.withType<Test>().configureEach {
// ReleasePermissionTest reads .github/workflows/build.yml, and Gradle cannot infer that a
// test depends on a file outside the source set. Without this the task stays UP-TO-DATE
// when the workflow changes, so the guard goes stale exactly when it matters. Measured:
// deleting the release job's `contents: write` and re-running gave "BUILD SUCCESSFUL in
// 614ms" with the test never executing; the same mutation under --rerun-tasks failed it.
// A guard that does not re-run when its subject changes is not a guard.
inputs.file(rootProject.file(".github/workflows/build.yml"))
.withPropertyName("releaseWorkflow")
.withPathSensitivity(PathSensitivity.RELATIVE)
extensions.configure<JacocoTaskExtension> {
isIncludeNoLocationClasses = true
excludes = listOf("jdk.internal.*")
@@ -0,0 +1,67 @@
package org.libremediaconverter.ci
import org.junit.Assert.assertTrue
import org.junit.Test
import java.io.File
/**
* That the release job still holds the one permission it needs to publish.
*
* `build.yml`'s `release` job declares `contents: write`, and nothing was checking it. Deleting
* those two lines leaves actionlint clean and CodeQL silent — a *narrower* permission is not an
* alert — and the job is `if: startsWith(github.ref, 'refs/tags/v')`, so no pull request and no
* merge to `main` can exercise it. Measured: with the declaration removed, every gating check
* still passes. The first thing that would notice is a release failing to publish, at the moment
* someone is trying to cut one.
*
* The deletion also looks like tidying. A top-level `permissions: contents: read` now sits
* directly above it, so a reader could reasonably take the job-level block for a duplicate. It is
* an override, not a duplicate, and a comment saying so is not a check.
*
* `BackupExclusionsTest` is the precedent: a file that is configuration rather than code, load
* bearing, and unguarded because nothing compiles it.
*
* **What this pins, and what it does not.** It asserts the declaration exists in the `release`
* job's block. It cannot assert that a release actually publishes — that needs a tag push, which
* is the thing no PR can do. So this is a tripwire against silent removal, not proof the release
* path works.
*/
class ReleasePermissionTest {
@Test
fun `the release job declares the write permission it needs to publish`() {
val release = jobBlock("release")
assertTrue(
"build.yml's `release` job no longer declares `contents: write`. It is the only " +
"permission that lets the job create a release, the top-level block above it is " +
"`contents: read`, and nothing else in CI would catch this until a tag failed to " +
"publish. If the release moved elsewhere, delete this test deliberately.",
release.any { it.trimStart().startsWith("contents: write") },
)
}
/**
* The lines of one top-level job, from its ` <name>:` header to the next job at that indent.
*
* Line-based rather than parsed: the module has no YAML dependency, and adding one to read two
* lines would be a worse trade than a scan that fails loudly when the shape changes.
*/
private fun jobBlock(name: String): List<String> {
val lines = workflow.readLines()
val start = lines.indexOfFirst { it == " $name:" }
check(start >= 0) { "no ` $name:` job in ${workflow.path} — has the file been restructured?" }
val rest = lines.drop(start + 1)
val end = rest.indexOfFirst { it.matches(Regex("^ {2}[A-Za-z0-9_-]+:.*")) }
return if (end < 0) rest else rest.take(end)
}
/**
* Found by walking up rather than by a fixed relative path: Gradle's working directory for the
* unit tests is the module, but that is a default rather than a promise.
*/
private val workflow: File
get() = generateSequence(File(".").absoluteFile) { it.parentFile }
.map { File(it, ".github/workflows/build.yml") }
.firstOrNull { it.isFile }
?: error("could not find .github/workflows/build.yml above ${File(".").absolutePath}")
}
+9 -4
View File
@@ -32,8 +32,12 @@ reached* is not. Re-measured on 2026-08-22, seven runs, one variable at a time:
| r06 | `android-37.1` rev 8 | `swangle_indirect` | ANGLE | **yes, 285 s** | 23 |
| r07 | `android-37.0` rev 6 | `host` + `-feature -HostComposition` | host | **no**, wedged adb at 208 s | not readable |
The discriminator is exact across all seven: **a run boots if and only if the emulator log says
something other than `gles_mode_selected:host`.**
The discriminator is exact across the **six runs that reported**: a run boots if and only if the
emulator log says something other than `gles_mode_selected:host`. r07 is excluded on purpose — it
wedged adb at 208 s and is recorded below as inconclusive rather than ruled out, and a row this
page calls inconclusive cannot also be counted as evidence. Excluding it costs nothing: r07 is a
`host` row, so the discriminator predicts it would not boot, and confirming a prediction with the
one run whose evidence did not come back would add no information either way.
One caveat about how independent those rows are, because the table flatters itself. `-gpu
angle_indirect` (r05) and `-gpu swangle_indirect` (r03) both logged `gles_mode_selected:swangle`
@@ -509,8 +513,9 @@ API 37", not "is that codec broken".
`.github/workflows/api37-debug.yml` carried "roughly every 20 s" for the kill cycle in its own
comments. That number was the watchdog's **sampling** interval, not the cadence, and the two got
conflated. Measured across the seven runs above, gaps between successive `hasReadColorBufferDma`
aborts run **20 s to 90 s, median 60–70 s — three to five aborts in a four-minute window**.
conflated. Measured across the **six runs whose crash buffer could be read** — r07 wedged adb
before one could be taken, so it contributes no gaps — successive `hasReadColorBufferDma` aborts
run **20 s to 90 s, median 60–70 s — three to five aborts in a four-minute window**.
Slower than assumed, and still not slow enough: install, data-directory creation and
instrumentation start-up do not fit inside one gap.