JMR-dev/LibreMediaConverter
1
0
Fork 0
Code Issues 16 Pull Requests Actions Packages Projects Releases Wiki Activity

Commit the FFmpeg binary so test runs stop depending on a rebuild

Browse Source 
CI rebuilt FFmpeg on every cold cache, which made results ambiguous: a red run
could mean the code was broken or that a forty-minute cross-compile of FFmpeg,
x264, x265 and SVT-AV1 had hiccuped. Those are not the same signal, and only
one of them is worth a developer's attention. The archive is now checked in
under bin/, so a failing run points at code.

It also removes roughly forty minutes from a cold run and lets a fresh clone
build without a container toolchain.

bin/README.md records provenance -- upstream tag, FFmpeg version, NDK, ABIs,
SHA-256 and the full configure line read back out of the shipped libavutil --
so the binary is auditable rather than opaque. The recipe in tools/ffmpeg
remains the authority: this archive is its output, and is also what satisfies
the GPL corresponding-source obligation.

The status check is now seven independent runners: one validating the archive,
one for the JVM tests, and one per API level from 33 to 37. The FFmpeg job
verifies rather than builds. It asserts native libraries are present for both
ABIs and that every one is 16 KB aligned, which is a Play requirement that is
easy to lose in a rebuild and expensive to discover at submission. Checking
for file existence alone would not do: a Git LFS pointer checked out without
LFS passes that and then surfaces as an obscure linker error much later.

It is a separate job rather than a step in each emulator run so a bad archive
reports once, clearly, instead of five confusing emulator failures.

build.yml no longer builds FFmpeg either, and keeps only its post-merge and
release duties.

Two costs, deliberately accepted. The repository goes from about 1 MB to
35 MB, and every future rebuild adds another 35 MB blob to history
permanently, so bin/README.md says to regenerate only when the FFmpeg version
or the configure flags actually change. And F-Droid's scanner flags checked-in
native libraries, so submitting there needs a scandelete entry for bin/ --
noted in bin/README.md, and nothing prevents a from-source build.

Verified against the relocated archive: 66 unit tests, and 40 instrumented
tests on an API 36 emulator, 0 failures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
JMR-dev
2026-08-20 13:18:10 -05:00
co-authored by Claude Opus 5
parent 54d6e9c57b
commit 128763e99c
8 changed files with 152 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/status_check.yml
+51 -72
View File
@@ -4,93 +4,70 @@ on:
pull_request:
branches: [main]
# A newer push to the same PR makes the in-flight run obsolete. Emulator matrices are
# expensive, so cancel rather than let them pile up.
# A newer push to the same PR makes the in-flight run obsolete. An emulator matrix is
# expensive, so cancel rather than let runs pile up.
concurrency:
group: status-check-${{ github.ref }}
cancel-in-progress: true
env:
# Must match the coordinate app/build.gradle.kts loads from app/libs/.
FFMPEG_AAR: ffmpeg-kit-next-8.1.1.aar
jobs:
# ---------------------------------------------------------------------------
# FFmpeg is not committed: the AAR is ~35 MB of native code, and F-Droid strips
# checked-in binaries. Every other job needs it, because the app module compiles
# against it and the instrumented tests exercise it for real.
# Validates the committed FFmpeg archive. It does not build anything: the whole
# point of checking the binary in is that a red run means broken code rather than
# a cross-compile that hiccuped.
#
# Building it is a full cross-compile of FFmpeg, x264, x265 and SVT-AV1, so it is
# cached on the contents of tools/ffmpeg. That directory pins the upstream tag and
# the configure flags, which is exactly what determines the output.
# Its own job so a bad archive reports once, clearly, instead of surfacing as five
# confusing emulator failures. It takes seconds, so gating the matrix on it costs
# almost nothing.
# ---------------------------------------------------------------------------
ffmpeg:
name: FFmpeg AAR
name: FFmpeg binary
runs-on: ubuntu-latest
timeout-minutes: 120
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Restore cached AAR
id: cache
uses: actions/cache@v4
with:
path: tools/ffmpeg/out
key: ffmpeg-aar-${{ hashFiles('tools/ffmpeg/Containerfile', 'tools/ffmpeg/build-ffmpeg.sh') }}
# The Nix store plus the build tree runs to several gigabytes, which does not fit
# alongside the runner's preinstalled toolchains.
- name: Free disk space
if: steps.cache.outputs.cache-hit != 'true'
- name: Verify the committed archive
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /usr/local/share/boost
sudo docker image prune -af || true
df -h /
AAR=bin/ffmpeg-kit-next-8.1.1.aar
test -f "$AAR" || { echo "::error::$AAR is missing"; exit 1; }
- name: Build the AAR
if: steps.cache.outputs.cache-hit != 'true'
working-directory: tools/ffmpeg
run: |
mkdir -p out
docker build -t ffmpeg-kit-builder:ci -f Containerfile .
docker run --rm -v "$PWD/out":/work/out ffmpeg-kit-builder:ci full
- name: Check the AAR is present and plausibly complete
run: |
AAR=$(find tools/ffmpeg/out -name 'ffmpeg-kit-next*.aar' | head -1)
test -n "$AAR" || { echo "::error::no AAR produced"; exit 1; }
# A truncated or stub AAR would still satisfy `test -f`, so check it carries
# native libraries for both ABIs before letting the matrix depend on it.
# A truncated file, or a Git LFS pointer checked out without LFS, would pass
# a file-exists check and then surface much later as a confusing linker
# error. Assert the archive actually carries native libraries for both ABIs.
for abi in arm64-v8a x86_64; do
n=$(unzip -l "$AAR" | grep -c "jni/$abi/.*\.so$" || true)
echo " $abi: $n shared libraries"
test "$n" -gt 0 || { echo "::error::AAR has no $abi libraries"; exit 1; }
done
cp "$AAR" "${{ env.FFMPEG_AAR }}"
- uses: actions/upload-artifact@v4
with:
name: ffmpeg-aar
path: ${{ env.FFMPEG_AAR }}
retention-days: 1
# 16 KB alignment is a Play requirement and is easy to lose in a rebuild,
# so it is checked here rather than discovered at submission.
unzip -q -o "$AAR" 'jni/*' -d /tmp/aarcheck
bad=0
for f in /tmp/aarcheck/jni/*/*.so; do
align=$(readelf -lW "$f" | awk '$1=="LOAD"{print $NF}' | sort -u)
if [ "$align" != "0x4000" ]; then
echo "::error::$(basename "$f") is $align, not 16 KB aligned"; bad=1
fi
done
test "$bad" -eq 0 || exit 1
echo " all libraries are 16 KB aligned"
# Record what shipped, so a failing run elsewhere can be tied to a version.
echo " sha256: $(sha256sum "$AAR" | cut -d' ' -f1)"
# ---------------------------------------------------------------------------
# JVM tests: the routing matrix, the FFmpeg argument builder, the concat planner
# and the retry rule. These need no device and are the fastest signal on a PR.
# and the retry rule. No device needed, so this is the fastest signal on a PR.
# ---------------------------------------------------------------------------
unit:
name: Unit tests
runs-on: ubuntu-latest
needs: ffmpeg
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
name: ffmpeg-aar
path: app/libs/
- uses: actions/setup-java@v4
with:
distribution: temurin
@@ -98,7 +75,8 @@ jobs:
- uses: gradle/actions/setup-gradle@v4
- run: ./gradlew :app:testDebugUnitTest
- name: Unit tests
run: ./gradlew :app:testDebugUnitTest
- uses: actions/upload-artifact@v4
if: always()
@@ -107,18 +85,24 @@ jobs:
path: app/build/reports/tests/
# ---------------------------------------------------------------------------
# Instrumented tests across the whole supported range. minSdk is 33 and targetSdk
# is 37, and the foreground-service type differs across that range -- none below
# 34, dataSync at 34, mediaProcessing from 35 -- so a single API level would leave
# two thirds of that branch unexercised.
# One runner per API level, across the whole supported range.
#
# The range is the point: minSdk is 33 and targetSdk is 37, and the foreground
# service type differs across it -- none below 34, dataSync at 34, mediaProcessing
# from 35. Testing a single level would leave two thirds of that branch unexercised,
# and running the range locally is what caught a test that had baked in an
# assumption about the host's encoders.
#
# FFmpeg is not built here. The AAR is committed under bin/, so a red run means the
# code is broken rather than that a cross-compile hiccuped.
# ---------------------------------------------------------------------------
instrumented:
e2e:
name: E2E API ${{ matrix.api-level }}
runs-on: ubuntu-latest
needs: ffmpeg
timeout-minutes: 60
strategy:
# Report every API level rather than stopping at the first red one: knowing
# Report every API level rather than stopping at the first red one. Knowing
# whether a failure is universal or specific to one level is most of the
# diagnosis.
fail-fast: false
@@ -132,18 +116,13 @@ jobs:
system-image-api-level: 35
- api-level: 36
system-image-api-level: 36
# API 37 is published as android-37.0, not android-37, so the image level
# has to be given separately or the download resolves to nothing.
# API 37 is published as android-37.0, not android-37, so the image level has
# to be given separately or the download resolves to no package at all.
- api-level: 37
system-image-api-level: "37.0"
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
name: ffmpeg-aar
path: app/libs/
- uses: actions/setup-java@v4
with:
distribution: temurin
@@ -168,9 +147,9 @@ jobs:
target: google_apis
arch: x86_64
profile: pixel_6
# -gpu swiftshader_indirect is the usual CI choice. It is correct here only
# because runners have no GPU to pass through; on a workstation the same
# setting routes through SwiftShader's JIT, which is a known crash source.
# swiftshader_indirect is correct here only because runners have no GPU to
# pass through. On a workstation the same setting routes through
# SwiftShader's JIT, which is a known crash source.
emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
disable-animations: true
script: ./gradlew :app:connectedDebugAndroidTest