Compare commits

..
Author SHA1 Message Date
JMR-devandClaude Opus 5 98c0e4dba2 Cancel a running FFmpeg session, which nothing had ever done (#224)
Every cancel in app/src/androidTest is WorkManager.cancelWorkById against
work that is queued or already finished: ReattachOnLaunchTest cancels a job
carrying a one-hour initial delay, and another immediately after enqueue.
On the JVM, WorkerCancellationTest and HardwareFallbackTest's cancellation
case drive a SoftwareTranscoder double that records the call. No test on
any source set had asked a real native session to stop. That is
docs/defect-audit.md D10's forcing condition.

It is the one path where cancelling wrong is silently expensive rather than
loudly broken: a missed FFmpegKit.cancel leaves the native process encoding
to completion while the UI says the job is cancelled.

Two things were measured rather than assumed, and both changed the test.

The output file cannot be the assertion. invokeOnCancellation deletes the
path, and on POSIX unlinking a file ffmpeg still holds open leaves ffmpeg
writing to the unlinked inode -- so the path stays gone whether or not the
cancel reached the session, and removing FFmpegKit.cancel passes that check
every time. The session's own verdict is what separates them: a cancelled
session ends with the cancel return code, a completed one does not.

Cancelling from the first progress callback loses the race. It was tried
first and failed with state=COMPLETED rc=0: every committed fixture is
2-3 s at 320x240, and the encode finishes before the first statistics
callback is delivered and acted on. FFmpegKit.listSessions shows the
session RUNNING far earlier, so that is what the test waits for.
QualityTier.BEST is deliberate for the same reason -- preset medium leaves
more of the encode ahead of the cancel.

Verified on a local API 34 emulator. Four consecutive runs green at
62/0/0/3, and removing FFmpegKit.cancel while keeping output.delete fails
this test and nothing else, with state=COMPLETED rc=1.

ConcatEngine and Media3Engine carry the same shape and are not covered
here; #224 stays open for them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-05 23:45:34 -05:00
Jason Ross cf540f1ecc Merge pull request #234 from JMR-dev/test/ffmpeg-progress-is-observed
Read the progress percentage FFmpeg has always been computing
2026-09-05 23:21:54 -05:00
JMR-devandClaude Opus 5 948d53b67e Read the progress percentage FFmpeg has always been computing (#229)
FFmpegEngine derives progress as stats.time / durationMs * 100, and the
statistics callback runs on every conversion in FFmpegEngineTest -- they
all pass durationMs = 3_000. But every call site omits onProgress, so
nothing on any source set had ever looked at the number. Replacing percent
with a constant reddened nothing.

What already existed covers the plumbing downstream and not this: #196
covered the worker's progress lambda with a fake engine that reports
whatever the test tells it to, and ProgressNotificationTest covers the
throttling the same way. The arithmetic was the one part with no reader.

The new test passes 30 s as the duration for a fixture that is exactly
3.000 s, so the conversion still encodes the whole clip and the reported
percentage tops out around 10 rather than 100.

That is what makes it bite. A range check alone is worthless: a constant 0
satisfies both "every value is in 0..100" and "the values never go
backwards", and so does a list of [0, 100]. Pinning the band rejects every
constant, and because the band sits a tenth of the way up it also rejects
an implementation that ignores durationMs, which would report ~100 for the
same run. The bound is loose -- 5..25 for an expected 10 -- because the
last statistics callback can land slightly before the final frame.

Verified on a local API 34 emulator: 61 tests, 0 failures, 3 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-05 23:13:36 -05:00
Jason Ross 54932e97c6 Merge pull request #232 from JMR-dev/test/fallback-asserts-the-path
Make the fallback test say when it cannot test the fallback
2026-09-05 23:06:41 -05:00
JMR-devandClaude Opus 5 ba16f5a89b Make the fallback test say when it cannot test the fallback (#223)
HardwareFallbackTest is the only automated check of the hardware->software
fallback against a real codec failure, and it passed on every CI leg
without ever attempting the hardware path.

Measured on run 34004304566: the API 33, 34, 35 and 37 legs each log

  Routing sample_h264_444.mp4 -> ... via FFMPEG (NO_HARDWARE_ENCODER)

Emulators expose no hardware encoder, so the router never chooses Media3
and runMedia3OrFallBack's catch is never entered. The test's assertions --
succeeded, output non-empty -- are true of that conversion too. It
finished in 448 ms, which is not long enough to fail an export and then
software-encode a three-second clip. Deleting the catch reddened nothing.

The ticket offered two fixes and left the choice open. Trying the first
one answered it, and not the way the ticket expected. Pinning
deviceCodecs to PERMISSIVE, as ForcedFailureTest does, makes the router
choose Media3 -- and the export then SUCCEEDS. On a local API 34
emulator, MediaCodecInfo logs

  NoSupport [codec.profileLevel, avc1.F4000C, video/avc]

for both c2.goldfish.h264.decoder and c2.android.avc.decoder, and
ExoPlayer allocates the goldfish decoder anyway, which decodes the High
4:4:4 fixture regardless of the profile it declares. c2.android.hevc.encoder
then encodes it and the job reports MEDIA3.

So the class KDoc's "Media3 fails partway through the export on every
device" is not true of the emulator images, and no routing pressure makes
this fixture force a fallback there. Pinning would also swap in software
codecs, which is not the path a real device takes -- it is what made the
forced run succeed.

That leaves assumeTrue on the production premise as the honest answer, now
with a measurement behind it rather than a coin flip. The test skips where
it cannot mean anything and runs on the Pixel, where it always could.

When it does run the assertion is a pair, because KEY_ENGINE_USED is
FFMPEG whether the fallback fired or the router went straight there:
the router chose MEDIA3 for this request on this device, AND the worker
reported FFMPEG. Together, and only together, that is the fallback.

Verified on a local API 34 emulator: the test reports SKIPPED and the
level reports skipped=3. ForcedFailureTest still covers the fallback
wiring on every leg with a double; what needs a real encoder is two real
engines disagreeing about a real file.

The third permanent skip is recorded in docs/local-emulator.md and beside
SafPickerRoundTripTest's run-shape note.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-05 22:56:18 -05:00
Jason Ross bffcff92c7 Merge pull request #233 from JMR-dev/test/flac-and-opus-assert-their-format
Check that FLAC and Opus are FLAC and Opus
2026-09-05 22:45:47 -05:00
4 changed files with 236 additions and 0 deletions
@@ -12,11 +12,17 @@ import kotlinx.coroutines.withTimeout
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Assume.assumeTrue
import org.junit.Before
import org.junit.Test
import org.junit.runner.RunWith
import org.libremediaconverter.codec.AndroidDeviceCodecs
import org.libremediaconverter.model.ConversionRequest
import org.libremediaconverter.model.ConversionRouter
import org.libremediaconverter.model.Engine
import org.libremediaconverter.model.OutputFormat
import org.libremediaconverter.model.QualityTier
import org.libremediaconverter.model.VideoCodec
import org.libremediaconverter.work.ConversionWorker
import java.io.File
@@ -34,6 +40,47 @@ import java.io.File
* hand — a regression test that silently skips is worse than no test, because the count
* still reads as coverage.
*
* ## Why this skips on emulators, and why that is the honest answer (#223)
*
* **This test used to pass everywhere while proving nothing.** Two independent facts stop the
* fallback happening on an emulator, and both were measured rather than reasoned:
*
* 1. **The router never sends the job to Media3.** A Fast MP4/H.265 job goes to the hardware path
* only when `device.canEncode(H265)`, and emulators expose no hardware encoder — every leg of
* run `34004304566` logged
* `Routing sample_h264_444.mp4 -> ... via FFMPEG (NO_HARDWARE_ENCODER)`. The whole test
* finished in 448 ms, which is not long enough to fail an export and then re-encode.
* 2. **Forcing it to Media3 does not help either, which is the part that settles it.** Pinning
* `ConversionDependencies.deviceCodecs` to [DeviceCodecs.PERMISSIVE] — the trick
* [ForcedFailureTest] uses — makes the router choose Media3, and the export then *succeeds*.
* Measured on a local API 34 emulator: `MediaCodecInfo` logs
* `NoSupport [codec.profileLevel, avc1.F4000C, video/avc]` for **both**
* `c2.goldfish.h264.decoder` and `c2.android.avc.decoder`, and ExoPlayer allocates the
* goldfish decoder anyway, which decodes the file regardless of the profile it declares.
* `c2.android.hevc.encoder` then encodes the result and the job reports `MEDIA3`.
*
* So the class KDoc above — "Media3 fails partway through the export on every device" — **is not
* true of the emulator images**, and no amount of routing pressure makes this fixture force a
* fallback there. The emulator cannot answer this question, so the test says so out loud instead
* of passing.
*
* That is why the gate is [assumeTrue] on the *production* premise (`canEncode(H265)`) rather than
* a pinned profile: pinning would also swap in software codecs, which is not the path a real
* device takes and is what made the forced run succeed. **This is now the third permanent skip**;
* the other two are [org.libremediaconverter.bench.RealMediaBenchmark]'s.
*
* `ForcedFailureTest.hardwareFailureFallsBackToSoftware` still covers the fallback *wiring* on
* every leg, with an `ExplodingHardware` double. What only a device with a real hardware encoder
* can show is two real engines disagreeing about a real file, and that is what this is for.
*
* ## Why the assertion is a pair
*
* `KEY_ENGINE_USED` is `FFMPEG` whether the fallback fired **or** the router went straight there,
* so asserting it alone would not have caught any of the above. The premise is asserted
* separately: [ConversionRouter.route] chooses `MEDIA3` for this request on this device. Static
* routing wanted hardware, the runtime result was software — together, and only together, that is
* the fallback.
*
* The fixture was produced with x264, which the host toolchain cannot do (Fedora's
* ffmpeg ships openh264, which is Constrained Baseline only):
*
@@ -66,6 +113,15 @@ class HardwareFallbackTest {
@Test
fun aFileMedia3CannotDecodeStillConvertsViaFfmpeg(): Unit = runBlocking {
// See "Why this skips on emulators" on the class. Without a real hardware encoder the
// router never chooses Media3, and forcing it makes the export succeed instead of fail --
// so there is no fallback to observe and a green run would mean nothing.
assumeTrue(
"no hardware HEVC encoder, so the router cannot choose Media3 and there is no " +
"fallback to exercise",
AndroidDeviceCodecs.get().canEncode(VideoCodec.H265),
)
val request = ConversionWorker.request(
inputUri = Uri.fromFile(input),
displayName = SAMPLE,
@@ -75,6 +131,19 @@ class HardwareFallbackTest {
// the tier where the fallback has to rescue the conversion.
quality = QualityTier.FAST,
)
// The premise, asserted rather than assumed: this request is one the router wants to send
// to hardware on this device. Without it the test is green whether the fallback fired or
// the job never went near Media3, which is exactly how #223 stayed invisible.
val decision = ConversionRouter.route(
ConversionRequest(OutputFormat.MP4_H265.spec, quality = QualityTier.FAST),
AndroidDeviceCodecs.get(),
)
assertEquals(
"this test only means something if the router sends this job to Media3",
Engine.MEDIA3,
decision.engine,
)
workManager.enqueue(request).result.get()
val terminal = withTimeout(TIMEOUT_MS) {
@@ -88,6 +157,14 @@ class HardwareFallbackTest {
terminal?.state,
)
// The outcome. Paired with the routing assertion above this is the fallback and nothing
// else: hardware was chosen, software is what ran.
assertEquals(
"the router chose Media3, so a successful job must have fallen back to FFmpeg",
Engine.FFMPEG.name,
terminal?.outputData?.getString(ConversionWorker.KEY_ENGINE_USED),
)
val out = File(terminal!!.outputData.getString(ConversionWorker.KEY_OUTPUT_PATH)!!)
assertTrue("no output produced", out.exists() && out.length() > 0)
out.delete()
@@ -4,7 +4,16 @@ import android.media.MediaExtractor
import android.media.MediaFormat
import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.platform.app.InstrumentationRegistry
import com.arthenica.ffmpegkit.FFmpegKit
import com.arthenica.ffmpegkit.FFmpegSession
import com.arthenica.ffmpegkit.ReturnCode
import com.arthenica.ffmpegkit.SessionState
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.cancelAndJoin
import kotlinx.coroutines.delay
import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.withTimeout
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
@@ -139,6 +148,136 @@ class FFmpegEngineTest {
assertEquals("OggS", magic)
}
/**
* The percentage itself, which every other test in this class computes and none of them reads.
*
* `FFmpegEngine` derives progress as `stats.time / durationMs * 100`, and the statistics
* callback runs on every conversion here — but every call site omits `onProgress`, so until
* this test nothing on any source set had ever looked at the number (#229). #196 covered the
* *worker's* progress lambda, and did it with a fake engine that reports whatever the test
* tells it to; `ProgressNotificationTest` covers throttling the same way. The arithmetic was
* the one part with no reader.
*
* ## Why the duration is deliberately wrong
*
* `sample_h264.mp4` is exactly 3.000 s, and this passes **30 s** as the duration. So the
* conversion still encodes the whole clip, `stats.time` still climbs to about 3000 ms, and the
* reported percentage tops out around **10** rather than 100.
*
* That is what makes the assertion bite. A range check alone is worthless here: replacing
* `percent` with a constant `0` satisfies "every value is in 0..100" and "the values never go
* backwards", and so does a list of `[0, 100]`. Pinning the *band* rejects every constant, and
* — because the band is a tenth of the way up — it also rejects an implementation that ignores
* `durationMs`, which would report ~100 for the same run.
*
* The bound is deliberately loose (5..25 for an expected 10). The last statistics callback can
* land slightly before the final frame, so the peak is "about 3000 ms of a claimed 30 000",
* not exactly it.
*/
@Test
fun progressIsReportedAsAFractionOfTheDurationItWasGiven() {
val seen = mutableListOf<Int>()
val out = outputFor("out_progress.mp4")
runBlocking {
engine.run(
request = ConversionRequest(spec = OutputFormat.MP4_H264.spec, quality = QualityTier.BEST),
inputPath = input.absolutePath,
output = out,
// Ten times the fixture's real 3 s. See the KDoc.
durationMs = 30_000,
onProgress = { percent -> seen += percent },
)
}
assertTrue("the statistics callback never reported progress", seen.isNotEmpty())
assertTrue("progress out of range: $seen", seen.all { it in 0..100 })
assertEquals("progress went backwards: $seen", seen.sorted(), seen)
// The band. Rejects any constant, and rejects ignoring durationMs (which would read ~100).
val peak = seen.max()
assertTrue(
"3 s of media against a claimed 30 s should peak near 10%, got $peak from $seen",
peak in 5..25,
)
}
/**
* Cancelling a *running* conversion actually stops the native session.
*
* Nothing on any source set did this before (#224). Every `cancel` in `app/src/androidTest` is
* `WorkManager.cancelWorkById` against work that is **queued or already finished** — the two in
* `ReattachOnLaunchTest` cancel a job carrying a one-hour initial delay, and one immediately
* after enqueue. On the JVM, `WorkerCancellationTest` and `HardwareFallbackTest`'s cancellation
* case drive a `SoftwareTranscoder` double that records the call. No test had ever asked a real
* native session to stop. This is `docs/defect-audit.md` **D10**'s forcing condition.
*
* It is the one path where cancelling wrong is silently expensive rather than loudly broken: a
* missed `FFmpegKit.cancel` leaves the native process encoding to completion while the UI says
* the job is cancelled, and nothing reports the battery and thermal cost.
*
* ## Why the assertion is the session's return code, not the output file
*
* The obvious assertion — the partial output is gone — **cannot fail**, so it would have been a
* vacuous test. `invokeOnCancellation` deletes the path, and on POSIX unlinking a file ffmpeg
* still holds open leaves ffmpeg writing to the unlinked inode; the path stays gone whether or
* not the cancel ever reached the session. Deleting `FFmpegKit.cancel` and keeping
* `output.delete()` passes that check every time.
*
* What distinguishes them is the session's own verdict: a cancelled session ends with the
* cancel return code, a completed one ends successfully. That is a fact about the session
* rather than about timing, so it is read *after* waiting for the session to leave
* [SessionState.RUNNING] rather than at a fixed delay.
*
* ## Why it cancels on RUNNING rather than on the first progress callback
*
* Measured, and this is the part worth keeping. Cancelling from the first `onProgress` was
* tried first and **failed on a local API 34 emulator with `state=COMPLETED rc=0`** — every
* committed fixture is 2-3 s at 320x240, and the encode finishes before the first statistics
* callback has been delivered and acted on. The progress callback is proof the session is
* running, but it arrives too late to interrupt anything.
*
* `FFmpegKit.listSessions` shows the session as [SessionState.RUNNING] far earlier, so that is
* what is waited on. `QualityTier.BEST` is deliberate for the same reason: `-preset medium`
* leaves more of the encode ahead of the cancel than `veryfast` would.
*
* The session is identified by diffing against the ids present before the run, because this
* class has already produced eight of them by the time this executes.
*/
@Test
fun cancellingARunningConversionCancelsTheNativeSession(): Unit = runBlocking {
val before = FFmpegKit.listSessions().map { it.getSessionId() }.toSet()
val out = outputFor("out_cancelled.mp4")
val job = launch(Dispatchers.IO) {
engine.run(
request = ConversionRequest(spec = OutputFormat.MP4_H265.spec, quality = QualityTier.BEST),
inputPath = input.absolutePath,
output = out,
durationMs = 3_000,
)
}
// Interrupt as early as the session can be observed at all. See the KDoc: waiting for
// progress instead lost the race outright.
val ours = withTimeout(TIMEOUT_MS) {
var found: FFmpegSession? = null
while (found?.getState() != SessionState.RUNNING) {
found = FFmpegKit.listSessions().firstOrNull { it.getSessionId() !in before }
if (found?.getState() != SessionState.RUNNING) delay(POLL_MS)
}
found
}
job.cancelAndJoin()
withTimeout(TIMEOUT_MS) {
while (ours.getState() == SessionState.RUNNING) delay(POLL_MS)
}
assertTrue(
"the native session was not cancelled: state=${ours.getState()} rc=${ours.getReturnCode()}",
ReturnCode.isCancel(ours.getReturnCode()),
)
}
// --- the quality tier the GPL licence was taken for --------------------
@Test
@@ -176,4 +315,10 @@ class FFmpegEngineTest {
}.exceptionOrNull()
assertTrue("expected an FFmpegException, got $failure", failure is FFmpegEngine.FFmpegException)
}
private companion object {
/** Generous: it bounds a hang, and every wait here normally settles in well under a second. */
const val TIMEOUT_MS = 30_000L
const val POLL_MS = 50L
}
}
@@ -207,6 +207,8 @@ import java.util.concurrent.atomic.AtomicInteger
* driven there at all. That is why this gap survived as long as it did.
* `tools/local-emulator/run-e2e.sh` runs API 33-36 on the development host, and both tests pass
* there: **59 / 0 / 0 / 2 at API 33 and again at API 36**, whole suite, 2026-08-24.
* (Since #223 the skip column reads 3 on an emulator — `HardwareFallbackTest` now announces
* that it cannot run without a hardware HEVC encoder rather than passing vacuously.)
*
* ### Why only the rotation test carries [FailsOnEmulatorApi37]
*
+12
View File
@@ -312,6 +312,18 @@ on sample media that is deliberately not committed. Its third test,
`reportDeviceEncoderCapabilities`, has no such guard and runs. A level reporting 0 skipped
would mean someone had staged sample files, not that something improved.
**Since #223 there is a third, and it is the interesting one.**
`HardwareFallbackTest.aFileMedia3CannotDecodeStillConvertsViaFfmpeg` is `assumeTrue`-guarded on
`AndroidDeviceCodecs.get().canEncode(H265)`, which is false on every emulator image — so it now
skips here and runs only on the Pixel. It used to *pass* on emulators without ever attempting the
hardware path, which is worse. **Expect `skipped="3"` locally**, and note the guard is a property
of the machine rather than of staged files: a level reporting 2 would mean an emulator image had
gained a hardware HEVC encoder, which is worth knowing.
That test's KDoc carries the measurement, including the part that decides it: forcing the route to
Media3 anyway does *not* produce a fallback, because the goldfish decoder decodes the High 4:4:4
fixture despite declaring `NoSupport` for its profile.
### What the sweep adds, and what it does not
**The renderer rule held four more times.** No boot log contains the string