Stop selecting FFmpeg's MediaCodec encoders, and pin the pixel format
Testing on a Pixel 10 Pro XL with real footage found two defects that neither the emulator nor any unit test could surface. The sample is H.264 High 4:4:4 Predictive (avc1.F4001F). Media3 cannot decode it on any device: hardware AVC decoders implement High 4:2:0, and Android's software c2.google.avc.decoder does not cover 4:4:4 either, so the export dies with a codec exception. The static routing rules cannot predict this -- the container is MP4, the codec is "h264", and the device reports AVC decode and encode -- so it is exactly the case the runtime fallback exists for. Until a file like this was tried on hardware, that fallback had never actually fired. Following it through exposed the two bugs behind it. First, only one video encode path named a pixel format. FFmpeg decodes 4:4:4 to yuv444p and hands those frames to encoders that cannot accept them; naming yuv420p makes it insert the conversion instead. Every path now does. Second, and not fixed by that: hevc_mediacodec still failed with "Error submitting video frame to the encoder". That is the second distinct failure from FFmpeg's MediaCodec wrappers in one session -- the first being that on a device with no hardware encoder they silently bind to the platform software codec and crawl while presenting as the fast path. They are undocumented, per-device flaky, and duplicate badly something Media3 already does properly. A job only reaches FFmpeg because Media3 could not handle it, which is itself evidence hardware encode is unlikely to work for that input. So FFmpeg now always encodes in software, and Fast means a fast preset rather than a different encoder: libx264/libx265 at veryfast against medium, both with CRF. With that, the fallback completes and the file converts. Measured on the Pixel: AV1 1080p through the hardware path runs at 7.9x realtime, confirming the figure the two-engine design was based on; software x264 CRF at 720p runs at 4.4x. Hardware encoders present are av01, avc and hevc -- AV1 encode included, which is still rare. 29 instrumented tests pass on device, 63 unit tests on the JVM. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+39
-33
@@ -69,19 +69,41 @@ class FFmpegCommandBuilderTest {
|
||||
assertPair(cmd(OutputFormat.MP4_H265, QualityTier.BEST), "-tag:v", "hvc1")
|
||||
}
|
||||
|
||||
/**
|
||||
* Regression test for a failure found with real footage on a Pixel 10 Pro XL.
|
||||
*
|
||||
* The source was H.264 High 4:4:4 Predictive. FFmpeg decodes that to yuv444p and,
|
||||
* without an explicit pixel format, hands those frames straight to an encoder that
|
||||
* cannot accept them — hevc_mediacodec died with "Invalid to call at Released
|
||||
* state" partway through. Every video encode path has to name the format so FFmpeg
|
||||
* inserts the conversion, not just the one path that happened to have it.
|
||||
*/
|
||||
@Test
|
||||
fun `h264 output is forced to yuv420p for hardware player compatibility`() {
|
||||
assertPair(cmd(OutputFormat.MP4_H264, QualityTier.BEST), "-pix_fmt", "yuv420p")
|
||||
fun `every video encode path forces yuv420p`() {
|
||||
val videoFormats = listOf(
|
||||
OutputFormat.MP4_H264, OutputFormat.MP4_H265,
|
||||
OutputFormat.MKV_H264, OutputFormat.MKV_H265,
|
||||
OutputFormat.WEBM_VP9,
|
||||
)
|
||||
videoFormats.forEach { format ->
|
||||
listOf(QualityTier.FAST, QualityTier.BEST).forEach { quality ->
|
||||
listOf(true, false).forEach { hw ->
|
||||
val args = cmd(format, quality, hardwareEncodeAvailable = hw)
|
||||
assertPair(args, "-pix_fmt", "yuv420p")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `audio only outputs do not set a pixel format`() {
|
||||
listOf(OutputFormat.MP3, OutputFormat.FLAC, OutputFormat.WAV).forEach {
|
||||
assertFalse("${it.name} should not set -pix_fmt", cmd(it).contains("-pix_fmt"))
|
||||
}
|
||||
}
|
||||
|
||||
// --- the hardware fallback path ---------------------------------------
|
||||
|
||||
@Test
|
||||
fun `fast quality uses the mediacodec wrappers`() {
|
||||
assertPair(cmd(OutputFormat.MP4_H264, QualityTier.FAST), "-c:v", "h264_mediacodec")
|
||||
assertPair(cmd(OutputFormat.MP4_H265, QualityTier.FAST), "-c:v", "hevc_mediacodec")
|
||||
}
|
||||
|
||||
/**
|
||||
* Regression test for a defect found on a device with no hardware HEVC encoder.
|
||||
*
|
||||
@@ -92,24 +114,15 @@ class FFmpegCommandBuilderTest {
|
||||
* honest about what it is doing.
|
||||
*/
|
||||
@Test
|
||||
fun `fast quality without a hardware encoder uses a fast software preset`() {
|
||||
val h264 = cmd(OutputFormat.MP4_H264, QualityTier.FAST, hardwareEncodeAvailable = false)
|
||||
assertPair(h264, "-c:v", "libx264")
|
||||
assertPair(h264, "-preset", "veryfast")
|
||||
assertFalse("must not claim the mediacodec path", h264.contains("h264_mediacodec"))
|
||||
|
||||
val h265 = cmd(OutputFormat.MP4_H265, QualityTier.FAST, hardwareEncodeAvailable = false)
|
||||
assertPair(h265, "-c:v", "libx265")
|
||||
assertPair(h265, "-preset", "veryfast")
|
||||
assertFalse("must not claim the mediacodec path", h265.contains("hevc_mediacodec"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `fast quality with a hardware encoder still prefers mediacodec`() {
|
||||
assertPair(
|
||||
cmd(OutputFormat.MP4_H264, QualityTier.FAST, hardwareEncodeAvailable = true),
|
||||
"-c:v", "h264_mediacodec",
|
||||
)
|
||||
fun `the encoder choice no longer depends on hardware availability`() {
|
||||
// Once FFmpeg stopped selecting MediaCodec encoders, this flag only affects
|
||||
// whether the router sends the job to Media3 at all -- not what FFmpeg does.
|
||||
listOf(OutputFormat.MP4_H264, OutputFormat.MP4_H265).forEach { format ->
|
||||
assertEquals(
|
||||
cmd(format, QualityTier.FAST, hardwareEncodeAvailable = true),
|
||||
cmd(format, QualityTier.FAST, hardwareEncodeAvailable = false),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
@@ -122,13 +135,6 @@ class FFmpegCommandBuilderTest {
|
||||
assertEquals("medium", best[best.indexOf("-preset") + 1])
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `fast quality targets a bitrate because mediacodec has no crf`() {
|
||||
val args = cmd(OutputFormat.MP4_H264, QualityTier.FAST)
|
||||
assertTrue(args.contains("-b:v"))
|
||||
assertFalse(args.contains("-crf"))
|
||||
}
|
||||
|
||||
// --- audio -------------------------------------------------------------
|
||||
|
||||
@Test
|
||||
|
||||
Reference in New Issue
Block a user