diff --git a/README.md b/README.md index 8024ee2..f8c1cbb 100644 --- a/README.md +++ b/README.md @@ -23,20 +23,26 @@ Two conversion engines behind an explicit router, because neither one covers the ### AndroidX Media3 Transformer — the hardware path -Handles the common cases: MP4/MOV in and out, H.264/HEVC, resolution and frame-rate -changes, rotation, overlays, audio to AAC, and stream-copy transmuxing. Fully hardware -accelerated end to end — MediaCodec decodes to a GL surface and MediaCodec re-encodes, -so frames never round-trip through the CPU. Roughly 7–8× realtime on 720p. +Handles the common cases: H.264/HEVC, resolution and frame-rate changes, rotation, +overlays, and audio to AAC. Fully hardware accelerated end to end — MediaCodec decodes to +a GL surface and MediaCodec re-encodes, so frames never round-trip through the CPU. +Roughly 7–8× realtime on 720p. + +It writes MP4, WebM, Ogg, WAV and raw AAC — the five containers `media3-muxer` provides a +muxer for. It *reads* far more than it writes, Matroska included, which is what makes +MKV → MP4 a hardware remux. ### FFmpeg — the long tail Everything Media3 structurally cannot do: -- Containers outside MP4/WebM/Ogg/WAV/AAC — MKV, AVI, FLV, MPEG-TS +- Containers outside MP4/WebM/Ogg/WAV/AAC — MKV, MOV, AVI, FLV, MPEG-TS, WMV/ASF - **MP3 output** — Android has no MP3 encoder at any version; this is a platform gap - GIF and image sequences - Input codecs with no platform decoder on the device - CRF and 2-pass rate control, for the quality tier +- Codecs Media3's muxers decline even on a stream copy — its MP4 muxer carries AAC, Opus, + Vorbis and PCM, but neither MP3 nor FLAC ### Quality tiers @@ -66,14 +72,35 @@ extension appears in any Android Vulkan Profile tier. So this app is hardware accelerated via MediaCodec, and GPU accelerated for effects via GL shaders. Both are real; neither is "the GPU decoding video." +## Remuxing + +Changing the container without touching the streams. Copying an H.264 track from MKV into +MP4 moves the same samples into a different wrapper: it finishes in seconds instead of +minutes, costs no quality, and needs no encoder — which is why it stays on the hardware +path even on a device that cannot encode the codec in question. + +`Copy` is a codec choice like any other, so it can be mixed: copy the video and re-encode +only the audio, or the reverse. Picking a codec the source already uses is upgraded to a +copy automatically **when the container is changing** — if container and codec both already +match, the only reason to run the job is to re-encode it, so it does. + +A copy is never attempted on a stream whose codec could not be identified. A needless +re-encode costs time; a wrong stream copy costs a file that will not play. + ## Features | | Formats | |---|---| -| Video out | MP4 (H.264/H.265), MKV (H.264/H.265), WebM (VP9) | -| Audio out | MP3, AAC/M4A, FLAC, Opus, WAV | +| Video out | MP4, MOV, MKV, WebM, MPEG-TS, AVI, FLV, WMV/ASF | +| Video codecs | H.264, H.265, VP9, or copy the source stream | +| Audio out | MP3, AAC/M4A, FLAC, Opus, WAV, MKA | +| Audio codecs | AAC, Opus, MP3, FLAC, PCM, or copy the source stream | | Images | GIF, PNG frame sequences | -| Other | Join several files into one | +| Other | Remux without re-encoding; join several files into one | + +Presets cover the common combinations in one tap. The Advanced picker exposes the full +container × codec matrix — including combinations that cannot work, which it explains and +offers alternatives for rather than hiding. Conversions run as durable background work, so they survive leaving the app and are restored after a restart. @@ -101,14 +128,17 @@ obligation. ## Testing -Unit tests cover the parts that decide correctness without needing hardware: the -routing matrix, the FFmpeg argument builder, and the stream-copy-versus-re-encode -planner. They run against fabricated device profiles, so branches like "this device -cannot encode HEVC" are reachable regardless of what the test machine is. +Unit tests cover the parts that decide correctness without needing hardware: the routing +matrix, the container × codec capability matrix, the FFmpeg argument builder, and both +stream-copy-versus-re-encode planners. They run against fabricated device profiles, so +branches like "this device cannot encode HEVC" are reachable regardless of what the test +machine is. Instrumented tests cover the parts that only a device can prove: real hardware -transcoding, the foreground service type, and each FFmpeg output format asserted -against the produced file rather than the exit code. +transcoding, the foreground service type, and each FFmpeg output format asserted against +the produced file rather than the exit code. The remux tests additionally assert which +*engine* ran — a stream copy produces an identical file either way, so an output-only +assertion cannot tell a hardware transmux from FFmpeg's `-c copy`. ## Privacy diff --git a/app/src/androidTest/assets/sample_aac.m4a b/app/src/androidTest/assets/sample_aac.m4a new file mode 100644 index 0000000..f014643 Binary files /dev/null and b/app/src/androidTest/assets/sample_aac.m4a differ diff --git a/app/src/androidTest/assets/sample_h264.mkv b/app/src/androidTest/assets/sample_h264.mkv new file mode 100644 index 0000000..b5d33e8 Binary files /dev/null and b/app/src/androidTest/assets/sample_h264.mkv differ diff --git a/app/src/androidTest/assets/sample_still.png b/app/src/androidTest/assets/sample_still.png new file mode 100644 index 0000000..8398379 Binary files /dev/null and b/app/src/androidTest/assets/sample_still.png differ diff --git a/app/src/androidTest/assets/sample_vp9.webm b/app/src/androidTest/assets/sample_vp9.webm new file mode 100644 index 0000000..7703b05 Binary files /dev/null and b/app/src/androidTest/assets/sample_vp9.webm differ diff --git a/app/src/androidTest/java/org/libremediaconverter/bench/RealMediaBenchmark.kt b/app/src/androidTest/java/org/libremediaconverter/bench/RealMediaBenchmark.kt index 668fc6b..887e811 100644 --- a/app/src/androidTest/java/org/libremediaconverter/bench/RealMediaBenchmark.kt +++ b/app/src/androidTest/java/org/libremediaconverter/bench/RealMediaBenchmark.kt @@ -100,7 +100,7 @@ class RealMediaBenchmark { val hwOut = File(context.cacheDir, "bench_hw.mp4").apply { delete() } val engine = Media3Engine(context) val hwMs = try { - runCatching { timed { engine.transcode(uri, hwOut, OutputFormat.MP4_H265) } } + runCatching { timed { engine.transcode(uri, hwOut, ConversionRequest(OutputFormat.MP4_H265.spec)) } } .onFailure { Log.w(TAG, "BENCH hardware: UNSUPPORTED (${it.message})") } .getOrNull() } finally { @@ -112,7 +112,7 @@ class RealMediaBenchmark { val swMs = timed { FFmpegEngine().run( request = ConversionRequest( - format = OutputFormat.MP4_H264, + spec = OutputFormat.MP4_H264.spec, quality = QualityTier.BEST, ), inputPath = input.absolutePath, @@ -153,7 +153,7 @@ class RealMediaBenchmark { val probe = MediaProbe.probe(context, Uri.fromFile(input)) val decision = ConversionRouter.route( - ConversionRequest(format = OutputFormat.MP4_H265, probe = probe), + ConversionRequest(spec = OutputFormat.MP4_H265.spec, probe = probe), AndroidDeviceCodecs.get(), ) Log.i(TAG, "BENCH av1 probe: codec=${probe.videoCodec} duration=${probe.durationMs}ms") @@ -162,7 +162,7 @@ class RealMediaBenchmark { val out = File(context.cacheDir, "bench_av1_out.mp4").apply { delete() } val engine = Media3Engine(context) val ms = try { - timed { engine.transcode(Uri.fromFile(input), out, OutputFormat.MP4_H265) } + timed { engine.transcode(Uri.fromFile(input), out, ConversionRequest(OutputFormat.MP4_H265.spec)) } } finally { engine.close() } diff --git a/app/src/androidTest/java/org/libremediaconverter/convert/Media3EngineTest.kt b/app/src/androidTest/java/org/libremediaconverter/convert/Media3EngineTest.kt index cf6d1f0..32d5431 100644 --- a/app/src/androidTest/java/org/libremediaconverter/convert/Media3EngineTest.kt +++ b/app/src/androidTest/java/org/libremediaconverter/convert/Media3EngineTest.kt @@ -16,6 +16,7 @@ import org.junit.Assert.assertTrue import org.junit.Before import org.junit.Test import org.junit.runner.RunWith +import org.libremediaconverter.model.ConversionRequest import org.libremediaconverter.model.OutputFormat import java.io.File import java.util.concurrent.Executors @@ -62,7 +63,7 @@ class Media3EngineTest { engine.transcode( input = Uri.fromFile(input), output = output, - format = OutputFormat.MP4_H265, + request = ConversionRequest(OutputFormat.MP4_H265.spec), ) { percent -> seen += percent } assertTrue("export produced no file", output.exists()) @@ -95,7 +96,7 @@ class Media3EngineTest { val audio = File(context.cacheDir, "out_audio.m4a") audio.delete() try { - engine.transcode(Uri.fromFile(input), audio, OutputFormat.M4A_AAC) + engine.transcode(Uri.fromFile(input), audio, ConversionRequest(OutputFormat.M4A_AAC.spec)) assertTrue("export produced no file", audio.exists() && audio.length() > 0) val tracks = trackMimeTypesOf(audio) @@ -121,7 +122,7 @@ class Media3EngineTest { val wav = File(context.cacheDir, "out_audio.wav") wav.delete() try { - engine.transcode(Uri.fromFile(input), wav, OutputFormat.WAV) + engine.transcode(Uri.fromFile(input), wav, ConversionRequest(OutputFormat.WAV.spec)) assertTrue("export produced no file", wav.exists() && wav.length() > 0) assertEquals("RIFF", String(wav.readBytes().copyOfRange(0, 4), Charsets.US_ASCII)) @@ -141,7 +142,7 @@ class Media3EngineTest { val ogg = File(context.cacheDir, "out_audio.opus") ogg.delete() try { - engine.transcode(Uri.fromFile(input), ogg, OutputFormat.OPUS) + engine.transcode(Uri.fromFile(input), ogg, ConversionRequest(OutputFormat.OPUS.spec)) assertTrue("export produced no file", ogg.exists() && ogg.length() > 0) assertEquals("OggS", String(ogg.readBytes().copyOfRange(0, 4), Charsets.US_ASCII)) @@ -197,7 +198,7 @@ class Media3EngineTest { val failure = runCatching { runBlocking { withTimeout(30_000) { - engine.transcode(Uri.fromFile(input), impossible, OutputFormat.MP4_H265) {} + engine.transcode(Uri.fromFile(input), impossible, ConversionRequest(OutputFormat.MP4_H265.spec)) {} } } }.exceptionOrNull() diff --git a/app/src/androidTest/java/org/libremediaconverter/convert/RemuxTest.kt b/app/src/androidTest/java/org/libremediaconverter/convert/RemuxTest.kt new file mode 100644 index 0000000..827a212 --- /dev/null +++ b/app/src/androidTest/java/org/libremediaconverter/convert/RemuxTest.kt @@ -0,0 +1,277 @@ +package org.libremediaconverter.convert + +import android.media.MediaExtractor +import android.media.MediaFormat +import android.net.Uri +import androidx.media3.common.util.UnstableApi +import androidx.test.ext.junit.runners.AndroidJUnit4 +import androidx.test.platform.app.InstrumentationRegistry +import androidx.work.WorkInfo +import androidx.work.WorkManager +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.runBlocking +import kotlinx.coroutines.withTimeout +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNotNull +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.Container +import org.libremediaconverter.model.ConversionRequest +import org.libremediaconverter.model.Engine +import org.libremediaconverter.model.InputKind +import org.libremediaconverter.model.OutputSpec +import org.libremediaconverter.model.VideoCodec +import org.libremediaconverter.work.ConversionWorker +import java.io.File + +/** + * Remuxing end to end, on a device. + * + * ## Why these assert the engine, not just the file + * + * A remux routed to FFmpeg produces a perfectly correct file — `-c copy` moves the same samples + * into the same container. So an output-only assertion passes whether the hardware transmux path + * ran or never executed at all, and `COPY` belongs to none of the router's capability sets, which + * makes "silently always FFmpeg" the most likely way for this feature to regress. + * + * Timing is not a usable proxy either: it is flaky on emulators and cannot distinguish an FFmpeg + * stream copy from a Media3 transmux, since both are fast. `ConversionWorker` already reports + * `KEY_ENGINE_USED`, so these assert that. + * + * ## Fixtures + * + * Every committed asset was ISO MP4 before this feature, so a remux had no second container to + * move between. The new ones are generated rather than sourced, and committed rather than built at + * test time, following the convention `HardwareFallbackTest` documents — a regression test that + * silently skips is worse than no test. Recipes, run against FFmpeg 8.1.2: + * + * ``` + * # sample_h264.mkv — the existing fixture's streams in Matroska. A pure remux, so the two files + * # hold byte-identical samples and differ only in the container, which is exactly the axis under + * # test. No encoder is involved, so the recipe reproduces anywhere. + * ffmpeg -i sample_h264.mp4 -c copy -f matroska sample_h264.mkv + * + * # sample_aac.m4a — the same fixture's audio track alone, for AUDIO_ONLY probing and MKA output. + * ffmpeg -i sample_h264.mp4 -vn -c:a copy -f mp4 sample_aac.m4a + * + * # sample_vp9.webm — the one fixture that must be encoded, since no committed source is VP9. + * ffmpeg -i sample_h264.mp4 -c:v libvpx-vp9 -crf 40 -b:v 0 -deadline realtime -cpu-used 8 \ + * -c:a libopus -b:a 64k -f webm sample_vp9.webm + * + * # sample_still.png — a single frame, to reach the probe's IMAGE branch. + * ffmpeg -i sample_h264.mp4 -frames:v 1 -f image2 sample_still.png + * ``` + */ +@UnstableApi +@RunWith(AndroidJUnit4::class) +class RemuxTest { + + private val context = InstrumentationRegistry.getInstrumentation().targetContext + private val staged = mutableListOf() + + @Before + fun setUp() { + WorkManager.getInstance(context).cancelAllWork() + } + + @After + fun tearDown() { + staged.forEach { it.delete() } + WorkManager.getInstance(context).cancelAllWork() + } + + private fun asset(name: String): File = + File(context.cacheDir, name).also { file -> + file.delete() + staged += file + InstrumentationRegistry.getInstrumentation().context.assets + .open(name) + .use { asset -> file.outputStream().use { asset.copyTo(it) } } + } + + // --- probing ------------------------------------------------------------ + + @Test + fun probeIdentifiesTheSourceContainerOfEachFixture() { + assertEquals(Container.MP4, MediaProbe.probe(context, Uri.fromFile(asset("sample_h264.mp4"))).container) + assertEquals(Container.MKV, MediaProbe.probe(context, Uri.fromFile(asset("sample_h264.mkv"))).container) + assertEquals(Container.WEBM, MediaProbe.probe(context, Uri.fromFile(asset("sample_vp9.webm"))).container) + } + + /** + * The distinction the old probe could not make. + * + * It reported `hasVideo = true, videoCodec = UNPARSEABLE` for anything the extractor refused, + * so an audio file, a picture and a corrupt file were indistinguishable — and the source-info + * card cannot describe any of them honestly until they are separate. + */ + @Test + fun probeDistinguishesAudioFromImagesFromRubbish() { + val audio = MediaProbe.probe(context, Uri.fromFile(asset("sample_aac.m4a"))) + assertEquals(InputKind.AUDIO_ONLY, audio.kind) + assertEquals(false, audio.hasVideo) + assertNotNull("audio codec should still be identified", audio.audioCodec) + + val image = MediaProbe.probe(context, Uri.fromFile(asset("sample_still.png"))) + assertEquals(InputKind.IMAGE, image.kind) + + val rubbish = File(context.cacheDir, "not_media.bin").also { + staged += it + it.writeBytes(ByteArray(4096) { i -> (i % 251).toByte() }) + } + val unreadable = MediaProbe.probe(context, Uri.fromFile(rubbish)) + assertEquals(InputKind.UNPARSEABLE, unreadable.kind) + } + + // --- the hardware remux path -------------------------------------------- + + /** + * MKV in, MP4 out, nothing re-encoded — and it must stay on Media3. + * + * Media3 cannot *write* Matroska but reads it perfectly well, which is what makes this + * direction a hardware remux. If this reports FFMPEG, the transmux path is dead. + */ + @Test + fun mkvToMp4RemuxesOnHardware() { + val input = asset("sample_h264.mkv") + val result = runConversion( + input, + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + ) + + assertEquals(Engine.MEDIA3.name, result.engineUsed) + assertEquals("h264 must survive a copy untouched", "video/avc", videoMimeOf(result.output)) + assertTrue("output has no duration", durationMsOf(result.output) > 0) + } + + /** The reverse direction: Media3 has no Matroska muxer, so this is FFmpeg's. */ + @Test + fun mp4ToMkvRemuxesOnFFmpeg() { + val input = asset("sample_h264.mp4") + val result = runConversion( + input, + OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY), + ) + + assertEquals(Engine.FFMPEG.name, result.engineUsed) + assertEquals( + "expected an EBML header", + listOf(0x1A, 0x45, 0xDF, 0xA3), + result.output.readBytes().take(4).map { it.toInt() and 0xFF }, + ) + } + + @Test + fun webmToMkvKeepsVp9WithoutReencoding() { + val input = asset("sample_vp9.webm") + val result = runConversion( + input, + OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY), + ) + + assertTrue(result.output.length() > 0) + assertEquals("vp9 must survive a copy", "video/x-vnd.on2.vp9", videoMimeOf(result.output)) + } + + @Test + fun audioOnlySourceRemuxesIntoMka() { + val input = asset("sample_aac.m4a") + val spec = OutputSpec(Container.MKV, VideoCodec.NONE, AudioCodec.COPY) + val result = runConversion(input, spec) + + assertEquals("mka", spec.extension) + assertTrue(result.output.length() > 0) + val tracks = trackMimesOf(result.output) + assertTrue("expected audio only, got $tracks", tracks.none { it.startsWith("video/") }) + } + + @Test + fun mp4ToMpegTsAndAviProduceTheirOwnContainers() { + val input = asset("sample_h264.mp4") + + val ts = runConversion(input, OutputSpec(Container.MPEG_TS, VideoCodec.COPY, AudioCodec.COPY)) + assertEquals(Engine.FFMPEG.name, ts.engineUsed) + // Every MPEG-TS packet starts with the 0x47 sync byte. + assertEquals(0x47, ts.output.readBytes().first().toInt() and 0xFF) + + val avi = runConversion(input, OutputSpec(Container.AVI, VideoCodec.COPY, AudioCodec.COPY)) + assertEquals("RIFF", String(avi.output.readBytes().copyOfRange(0, 4), Charsets.US_ASCII)) + } + + // --- plumbing ----------------------------------------------------------- + + private class Result(val output: File, val engineUsed: String) + + /** + * Runs one conversion through the real worker. + * + * Deliberately through `ConversionWorker` rather than an engine directly: the routing decision + * is the thing under test, and only the worker makes it. + */ + private fun runConversion(input: File, spec: OutputSpec): Result = runBlocking { + val manager = WorkManager.getInstance(context) + val request = ConversionWorker.request( + inputUri = Uri.fromFile(input), + displayName = input.name, + sizeBytes = input.length(), + spec = spec, + ) + manager.enqueue(request).result.get() + + val terminal = withTimeout(TIMEOUT_MS) { + manager.getWorkInfoByIdFlow(request.id).first { info -> + info != null && info.state.isFinished + } + } + + assertEquals( + "conversion did not succeed: " + + terminal?.outputData?.getString(ConversionWorker.KEY_ERROR), + WorkInfo.State.SUCCEEDED, + terminal?.state, + ) + val path = requireNotNull(terminal?.outputData?.getString(ConversionWorker.KEY_OUTPUT_PATH)) + val output = File(path).also { staged += it } + assertTrue("output is empty", output.length() > 0) + Result( + output = output, + engineUsed = terminal?.outputData + ?.getString(ConversionWorker.KEY_ENGINE_USED).orEmpty(), + ) + } + + private fun trackMimesOf(file: File): List { + val extractor = MediaExtractor() + return try { + extractor.setDataSource(file.absolutePath) + (0 until extractor.trackCount) + .map { extractor.getTrackFormat(it).getString(MediaFormat.KEY_MIME).orEmpty() } + } finally { + extractor.release() + } + } + + private fun videoMimeOf(file: File): String? = + trackMimesOf(file).firstOrNull { it.startsWith("video/") } + + private fun durationMsOf(file: File): Long { + val extractor = MediaExtractor() + return try { + extractor.setDataSource(file.absolutePath) + (0 until extractor.trackCount) + .map { extractor.getTrackFormat(it) } + .filter { it.containsKey(MediaFormat.KEY_DURATION) } + .maxOfOrNull { it.getLong(MediaFormat.KEY_DURATION) / 1000 } ?: 0L + } finally { + extractor.release() + } + } + + private companion object { + const val TIMEOUT_MS = 180_000L + } +} diff --git a/app/src/androidTest/java/org/libremediaconverter/fallback/FakeFailures.kt b/app/src/androidTest/java/org/libremediaconverter/fallback/FakeFailures.kt index f2839d4..3171d9a 100644 --- a/app/src/androidTest/java/org/libremediaconverter/fallback/FakeFailures.kt +++ b/app/src/androidTest/java/org/libremediaconverter/fallback/FakeFailures.kt @@ -25,7 +25,7 @@ object FakeFailures { override suspend fun transcode( input: Uri, output: File, - format: OutputFormat, + request: ConversionRequest, onProgress: (Int) -> Unit, ) { called = true diff --git a/app/src/androidTest/java/org/libremediaconverter/fallback/ForcedFailureTest.kt b/app/src/androidTest/java/org/libremediaconverter/fallback/ForcedFailureTest.kt index 654468e..618308a 100644 --- a/app/src/androidTest/java/org/libremediaconverter/fallback/ForcedFailureTest.kt +++ b/app/src/androidTest/java/org/libremediaconverter/fallback/ForcedFailureTest.kt @@ -66,7 +66,7 @@ class ForcedFailureTest { inputUri = Uri.fromFile(input), displayName = SAMPLE, sizeBytes = input.length(), - format = format, + spec = format.spec, quality = QualityTier.FAST, ) diff --git a/app/src/androidTest/java/org/libremediaconverter/fallback/HardwareFallbackTest.kt b/app/src/androidTest/java/org/libremediaconverter/fallback/HardwareFallbackTest.kt index c6f3dac..97c64f3 100644 --- a/app/src/androidTest/java/org/libremediaconverter/fallback/HardwareFallbackTest.kt +++ b/app/src/androidTest/java/org/libremediaconverter/fallback/HardwareFallbackTest.kt @@ -70,7 +70,7 @@ class HardwareFallbackTest { inputUri = Uri.fromFile(input), displayName = SAMPLE, sizeBytes = input.length(), - format = OutputFormat.MP4_H265, + spec = OutputFormat.MP4_H265.spec, // Fast deliberately: this is the tier the router sends to Media3, so it is // the tier where the fallback has to rescue the conversion. quality = QualityTier.FAST, diff --git a/app/src/androidTest/java/org/libremediaconverter/ffmpeg/FFmpegEngineTest.kt b/app/src/androidTest/java/org/libremediaconverter/ffmpeg/FFmpegEngineTest.kt index 56f0c84..40264a2 100644 --- a/app/src/androidTest/java/org/libremediaconverter/ffmpeg/FFmpegEngineTest.kt +++ b/app/src/androidTest/java/org/libremediaconverter/ffmpeg/FFmpegEngineTest.kt @@ -53,7 +53,7 @@ class FFmpegEngineTest { val out = outputFor("out_${format.name.lowercase()}.${format.extension}") runBlocking { engine.run( - request = ConversionRequest(format = format, quality = quality), + request = ConversionRequest(spec = format.spec, quality = quality), inputPath = input.absolutePath, output = out, durationMs = 3_000, @@ -155,7 +155,7 @@ class FFmpegEngineTest { val failure = runCatching { runBlocking { engine.run( - request = ConversionRequest(format = OutputFormat.MP4_H264), + request = ConversionRequest(spec = OutputFormat.MP4_H264.spec), inputPath = "/does/not/exist.mp4", output = out, durationMs = 1_000, diff --git a/app/src/androidTest/java/org/libremediaconverter/work/ConversionWorkerTest.kt b/app/src/androidTest/java/org/libremediaconverter/work/ConversionWorkerTest.kt index 84cf9ff..9b4d5dd 100644 --- a/app/src/androidTest/java/org/libremediaconverter/work/ConversionWorkerTest.kt +++ b/app/src/androidTest/java/org/libremediaconverter/work/ConversionWorkerTest.kt @@ -10,9 +10,12 @@ import androidx.work.WorkInfo import androidx.work.WorkManager import org.libremediaconverter.codec.AndroidDeviceCodecs import org.libremediaconverter.model.Engine +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.Container import org.libremediaconverter.model.OutputFormat -import org.libremediaconverter.model.QualityTier +import org.libremediaconverter.model.OutputSpec import org.libremediaconverter.model.VideoCodec +import org.libremediaconverter.model.QualityTier import kotlinx.coroutines.flow.first import kotlinx.coroutines.runBlocking import kotlinx.coroutines.withTimeout @@ -78,7 +81,7 @@ class ConversionWorkerTest { inputUri = Uri.fromFile(input), displayName = "worker_sample.mp4", sizeBytes = input.length(), - format = OutputFormat.MP4_H264, + spec = OutputFormat.MP4_H264.spec, ) workManager.enqueue(request).result.get() @@ -114,7 +117,7 @@ class ConversionWorkerTest { inputUri = Uri.fromFile(input), displayName = "worker_sample.mp4", sizeBytes = input.length(), - format = OutputFormat.MP3, + spec = OutputFormat.MP3.spec, ) workManager.enqueue(request).result.get() @@ -152,7 +155,7 @@ class ConversionWorkerTest { inputUri = Uri.fromFile(input), displayName = "worker_sample.mp4", sizeBytes = input.length(), - format = OutputFormat.MP4_H265, + spec = OutputFormat.MP4_H265.spec, quality = QualityTier.FAST, ) workManager.enqueue(request).result.get() @@ -181,7 +184,7 @@ class ConversionWorkerTest { inputUri = Uri.fromFile(input), displayName = "worker_sample.mp4", sizeBytes = input.length(), - format = OutputFormat.MP4_H264, + spec = OutputFormat.MP4_H264.spec, quality = QualityTier.BEST, ) workManager.enqueue(request).result.get() @@ -199,9 +202,50 @@ class ConversionWorkerTest { @Test fun outputNameTakesTheExtensionOfTheChosenFormat() { - assertEquals("clip_converted.mp3", ConversionWorker.outputNameFor("clip.mp4", OutputFormat.MP3)) - assertEquals("clip_converted.gif", ConversionWorker.outputNameFor("clip.mov", OutputFormat.GIF)) - assertEquals("clip_converted.mkv", ConversionWorker.outputNameFor("clip", OutputFormat.MKV_H264)) + assertEquals( + "clip_converted.mp3", + ConversionWorker.outputNameFor("clip.mp4", OutputFormat.MP3.spec), + ) + assertEquals( + "clip_converted.gif", + ConversionWorker.outputNameFor("clip.mov", OutputFormat.GIF.spec), + ) + assertEquals( + "clip_converted.mkv", + ConversionWorker.outputNameFor("clip", OutputFormat.MKV_H264.spec), + ) + } + + /** + * The extension follows the container and whether a video track survives. + * + * It used to be a literal on each preset, so the only names reachable were the ones somebody + * had enumerated. Matroska without video is `.mka` and MP4 without video is `.m4a` — neither + * had a preset before, and both are now one tap away in the Advanced picker. + */ + @Test + fun outputNameDistinguishesAudioOnlyVariantsOfAContainer() { + assertEquals( + "clip_converted.mka", + ConversionWorker.outputNameFor( + "clip.mkv", + OutputSpec(Container.MKV, VideoCodec.NONE, AudioCodec.FLAC), + ), + ) + assertEquals( + "clip_converted.m4a", + ConversionWorker.outputNameFor( + "clip.mp4", + OutputSpec(Container.MP4, VideoCodec.NONE, AudioCodec.AAC), + ), + ) + assertEquals( + "clip_converted.mkv", + ConversionWorker.outputNameFor( + "clip.mp4", + OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY), + ), + ) } private companion object { diff --git a/app/src/main/java/org/libremediaconverter/codec/AndroidDeviceCodecs.kt b/app/src/main/java/org/libremediaconverter/codec/AndroidDeviceCodecs.kt index 8f91a82..de630bb 100644 --- a/app/src/main/java/org/libremediaconverter/codec/AndroidDeviceCodecs.kt +++ b/app/src/main/java/org/libremediaconverter/codec/AndroidDeviceCodecs.kt @@ -84,7 +84,10 @@ class AndroidDeviceCodecs private constructor( VideoCodec.VP8 -> MediaFormat.MIMETYPE_VIDEO_VP8 VideoCodec.VP9 -> MediaFormat.MIMETYPE_VIDEO_VP9 VideoCodec.AV1 -> MediaFormat.MIMETYPE_VIDEO_AV1 - VideoCodec.NONE -> null + // Nothing is encoded for either, so there is no encoder to look for. Returning null + // makes canEncode answer true, which is the right answer: a copied or absent track + // places no demand on the hardware. + VideoCodec.COPY, VideoCodec.NONE -> null } /** Maps an FFprobe-style codec name onto a MediaFormat MIME type. */ diff --git a/app/src/main/java/org/libremediaconverter/convert/ConversionViewModel.kt b/app/src/main/java/org/libremediaconverter/convert/ConversionViewModel.kt index 14ef94f..e53a045 100644 --- a/app/src/main/java/org/libremediaconverter/convert/ConversionViewModel.kt +++ b/app/src/main/java/org/libremediaconverter/convert/ConversionViewModel.kt @@ -8,15 +8,25 @@ import androidx.lifecycle.viewModelScope import androidx.media3.common.util.UnstableApi import androidx.work.WorkInfo import androidx.work.WorkManager +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.Container +import org.libremediaconverter.model.ContainerCapabilities import org.libremediaconverter.model.EnginePreference +import org.libremediaconverter.model.InputProbe import org.libremediaconverter.model.OutputFormat +import org.libremediaconverter.model.OutputSpec import org.libremediaconverter.model.QualityTier +import org.libremediaconverter.model.Validation +import org.libremediaconverter.model.VideoCodec import org.libremediaconverter.work.ConversionWorker import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Job import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.SharingStarted import kotlinx.coroutines.flow.StateFlow import kotlinx.coroutines.flow.asStateFlow +import kotlinx.coroutines.flow.combine +import kotlinx.coroutines.flow.stateIn import kotlinx.coroutines.flow.update import kotlinx.coroutines.launch import kotlinx.coroutines.withContext @@ -25,15 +35,27 @@ import java.util.UUID /** User-chosen conversion settings. */ data class ConversionSettings( - val format: OutputFormat = OutputFormat.MP4_H265, + val spec: OutputSpec = OutputFormat.MP4_H265.spec, val quality: QualityTier = QualityTier.FAST, val enginePreference: EnginePreference = EnginePreference.AUTO, -) +) { + /** The preset this spec corresponds to, or null once it has been edited past all of them. */ + val matchingPreset: OutputFormat? + get() = OutputFormat.entries.firstOrNull { it.spec == spec } +} data class InputFile( val uri: Uri, val displayName: String, val sizeBytes: Long, + /** + * What probing found. Null only while the probe is still running. + * + * Held here rather than recomputed because three things need it: the source-info card, the + * validity check for the chosen output, and the copy planner's decision about whether a track + * can be stream-copied. + */ + val probe: InputProbe? = null, ) sealed interface ConversionState { @@ -68,15 +90,52 @@ class ConversionViewModel(app: Application) : AndroidViewModel(app) { private val _settings = MutableStateFlow(ConversionSettings()) val settings: StateFlow = _settings.asStateFlow() - fun setFormat(format: OutputFormat) = _settings.update { it.copy(format = format) } + /** + * Whether the chosen output can actually be produced from the chosen input. + * + * Derived rather than stored so it cannot go stale: it recomputes when either the settings or + * the picked file changes. The Advanced picker deliberately allows an invalid combination to be + * selected, so this is what turns that into an explanation and a disabled Convert button. + */ + val validation: StateFlow = combine(_state, _settings) { state, settings -> + ContainerCapabilities.validate(settings.spec, state.probe() ?: InputProbe()) + }.stateIn(viewModelScope, SharingStarted.Eagerly, Validation.Valid) + + fun setPreset(format: OutputFormat) = _settings.update { it.copy(spec = format.spec) } + fun setContainer(container: Container) = + _settings.update { it.copy(spec = it.spec.copy(container = container)) } + + fun setVideoCodec(codec: VideoCodec) = + _settings.update { it.copy(spec = it.spec.copy(videoCodec = codec)) } + + fun setAudioCodec(codec: AudioCodec) = + _settings.update { it.copy(spec = it.spec.copy(audioCodec = codec)) } + + fun applySuggestion(spec: OutputSpec) = _settings.update { it.copy(spec = spec) } + fun setQuality(quality: QualityTier) = _settings.update { it.copy(quality = quality) } fun setEnginePreference(preference: EnginePreference) = _settings.update { it.copy(enginePreference = preference) } fun onInputPicked(uri: Uri) { viewModelScope.launch { - val info = withContext(Dispatchers.IO) { queryFile(uri) } - _state.value = ConversionState.Ready(info) + // Both the metadata query and the probe touch disk, and the probe spawns FFprobe. + // Neither belongs on the main thread. + val file = withContext(Dispatchers.IO) { queryFile(uri) } + // Show the file as soon as its name and size are known. Probing now runs FFprobe on + // every pick, which is a native process spawn, and making the whole screen wait on it + // would read as the app having ignored the tap. + _state.value = ConversionState.Ready(file) + + val probe = withContext(Dispatchers.IO) { MediaProbe.probe(getApplication(), uri) } + // Only fill in the probe if the user has not moved on in the meantime. + _state.update { current -> + if (current is ConversionState.Ready && current.input.uri == uri) { + ConversionState.Ready(file.copy(probe = probe)) + } else { + current + } + } } } @@ -95,7 +154,7 @@ class ConversionViewModel(app: Application) : AndroidViewModel(app) { inputUri = input.uri, displayName = input.displayName, sizeBytes = input.sizeBytes, - format = settings.format, + spec = settings.spec, quality = settings.quality, enginePreference = settings.enginePreference, ) @@ -169,7 +228,7 @@ class ConversionViewModel(app: Application) : AndroidViewModel(app) { _state.value = ConversionState.Saved( ConversionWorker.outputNameFor( converted.input.displayName, - _settings.value.format, + _settings.value.spec, ) ) }.onFailure { e -> @@ -188,9 +247,17 @@ class ConversionViewModel(app: Application) : AndroidViewModel(app) { fun suggestedOutputName(): String = ConversionWorker.outputNameFor( currentInput()?.displayName ?: "output", - _settings.value.format, + _settings.value.spec, ) + private fun ConversionState.probe(): InputProbe? = when (this) { + is ConversionState.Ready -> input.probe + is ConversionState.Converting -> input.probe + is ConversionState.Waiting -> input.probe + is ConversionState.Converted -> input.probe + else -> null + } + private fun currentInput(): InputFile? = when (val s = _state.value) { is ConversionState.Ready -> s.input is ConversionState.Converting -> s.input diff --git a/app/src/main/java/org/libremediaconverter/convert/ConverterScreen.kt b/app/src/main/java/org/libremediaconverter/convert/ConverterScreen.kt index 6fde369..12c999f 100644 --- a/app/src/main/java/org/libremediaconverter/convert/ConverterScreen.kt +++ b/app/src/main/java/org/libremediaconverter/convert/ConverterScreen.kt @@ -3,6 +3,7 @@ package org.libremediaconverter.convert import android.Manifest import androidx.activity.compose.rememberLauncherForActivityResult import androidx.activity.result.contract.ActivityResultContracts +import androidx.compose.animation.AnimatedVisibility import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.ExperimentalLayoutApi @@ -16,13 +17,19 @@ import androidx.compose.foundation.verticalScroll import androidx.compose.material3.AssistChip import androidx.compose.material3.Button import androidx.compose.material3.Card +import androidx.compose.material3.CardDefaults import androidx.compose.material3.FilterChip +import androidx.compose.material3.HorizontalDivider import androidx.compose.material3.LinearProgressIndicator import androidx.compose.material3.MaterialTheme import androidx.compose.material3.OutlinedButton import androidx.compose.material3.Text +import androidx.compose.material3.TextButton import androidx.compose.runtime.Composable import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.saveable.rememberSaveable +import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.text.style.TextAlign @@ -33,9 +40,16 @@ import org.libremediaconverter.ui.ScreenPaddingVertical import androidx.lifecycle.compose.collectAsStateWithLifecycle import androidx.lifecycle.viewmodel.compose.viewModel import androidx.media3.common.util.UnstableApi +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.CodecNames +import org.libremediaconverter.model.Container import org.libremediaconverter.model.EnginePreference +import org.libremediaconverter.model.InputKind import org.libremediaconverter.model.OutputFormat +import org.libremediaconverter.model.OutputSpec import org.libremediaconverter.model.QualityTier +import org.libremediaconverter.model.Validation +import org.libremediaconverter.model.VideoCodec import java.util.Locale @UnstableApi @@ -46,6 +60,7 @@ fun ConverterScreen( ) { val state by viewModel.state.collectAsStateWithLifecycle() val settings by viewModel.settings.collectAsStateWithLifecycle() + val validation by viewModel.validation.collectAsStateWithLifecycle() // ACTION_OPEN_DOCUMENT rather than the photo picker: the picker is images and video // only, offers no audio at all, and will not reliably surface .mkv/.flac/.webm. @@ -55,7 +70,7 @@ fun ConverterScreen( ) { uri -> uri?.let(viewModel::onInputPicked) } val chooseDestination = rememberLauncherForActivityResult( - ActivityResultContracts.CreateDocument(settings.format.mimeType) + ActivityResultContracts.CreateDocument(settings.spec.mimeType) ) { uri -> uri?.let(viewModel::save) } // Requested at the point of use rather than on first launch, so the ask carries its @@ -110,13 +125,24 @@ fun ConverterScreen( is ConversionState.Ready -> { FileCard(s.input) - FormatPicker(settings.format, viewModel::setFormat) + FormatPicker(settings.matchingPreset, viewModel::setPreset) + AdvancedPicker( + spec = settings.spec, + validation = validation, + onContainer = viewModel::setContainer, + onVideoCodec = viewModel::setVideoCodec, + onAudioCodec = viewModel::setAudioCodec, + onSuggestion = viewModel::applySuggestion, + ) QualityPicker(settings.quality, viewModel::setQuality) EnginePicker(settings.enginePreference, viewModel::setEnginePreference) Button( onClick = { requestNotifications.launch(Manifest.permission.POST_NOTIFICATIONS) }, + // The Advanced picker lets an impossible combination be selected on + // purpose, so this is what stops it from being run. + enabled = validation.isValid, modifier = Modifier.fillMaxWidth().height(PrimaryButtonHeight), ) { Text("Convert") } OutlinedButton( @@ -159,8 +185,9 @@ fun ConverterScreen( ) if (s.routeReason.isNotBlank()) { // Surfacing the routing decision rather than hiding it: it - // explains why a job was slow, and makes the software - // fallback visible. + // explains why a job was slow, makes the software fallback + // visible, and is how the user learns a remux happened rather + // than a re-encode. AssistChip(onClick = {}, label = { Text(s.routeReason) }) } Button( @@ -200,7 +227,7 @@ fun ConverterScreen( @OptIn(ExperimentalLayoutApi::class) @Composable -private fun FormatPicker(selected: OutputFormat, onSelect: (OutputFormat) -> Unit) { +private fun FormatPicker(selected: OutputFormat?, onSelect: (OutputFormat) -> Unit) { Text("Output format", style = MaterialTheme.typography.titleSmall) FlowRow(horizontalArrangement = Arrangement.spacedBy(8.dp)) { OutputFormat.entries.forEach { format -> @@ -211,6 +238,128 @@ private fun FormatPicker(selected: OutputFormat, onSelect: (OutputFormat) -> Uni ) } } + if (selected == null) { + Text( + "Custom — set below.", + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + } +} + +/** + * The full container × codec matrix. + * + * Every combination stays selectable, including the ones that cannot work. Disabling or hiding + * them would leave the user guessing why the option they wanted is not there; letting them pick it + * and then saying what is wrong — and what would work instead — teaches the constraint. The + * Convert button is what actually blocks the job. + */ +@OptIn(ExperimentalLayoutApi::class) +@Composable +private fun AdvancedPicker( + spec: OutputSpec, + validation: Validation, + onContainer: (Container) -> Unit, + onVideoCodec: (VideoCodec) -> Unit, + onAudioCodec: (AudioCodec) -> Unit, + onSuggestion: (OutputSpec) -> Unit, +) { + var expanded by rememberSaveable { mutableStateOf(false) } + + TextButton(onClick = { expanded = !expanded }) { + Text(if (expanded) "Hide advanced" else "Advanced") + } + + AnimatedVisibility(visible = expanded) { + Column(verticalArrangement = Arrangement.spacedBy(12.dp)) { + Text("Container", style = MaterialTheme.typography.titleSmall) + FlowRow(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + Container.entries.forEach { container -> + FilterChip( + selected = container == spec.container, + onClick = { onContainer(container) }, + label = { Text(container.label) }, + ) + } + } + + Text("Video", style = MaterialTheme.typography.titleSmall) + FlowRow(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + VideoCodec.entries.forEach { codec -> + FilterChip( + selected = codec == spec.videoCodec, + onClick = { onVideoCodec(codec) }, + label = { Text(codec.label) }, + ) + } + } + + Text("Audio", style = MaterialTheme.typography.titleSmall) + FlowRow(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + AudioCodec.entries.forEach { codec -> + FilterChip( + selected = codec == spec.audioCodec, + onClick = { onAudioCodec(codec) }, + label = { Text(codec.label) }, + ) + } + } + + Text( + "Copy keeps the original stream — no re-encoding, so it finishes in seconds.", + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + } + } + + if (validation is Validation.Invalid) { + ValidationError(validation, onSuggestion) + } +} + +@OptIn(ExperimentalLayoutApi::class) +@Composable +private fun ValidationError(invalid: Validation.Invalid, onSuggestion: (OutputSpec) -> Unit) { + Card( + modifier = Modifier.fillMaxWidth(), + colors = CardDefaults.cardColors( + containerColor = MaterialTheme.colorScheme.errorContainer, + contentColor = MaterialTheme.colorScheme.onErrorContainer, + ), + ) { + Column( + modifier = Modifier.padding(16.dp), + verticalArrangement = Arrangement.spacedBy(8.dp), + ) { + Text(invalid.message, style = MaterialTheme.typography.bodyMedium) + if (invalid.suggestions.isNotEmpty()) { + Text("Try instead:", style = MaterialTheme.typography.labelMedium) + FlowRow(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + invalid.suggestions.forEach { suggestion -> + AssistChip( + onClick = { onSuggestion(suggestion) }, + label = { Text(describe(suggestion)) }, + ) + } + } + } + } + } +} + +private fun describe(spec: OutputSpec): String { + val video = when (spec.videoCodec) { + VideoCodec.NONE -> null + else -> spec.videoCodec.label + } + val audio = when (spec.audioCodec) { + AudioCodec.NONE -> null + else -> spec.audioCodec.label + } + val tracks = listOfNotNull(video, audio).joinToString(" + ") + return if (tracks.isEmpty()) spec.container.label else "${spec.container.label} · $tracks" } @OptIn(ExperimentalLayoutApi::class) @@ -250,16 +399,87 @@ private fun EnginePreference.label(): String = when (this) { EnginePreference.FORCE_SOFTWARE -> "Force software" } +/** + * Name, size, and what the file actually turned out to be. + * + * The codec lines are what make "Copy" a meaningful choice — without knowing the source is H.264, + * "copy the video" is a guess. They degrade explicitly rather than silently: an audio file says so + * instead of showing a blank video row, and a file nothing could read says that rather than + * pretending it has an unknown codec. + */ @Composable private fun FileCard(input: InputFile) { Card(modifier = Modifier.fillMaxWidth()) { Column(modifier = Modifier.padding(16.dp)) { Text(input.displayName, style = MaterialTheme.typography.titleMedium) Text(formatBytes(input.sizeBytes), style = MaterialTheme.typography.bodySmall) + + val probe = input.probe + if (probe == null) { + Text("Reading…", style = MaterialTheme.typography.bodySmall) + return@Column + } + + HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp)) + + when (probe.kind) { + InputKind.UNPARSEABLE -> Text( + "Could not identify this file. It will be converted with FFmpeg.", + style = MaterialTheme.typography.bodySmall, + ) + + InputKind.IMAGE -> { + DetailRow("Type", "Image") + if (probe.width > 0) DetailRow("Size", "${probe.width}×${probe.height}") + } + + InputKind.AUDIO_ONLY -> { + DetailRow("Container", probe.container?.label ?: "Unknown") + DetailRow("Video", "No video track") + DetailRow("Audio", CodecNames.describeAudio(probe.audioCodec)) + if (probe.durationMs > 0) DetailRow("Length", formatDuration(probe.durationMs)) + } + + InputKind.VIDEO -> { + DetailRow("Container", probe.container?.label ?: "Unknown") + DetailRow( + "Video", + buildString { + append(CodecNames.describeVideo(probe.videoCodec)) + if (probe.width > 0) append(" · ${probe.width}×${probe.height}") + }, + ) + DetailRow( + "Audio", + if (probe.audioCodec == null) { + "No audio track" + } else { + CodecNames.describeAudio(probe.audioCodec) + }, + ) + if (probe.durationMs > 0) DetailRow("Length", formatDuration(probe.durationMs)) + } + } } } } +@Composable +private fun DetailRow(label: String, value: String) { + Text( + "$label: $value", + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) +} + +private fun formatDuration(ms: Long): String { + val totalSeconds = ms / 1000 + val minutes = totalSeconds / 60 + val seconds = totalSeconds % 60 + return String.format(Locale.US, "%d:%02d", minutes, seconds) +} + private fun formatBytes(bytes: Long): String = when { bytes >= 1_000_000_000 -> String.format(Locale.US, "%.1f GB", bytes / 1e9) bytes >= 1_000_000 -> String.format(Locale.US, "%.1f MB", bytes / 1e6) diff --git a/app/src/main/java/org/libremediaconverter/convert/Media3Engine.kt b/app/src/main/java/org/libremediaconverter/convert/Media3Engine.kt index 0c78040..71d1b79 100644 --- a/app/src/main/java/org/libremediaconverter/convert/Media3Engine.kt +++ b/app/src/main/java/org/libremediaconverter/convert/Media3Engine.kt @@ -9,6 +9,7 @@ import androidx.media3.common.MimeTypes import androidx.media3.common.util.UnstableApi import androidx.media3.transformer.Composition import androidx.media3.transformer.EditedMediaItem +import androidx.media3.transformer.EditedMediaItemSequence import androidx.media3.transformer.ExportException import androidx.media3.transformer.ExportResult import androidx.media3.transformer.ProgressHolder @@ -16,8 +17,12 @@ import androidx.media3.transformer.Transformer import kotlinx.coroutines.CancellableContinuation import kotlinx.coroutines.suspendCancellableCoroutine import org.libremediaconverter.model.AudioCodec -import org.libremediaconverter.model.OutputFormat +import org.libremediaconverter.model.AudioPlan +import org.libremediaconverter.model.ConversionPlan +import org.libremediaconverter.model.ConversionRequest +import org.libremediaconverter.model.CopyPlanner import org.libremediaconverter.model.VideoCodec +import org.libremediaconverter.model.VideoPlan import java.io.File import kotlin.coroutines.resume import kotlin.coroutines.resumeWithException @@ -58,18 +63,27 @@ class Media3Engine(private val context: Context) : HardwareTranscoder { override suspend fun transcode( input: Uri, output: File, - format: OutputFormat, + request: ConversionRequest, onProgress: (Int) -> Unit, ): Unit = suspendCancellableCoroutine { cont -> + val plan = CopyPlanner.plan(request.spec, request.probe) handler.post { - val transformer = runCatching { buildTransformer(format, cont) } + val transformer = runCatching { buildTransformer(plan, cont) } .getOrElse { cont.resumeWithException(it); return@post } - // Dropping the tracks the target format does not have is what stops an audio-only - // export from carrying a re-encoded video track. Without setRemoveVideo, asking for - // M4A produced an HEVC stream in a file named .m4a. + + // Dropping the tracks the target does not have is what stops an audio-only export + // from carrying a re-encoded video track. Without setRemoveVideo, asking for M4A + // produced an HEVC stream in a file named .m4a. val item = EditedMediaItem.Builder(MediaItem.fromUri(input)) - .setRemoveVideo(format.videoCodec == VideoCodec.NONE) - .setRemoveAudio(format.audioCodec == AudioCodec.NONE) + .setRemoveVideo(plan.video == VideoPlan.Drop) + .setRemoveAudio(plan.audio == AudioPlan.Drop) + .build() + + // A Composition is the only way to ask for transmuxing; the plain + // start(EditedMediaItem, path) overload always re-encodes. This is the remux path. + val composition = Composition.Builder(EditedMediaItemSequence.Builder(item).build()) + .setTransmuxVideo(plan.video == VideoPlan.Copy) + .setTransmuxAudio(plan.audio == AudioPlan.Copy) .build() cont.invokeOnCancellation { @@ -77,7 +91,7 @@ class Media3Engine(private val context: Context) : HardwareTranscoder { handler.post { runCatching { transformer.cancel() } } } - runCatching { transformer.start(item, output.absolutePath) } + runCatching { transformer.start(composition, output.absolutePath) } .onFailure { cont.resumeWithException(it); return@post } pollProgress(transformer, cont, onProgress) @@ -85,27 +99,28 @@ class Media3Engine(private val context: Context) : HardwareTranscoder { } /** - * @throws IllegalArgumentException if [format] names a container Media3 cannot mux. That is a + * @throws IllegalArgumentException if [plan] names a container Media3 cannot mux. That is a * routing bug rather than a runtime condition — [org.libremediaconverter.model.ConversionRouter] * is supposed to have sent such a job to FFmpeg — so it fails loudly instead of quietly * writing MP4, which is what the old code did. */ private fun buildTransformer( - format: OutputFormat, + plan: ConversionPlan, cont: CancellableContinuation, ): Transformer { - val muxerFactory = requireNotNull(Media3Muxers.factoryFor(format.container)) { - "Media3 cannot mux ${format.container}; this job should have routed to FFmpeg." + val muxerFactory = requireNotNull(Media3Muxers.factoryFor(plan.container)) { + "Media3 cannot mux ${plan.container}; this job should have routed to FFmpeg." } val builder = Transformer.Builder(context) .setLooper(thread.looper) .setMuxerFactory(muxerFactory) - // Only name a MIME type for a track the output actually keeps. Naming one for a removed + // Name a MIME type only for a track that is actually being encoded. Setting one for a + // transmuxed track contradicts setTransmuxVideo/Audio, and setting one for a removed // track makes Transformer build an encoder for samples that will never arrive. - videoMimeTypeFor(format.videoCodec)?.let(builder::setVideoMimeType) - audioMimeTypeFor(format.audioCodec)?.let(builder::setAudioMimeType) + (plan.video as? VideoPlan.Encode)?.let { videoMimeTypeFor(it.codec)?.let(builder::setVideoMimeType) } + (plan.audio as? AudioPlan.Encode)?.let { audioMimeTypeFor(it.codec)?.let(builder::setAudioMimeType) } return builder .addListener(object : Transformer.Listener { @@ -134,7 +149,8 @@ class Media3Engine(private val context: Context) : HardwareTranscoder { private fun videoMimeTypeFor(codec: VideoCodec): String? = when (codec) { VideoCodec.H264 -> MimeTypes.VIDEO_H264 VideoCodec.H265 -> MimeTypes.VIDEO_H265 - VideoCodec.NONE -> null + // Never reached: only an Encode plan consults this, and COPY/NONE are not Encode. + VideoCodec.COPY, VideoCodec.NONE -> null VideoCodec.VP8, VideoCodec.VP9, VideoCodec.AV1 -> null } @@ -143,7 +159,7 @@ class Media3Engine(private val context: Context) : HardwareTranscoder { AudioCodec.OPUS -> MimeTypes.AUDIO_OPUS AudioCodec.VORBIS -> MimeTypes.AUDIO_VORBIS AudioCodec.PCM -> MimeTypes.AUDIO_RAW - AudioCodec.NONE -> null + AudioCodec.COPY, AudioCodec.NONE -> null // MP3 and FLAC have no Android encoder; the router routes them to FFmpeg. AudioCodec.MP3, AudioCodec.FLAC -> null } diff --git a/app/src/main/java/org/libremediaconverter/convert/Media3Muxers.kt b/app/src/main/java/org/libremediaconverter/convert/Media3Muxers.kt index 5417186..9fc4786 100644 --- a/app/src/main/java/org/libremediaconverter/convert/Media3Muxers.kt +++ b/app/src/main/java/org/libremediaconverter/convert/Media3Muxers.kt @@ -49,9 +49,16 @@ object Media3Muxers { Container.OGG -> OggFactory Container.WAV -> WavFactory Container.AAC_ADTS -> AacFactory - // Matroska, MP3 and the image outputs have no Media3 muxer. FFmpeg owns them. + // Everything else has no Media3 muxer. Matroska and the legacy containers are FFmpeg's, + // and `Mp4Muxer` exposes no QuickTime file format, so MOV is too. Container.MKV, + Container.MOV, + Container.MPEG_TS, + Container.AVI, + Container.FLV, + Container.ASF, Container.MP3, + Container.FLAC, Container.GIF, Container.IMAGE_SEQUENCE, -> null diff --git a/app/src/main/java/org/libremediaconverter/convert/MediaProbe.kt b/app/src/main/java/org/libremediaconverter/convert/MediaProbe.kt index 20889c1..c4f73c7 100644 --- a/app/src/main/java/org/libremediaconverter/convert/MediaProbe.kt +++ b/app/src/main/java/org/libremediaconverter/convert/MediaProbe.kt @@ -5,26 +5,99 @@ import android.media.MediaExtractor import android.media.MediaFormat import android.net.Uri import android.util.Log +import com.arthenica.ffmpegkit.FFmpegKitConfig +import com.arthenica.ffmpegkit.FFprobeKit +import com.arthenica.ffmpegkit.MediaInformation import org.libremediaconverter.model.ConcatInput +import org.libremediaconverter.model.Container +import org.libremediaconverter.model.InputKind import org.libremediaconverter.model.InputProbe /** - * Reads just enough about an input to route it. + * Reads what the router, the copy planner and the source-info card each need to know. * - * Uses the platform extractor rather than FFprobe: it needs no native library, works - * directly on a content:// URI, and the router only needs the codec and duration. If - * the platform cannot parse the file at all, that is itself the answer — a file - * MediaExtractor cannot open is one Media3 cannot convert, so it routes to FFmpeg. + * ## Two probes, deliberately + * + * The platform extractor answers the routing question directly: a file `MediaExtractor` cannot open + * is a file Media3 cannot convert, so a failure there is itself the answer. It needs no native + * library and works straight off a `content://` URI. + * + * What it cannot do is name the *container*. There is no `MediaExtractor` API for it at all — and + * [org.libremediaconverter.model.CopyPlanner] needs the source container to tell a remux (change the + * container, keep the streams) from a re-encode of something already in the target container. It + * also refuses to open images, which the picker has to describe rather than reject. + * + * So FFprobe runs too, and the two are merged. FFprobe is not a fallback here: on the common path, + * where the extractor succeeds, it is still the only source of the container name. That costs a + * native process spawn per file pick, which is why callers run this off the main thread. */ object MediaProbe { fun probe(context: Context, uri: Uri): InputProbe { + val extracted = probeWithExtractor(context, uri) + val info = probeWithFFprobe(context, uri) + + val videoCodec = extracted?.videoCodec ?: info?.videoCodec + val audioCodec = extracted?.audioCodec ?: info?.audioCodec + val kind = classify(extracted, info) + + if (kind == InputKind.UNPARSEABLE) { + // Not a failure: an unparseable input is a strong signal that this job belongs on + // FFmpeg. Reporting an unknown codec makes the router say so. + Log.i(TAG, "Neither MediaExtractor nor FFprobe could read $uri; routing to FFmpeg.") + return InputProbe( + videoCodec = InputProbe.UNPARSEABLE, + hasVideo = true, + durationMs = 0, + kind = InputKind.UNPARSEABLE, + ) + } + + return InputProbe( + videoCodec = videoCodec, + audioCodec = audioCodec, + hasVideo = videoCodec != null, + durationMs = maxOf(extracted?.durationMs ?: 0L, info?.durationMs ?: 0L), + kind = kind, + container = info?.container, + width = extracted?.width ?: info?.width ?: 0, + height = extracted?.height ?: info?.height ?: 0, + ) + } + + /** + * Distinguishes "no video track" from "could not parse". + * + * The old code collapsed both into `hasVideo = true, videoCodec = UNPARSEABLE`, which made an + * audio file and a corrupt file indistinguishable. The source-info card cannot describe either + * honestly until they are separate, and neither can the copy planner. + */ + private fun classify(extracted: Extracted?, info: FFprobeInfo?): InputKind = when { + info?.isImage == true -> InputKind.IMAGE + extracted == null && info == null -> InputKind.UNPARSEABLE + (extracted?.videoCodec ?: info?.videoCodec) != null -> InputKind.VIDEO + (extracted?.audioCodec ?: info?.audioCodec) != null -> InputKind.AUDIO_ONLY + // Parsed, but with no stream either probe recognised. Nothing to convert. + else -> InputKind.UNPARSEABLE + } + + private class Extracted( + val videoCodec: String?, + val audioCodec: String?, + val durationMs: Long, + val width: Int, + val height: Int, + ) + + private fun probeWithExtractor(context: Context, uri: Uri): Extracted? { val extractor = MediaExtractor() return try { extractor.setDataSource(context, uri, null) var video: String? = null var audio: String? = null var durationUs = 0L + var width = 0 + var height = 0 for (i in 0 until extractor.trackCount) { val format = extractor.getTrackFormat(i) @@ -33,31 +106,132 @@ object MediaProbe { durationUs = maxOf(durationUs, format.getLong(MediaFormat.KEY_DURATION)) } when { - mime.startsWith("video/") && video == null -> video = shortName(mime) + mime.startsWith("video/") && video == null -> { + video = shortName(mime) + width = format.intOr(MediaFormat.KEY_WIDTH) + height = format.intOr(MediaFormat.KEY_HEIGHT) + } + mime.startsWith("audio/") && audio == null -> audio = shortName(mime) } } - - InputProbe( - videoCodec = video, - audioCodec = audio, - hasVideo = video != null, - durationMs = durationUs / 1000, - ) + Extracted(video, audio, durationUs / 1000, width, height) } catch (e: Exception) { - // Not a failure: an unparseable input is a strong signal that this job - // belongs on FFmpeg. Reporting an unknown codec makes the router say so. - Log.i(TAG, "Platform extractor could not read $uri; routing to FFmpeg.", e) - InputProbe(videoCodec = InputProbe.UNPARSEABLE, hasVideo = true, durationMs = 0) + Log.i(TAG, "Platform extractor could not read $uri.", e) + null } finally { runCatching { extractor.release() } } } + private class FFprobeInfo( + val container: Container?, + val videoCodec: String?, + val audioCodec: String?, + val durationMs: Long, + val width: Int, + val height: Int, + val isImage: Boolean, + ) + + private fun probeWithFFprobe(context: Context, uri: Uri): FFprobeInfo? = try { + // The same SAF bridge ConversionWorker uses for the FFmpeg read side. + val path = if (uri.scheme == "content") { + FFmpegKitConfig.getSafParameterForRead(context, uri) + } else { + uri.path + } + path?.let { readMediaInformation(it) } + } catch (e: Exception) { + Log.i(TAG, "FFprobe could not read $uri.", e) + null + } + + private fun readMediaInformation(path: String): FFprobeInfo? { + // ffmpeg-kit-next is compiled from Kotlin with private backing fields, so these have to go + // through the Java getters rather than property syntax. + val info: MediaInformation = FFprobeKit.getMediaInformation(path).getMediaInformation() + ?: return null + + val streams = info.getStreams().orEmpty() + val video = streams.firstOrNull { it.getType() == "video" } + val audio = streams.firstOrNull { it.getType() == "audio" } + val formatName = info.getFormat().orEmpty() + + return FFprobeInfo( + container = containerFrom(formatName, video?.getCodec()), + videoCodec = video?.getCodec(), + audioCodec = audio?.getCodec(), + durationMs = info.getDuration()?.toDoubleOrNull()?.times(1000)?.toLong() ?: 0L, + width = video?.getWidth()?.toInt() ?: 0, + height = video?.getHeight()?.toInt() ?: 0, + isImage = isImageFormat(formatName), + ) + } + + /** + * Maps FFprobe's `format_name` onto a [Container]. + * + * FFprobe reports a comma-separated list of every format that shares the demuxer, so a plain + * MP4 comes back as `mov,mp4,m4a,3gp,3g2,mj2` — the first entry is not authoritative and the + * whole string never equals one container name. Matching against the set is the only correct + * reading, and getting this wrong silently disables the remux fast path rather than failing. + * + * ## Matroska and WebM are genuinely indistinguishable here + * + * WebM *is* a Matroska profile and they share a demuxer, so FFprobe reports `matroska,webm` for + * both — a `.mkv` of H.264 and a `.webm` of VP9 give byte-identical format names, which the + * committed fixtures confirm. The only signal left is the codec: WebM permits VP8/VP9/AV1 and + * nothing else, so anything outside that set is certainly Matroska. A VP9 file could still be + * either, and is reported as WebM, which is right far more often than not. + * + * That residual ambiguity is safe for the copy planner: each container accepts the codecs the + * other holds, so a wrong guess still reaches a valid decision and only the label on the + * source-info card suffers. + * + * Returns null for anything unrecognised, which the copy planner treats as "cannot prove the + * container is changing" and therefore declines to upgrade to a stream copy. + */ + internal fun containerFrom(formatName: String, videoCodec: String? = null): Container? { + val names = formatName.split(',').map { it.trim().lowercase() }.filter { it.isNotEmpty() } + if (names.isEmpty()) return null + + // Order matters: the MP4 demuxer also claims mov, so check the more specific membership + // first or every MP4 is reported as QuickTime. + return when { + "matroska" in names || "webm" in names -> matroskaOrWebm(videoCodec) + "mp4" in names -> Container.MP4 + "mov" in names || "qt" in names -> Container.MOV + "mpegts" in names || "mpegtsraw" in names -> Container.MPEG_TS + "avi" in names -> Container.AVI + "flv" in names -> Container.FLV + "asf" in names || "asf_o" in names -> Container.ASF + "ogg" in names -> Container.OGG + "wav" in names -> Container.WAV + "aac" in names || "adts" in names -> Container.AAC_ADTS + "mp3" in names -> Container.MP3 + "flac" in names -> Container.FLAC + "gif" in names -> Container.GIF + else -> null + } + } + + /** WebM's codec whitelist is the only thing separating it from Matroska. See [containerFrom]. */ + private fun matroskaOrWebm(videoCodec: String?): Container = when (videoCodec?.lowercase()) { + "vp8", "vp9", "vp09", "av1", "av01" -> Container.WEBM + else -> Container.MKV + } + + /** FFprobe describes still images through the image demuxers rather than a media container. */ + private fun isImageFormat(formatName: String): Boolean { + val names = formatName.split(',').map { it.trim().lowercase() } + return names.any { it == "image2" || it.endsWith("_pipe") } + } + /** * Reads the properties that decide whether inputs can be joined without - * re-encoding. Unknown values stay null, which [ConcatPlanner] treats as "cannot - * prove a match" rather than as agreement. + * re-encoding. Unknown values stay null, which [org.libremediaconverter.model.ConcatPlanner] + * treats as "cannot prove a match" rather than as agreement. */ fun probeForConcat(context: Context, uri: Uri): ConcatInput { val extractor = MediaExtractor() @@ -105,6 +279,7 @@ object MediaProbe { MediaFormat.MIMETYPE_AUDIO_OPUS -> "opus" MediaFormat.MIMETYPE_AUDIO_FLAC -> "flac" MediaFormat.MIMETYPE_AUDIO_VORBIS -> "vorbis" + MediaFormat.MIMETYPE_AUDIO_RAW -> "pcm" else -> mime.substringAfter('/') } diff --git a/app/src/main/java/org/libremediaconverter/convert/Transcoders.kt b/app/src/main/java/org/libremediaconverter/convert/Transcoders.kt index ac284d1..ea7f810 100644 --- a/app/src/main/java/org/libremediaconverter/convert/Transcoders.kt +++ b/app/src/main/java/org/libremediaconverter/convert/Transcoders.kt @@ -12,16 +12,17 @@ import java.io.File /** The hardware conversion path. Implemented by [Media3Engine]. */ interface HardwareTranscoder : AutoCloseable { /** - * Takes the whole [OutputFormat] rather than just a video MIME type. + * Takes the whole [ConversionRequest] rather than just a video MIME type. * * The narrower signature was the reason "extract audio to M4A" produced an HEVC video track: * the container, the audio codec and "this output has no video at all" had nowhere to travel, - * so the engine defaulted all three. + * so the engine defaulted all three. Passing the request also carries the input probe, which + * is what `CopyPlanner` needs to decide whether a track can be transmuxed. */ suspend fun transcode( input: Uri, output: File, - format: OutputFormat = OutputFormat.MP4_H265, + request: ConversionRequest = ConversionRequest(OutputFormat.MP4_H265.spec), onProgress: (Int) -> Unit = {}, ) } diff --git a/app/src/main/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilder.kt b/app/src/main/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilder.kt index d76b653..4dde3ba 100644 --- a/app/src/main/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilder.kt +++ b/app/src/main/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilder.kt @@ -1,8 +1,15 @@ package org.libremediaconverter.ffmpeg +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.AudioPlan +import org.libremediaconverter.model.CodecNames +import org.libremediaconverter.model.Container +import org.libremediaconverter.model.ConversionPlan import org.libremediaconverter.model.ConversionRequest -import org.libremediaconverter.model.OutputFormat +import org.libremediaconverter.model.CopyPlanner import org.libremediaconverter.model.QualityTier +import org.libremediaconverter.model.VideoCodec +import org.libremediaconverter.model.VideoPlan /** * Builds FFmpeg argument lists. @@ -13,6 +20,9 @@ import org.libremediaconverter.model.QualityTier * * Arguments are produced as a list rather than a shell string: paths routinely contain * spaces, and a list has no quoting rules to get wrong. + * + * Every decision comes from [CopyPlanner], so "copy this track" and "re-encode that one" are + * settled before any flag is chosen, and the same plan drives the routing decision. */ object FFmpegCommandBuilder { @@ -34,122 +44,158 @@ object FFmpegCommandBuilder { * "Invalid to call at Released state" partway through the export — and hardware * players reject 4:4:4 output anyway. Naming the pixel format makes FFmpeg insert * the conversion instead of failing. + * + * Deliberately absent from the copy path: a stream copy never decodes frames, so there is + * no pixel format to convert and the flag would be meaningless. */ private val PIX_FMT = listOf("-pix_fmt", "yuv420p") + /** Containers in the ISO base-media family, where HEVC needs the hvc1 brand. */ + private val MP4_FAMILY = setOf(Container.MP4, Container.MOV) + fun build( request: ConversionRequest, inputPath: String, outputPath: String, - ): List = buildList { - add("-hide_banner") - // Overwrite: the output path is one we just created in our own cache. - add("-y") - add("-i"); add(inputPath) + ): List { + val plan = CopyPlanner.plan(request.spec, request.probe) + return buildList { + add("-hide_banner") + // Overwrite: the output path is one we just created in our own cache. + add("-y") + add("-i"); add(inputPath) - addAll(streamSelection(request.format)) - addAll(videoArgs(request)) - addAll(audioArgs(request.format)) - addAll(containerArgs(request.format)) - - add(outputPath) - } - - private fun streamSelection(format: OutputFormat): List = when { - // -vn drops video entirely. Without it FFmpeg will happily try to carry a video - // stream into an audio container and fail at the muxer. - format.isAudioOnly -> listOf("-vn") - format == OutputFormat.GIF || format == OutputFormat.FRAMES_PNG -> listOf("-an") - else -> emptyList() - } - - private fun videoArgs(request: ConversionRequest): List { - val format = request.format - if (format.isAudioOnly) return emptyList() - - return when (format) { - OutputFormat.GIF -> listOf( - // One pass with a generated palette. GIF is limited to 256 colours, and - // the default palette produces visibly banded output; split+palettegen - // and paletteuse in a single graph avoids a temporary palette file. - "-vf", - "fps=12,scale=480:-1:flags=lanczos,split[a][b];" + - "[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer", - "-loop", "0", - ) - - OutputFormat.FRAMES_PNG -> listOf("-vf", "fps=1", "-vsync", "0") - - else -> when (request.quality) { - // Software encoding: this is what the GPL licence buys. CRF targets a - // quality level and lets the bitrate fall where it may, which is what - // "compress this well" actually needs. No hardware encoder on Android - // exposes it. - QualityTier.BEST -> when (format.videoCodec) { - org.libremediaconverter.model.VideoCodec.H265 -> listOf( - "-c:v", "libx265", "-crf", "$CRF_H265", "-preset", "medium", - // Without this, many players and Apple devices refuse HEVC in MP4. - "-tag:v", "hvc1", - ) + PIX_FMT - org.libremediaconverter.model.VideoCodec.VP9 -> listOf( - "-c:v", "libvpx-vp9", "-crf", "31", "-b:v", "0", - ) + PIX_FMT - else -> listOf( - "-c:v", "libx264", "-crf", "$CRF_H264", "-preset", "medium", - ) + PIX_FMT - } - - // Software encoding, always. - // - // FFmpeg's *_mediacodec encoders used to be selected here, on the theory - // that a job routed to FFmpeg for container reasons could still encode in - // hardware. In practice they are undocumented, per-device flaky, and were - // observed failing twice on real footage on a Pixel 10 Pro XL -- once - // binding to a software codec while claiming to be the fast path, and once - // dying mid-export with "Error submitting video frame to the encoder" even - // after the pixel format was pinned. - // - // They also duplicate, badly, something Media3 already does properly. A - // job only reaches FFmpeg because Media3 could not handle it, which is - // itself evidence that hardware encoding is unlikely to work for that - // input. Fast therefore means a fast *preset*, not a different encoder. - QualityTier.FAST -> when (format.videoCodec) { - org.libremediaconverter.model.VideoCodec.H265 -> listOf( - "-c:v", "libx265", "-crf", "$CRF_H265", "-preset", "veryfast", - "-tag:v", "hvc1", - ) + PIX_FMT - org.libremediaconverter.model.VideoCodec.VP9 -> listOf( - "-c:v", "libvpx-vp9", "-crf", "31", "-b:v", "0", - "-deadline", "realtime", - ) + PIX_FMT - else -> listOf( - "-c:v", "libx264", "-crf", "$CRF_H264", "-preset", "veryfast", - ) + PIX_FMT - } + if (request.spec.isImageOutput) { + addAll(imageArgs(request)) + } else { + addAll(videoArgs(plan, request)) + addAll(audioArgs(plan)) } + addAll(containerArgs(plan)) + + add(outputPath) } } - private fun audioArgs(format: OutputFormat): List = when (format) { - OutputFormat.MP3 -> listOf("-c:a", "libmp3lame", "-q:a", "2") - OutputFormat.FLAC -> listOf("-c:a", "flac") - OutputFormat.WAV -> listOf("-c:a", "pcm_s16le") - OutputFormat.OPUS -> listOf("-c:a", "libopus", "-b:a", "128k") - OutputFormat.M4A_AAC -> listOf("-c:a", "aac", "-b:a", "192k") - OutputFormat.GIF, OutputFormat.FRAMES_PNG -> emptyList() - OutputFormat.WEBM_VP9 -> listOf("-c:a", "libopus", "-b:a", "128k") - else -> listOf("-c:a", "aac", "-b:a", "192k") + private fun imageArgs(request: ConversionRequest): List = when (request.container) { + Container.GIF -> listOf( + "-an", + // One pass with a generated palette. GIF is limited to 256 colours, and + // the default palette produces visibly banded output; split+palettegen + // and paletteuse in a single graph avoids a temporary palette file. + "-vf", + "fps=12,scale=480:-1:flags=lanczos,split[a][b];" + + "[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer", + "-loop", "0", + ) + + else -> listOf("-an", "-vf", "fps=1", "-vsync", "0") } - private fun containerArgs(format: OutputFormat): List = when (format.container) { - // Move the moov atom to the front so the file starts playing before it is - // fully downloaded. This is also the reason output never goes through a SAF - // file descriptor: faststart has to seek backwards to rewrite the header. - org.libremediaconverter.model.Container.MP4 -> listOf("-movflags", "+faststart") - else -> emptyList() + private fun videoArgs(plan: ConversionPlan, request: ConversionRequest): List = + when (val video = plan.video) { + // -vn drops video entirely. Without it FFmpeg will happily try to carry a video + // stream into an audio container and fail at the muxer. + VideoPlan.Drop -> listOf("-vn") + + VideoPlan.Copy -> buildList { + add("-c:v"); add("copy") + // The hvc1 brand matters on the copy path too, not just when encoding: remuxing + // HEVC out of Matroska into MP4 otherwise produces a file Apple devices and many + // hardware players refuse, even though the samples are byte-identical. + addAll(hevcTagIfNeeded(plan, request)) + } + + is VideoPlan.Encode -> encodeVideo(video.codec, request.quality) + } + + private fun encodeVideo(codec: VideoCodec, quality: QualityTier): List { + val preset = if (quality == QualityTier.BEST) "medium" else "veryfast" + return when (codec) { + // Software encoding: this is what the GPL licence buys. CRF targets a + // quality level and lets the bitrate fall where it may, which is what + // "compress this well" actually needs. No hardware encoder on Android + // exposes it. + // + // Software encoding, always. FFmpeg's *_mediacodec encoders used to be selected + // here, on the theory that a job routed to FFmpeg for container reasons could still + // encode in hardware. In practice they are undocumented, per-device flaky, and were + // observed failing twice on real footage on a Pixel 10 Pro XL -- once binding to a + // software codec while claiming to be the fast path, and once dying mid-export with + // "Error submitting video frame to the encoder" even after the pixel format was + // pinned. They also duplicate, badly, something Media3 already does properly. A job + // only reaches FFmpeg because Media3 could not handle it, which is itself evidence + // that hardware encoding is unlikely to work for that input. Fast therefore means a + // fast *preset*, not a different encoder. + VideoCodec.H265 -> listOf( + "-c:v", "libx265", "-crf", "$CRF_H265", "-preset", preset, + // Without this, many players and Apple devices refuse HEVC in MP4. + "-tag:v", "hvc1", + ) + PIX_FMT + + VideoCodec.VP9 -> buildList { + addAll(listOf("-c:v", "libvpx-vp9", "-crf", "31", "-b:v", "0")) + if (quality == QualityTier.FAST) addAll(listOf("-deadline", "realtime")) + addAll(PIX_FMT) + } + + VideoCodec.H264 -> listOf( + "-c:v", "libx264", "-crf", "$CRF_H264", "-preset", preset, + ) + PIX_FMT + + // No silent substitution. A trailing `else -> libx264` would hand back H.264 for a + // VP8 or AV1 request without a word — structurally the same defect as the old + // `media3MimeType()`, whose `else -> VIDEO_H265` is what put an HEVC video track in a + // file named `.m4a`. ContainerCapabilities refuses these combinations and + // ConversionWorker checks before enqueuing, so reaching here is a bug worth hearing + // about rather than papering over. + VideoCodec.VP8, VideoCodec.AV1 -> error( + "This app cannot encode ${codec.label}; it can only copy an existing " + + "${codec.label} stream." + ) + + VideoCodec.COPY, VideoCodec.NONE -> error( + "encodeVideo called for $codec, which is not an encode" + ) + } + } + + private fun hevcTagIfNeeded(plan: ConversionPlan, request: ConversionRequest): List { + if (plan.container !in MP4_FAMILY) return emptyList() + val sourceIsHevc = request.probe.videoCodec + ?.let(CodecNames::videoFromName) == VideoCodec.H265 + return if (sourceIsHevc) listOf("-tag:v", "hvc1") else emptyList() + } + + private fun audioArgs(plan: ConversionPlan): List = + when (val audio = plan.audio) { + AudioPlan.Drop -> listOf("-an") + AudioPlan.Copy -> listOf("-c:a", "copy") + is AudioPlan.Encode -> when (audio.codec) { + AudioCodec.MP3 -> listOf("-c:a", "libmp3lame", "-q:a", "2") + AudioCodec.FLAC -> listOf("-c:a", "flac") + AudioCodec.PCM -> listOf("-c:a", "pcm_s16le") + AudioCodec.OPUS -> listOf("-c:a", "libopus", "-b:a", "128k") + AudioCodec.VORBIS -> listOf("-c:a", "libvorbis", "-q:a", "5") + else -> listOf("-c:a", "aac", "-b:a", "192k") + } + } + + private fun containerArgs(plan: ConversionPlan): List = buildList { + // Name the muxer rather than letting FFmpeg infer it from the output path. Inference is + // unreliable for MPEG-TS and ASF, and the app now lets the user pick a container + // independently of the preset that used to imply it. + add("-f"); add(plan.container.ffmpegFormat) + + if (plan.container in MP4_FAMILY) { + // Move the moov atom to the front so the file starts playing before it is + // fully downloaded. This is also the reason output never goes through a SAF + // file descriptor: faststart has to seek backwards to rewrite the header. + add("-movflags"); add("+faststart") + } } /** Output filename pattern for formats that emit many files. */ - fun outputPattern(format: OutputFormat, baseName: String): String = - if (format == OutputFormat.FRAMES_PNG) "${baseName}_%04d.png" else baseName + fun outputPattern(container: Container, baseName: String): String = + if (container == Container.IMAGE_SEQUENCE) "${baseName}_%04d.png" else baseName } diff --git a/app/src/main/java/org/libremediaconverter/model/CodecNames.kt b/app/src/main/java/org/libremediaconverter/model/CodecNames.kt new file mode 100644 index 0000000..3b68815 --- /dev/null +++ b/app/src/main/java/org/libremediaconverter/model/CodecNames.kt @@ -0,0 +1,50 @@ +package org.libremediaconverter.model + +/** + * Maps the short codec names probing produces onto the app's codec enums. + * + * Three vocabularies meet here: `MediaExtractor` MIME types (normalised to short names by + * `MediaProbe.shortName`), FFprobe's `codec_name`, and this app's enums. Stream copy needs the + * round trip — "the source says `hevc`; is that the same thing as [VideoCodec.H265], and can the + * target container hold it?" — so the mapping has to live somewhere both [CopyPlanner] and + * [ContainerCapabilities] can reach, and somewhere a JVM test can exercise it. + * + * An unrecognised name returns null. That is deliberately not "no match": a copy planner that + * treated unknown as compatible would stream-copy a codec into a container that cannot hold it, + * and the failure would land on the user as a file that will not play. + */ +object CodecNames { + + fun videoFromName(name: String?): VideoCodec? = when (name?.lowercase()) { + null, InputProbe.UNPARSEABLE -> null + "h264", "avc", "avc1", "x264" -> VideoCodec.H264 + "hevc", "h265", "hvc1", "hev1", "x265" -> VideoCodec.H265 + "vp8" -> VideoCodec.VP8 + "vp9", "vp09" -> VideoCodec.VP9 + "av1", "av01" -> VideoCodec.AV1 + else -> null + } + + fun audioFromName(name: String?): AudioCodec? = when (name?.lowercase()) { + null -> null + "aac", "mp4a", "aac_latm" -> AudioCodec.AAC + "opus" -> AudioCodec.OPUS + "vorbis" -> AudioCodec.VORBIS + "mp3", "mp3float", "mpga" -> AudioCodec.MP3 + "flac" -> AudioCodec.FLAC + "pcm", "raw", "pcm_s16le", "pcm_s24le", "pcm_f32le" -> AudioCodec.PCM + else -> null + } + + /** Human-readable name for the source-info card. Falls back to the raw probe string. */ + fun describeVideo(name: String?): String = when { + name == null -> "Unknown" + name == InputProbe.UNPARSEABLE -> "Unrecognised" + else -> videoFromName(name)?.label ?: name + } + + fun describeAudio(name: String?): String = when { + name == null -> "Unknown" + else -> audioFromName(name)?.label ?: name + } +} diff --git a/app/src/main/java/org/libremediaconverter/model/ContainerCapabilities.kt b/app/src/main/java/org/libremediaconverter/model/ContainerCapabilities.kt new file mode 100644 index 0000000..e5c3644 --- /dev/null +++ b/app/src/main/java/org/libremediaconverter/model/ContainerCapabilities.kt @@ -0,0 +1,323 @@ +package org.libremediaconverter.model + +/** + * Which codecs may go in which container, and whether this app can actually produce them. + * + * ## Why two questions, not one + * + * "Can MP4 carry AV1?" and "can this app make AV1?" have different answers, and remux is exactly + * where the difference shows. MP4 carries AV1 and ALAC happily; neither engine here encodes them. + * Matroska carries Vorbis; nothing in [org.libremediaconverter.ffmpeg.FFmpegCommandBuilder] emits a + * Vorbis encoder. A single `isValid` boolean would answer one of those questions and give the wrong + * error for the other — telling a user "MP4 cannot hold AV1" when the truth is "your AV1 file can be + * copied into MP4, just not re-encoded to it". + * + * So the matrix is indexed by mode: [CodecMode.COPY] asks only what the muxer accepts, + * [CodecMode.ENCODE] additionally asks what this app can encode. + * + * This object is the source of truth for container support. `Media3Muxers` reports a narrower set + * to Transformer — that is the *hardware* subset, and the router decides between them. + */ +object ContainerCapabilities { + + /** Video codecs each container can mux, regardless of whether this app can encode them. */ + private val CARRIES_VIDEO: Map> = mapOf( + Container.MP4 to setOf(VideoCodec.H264, VideoCodec.H265, VideoCodec.VP9, VideoCodec.AV1), + Container.MOV to setOf(VideoCodec.H264, VideoCodec.H265, VideoCodec.VP9, VideoCodec.AV1), + // Matroska is the permissive one: it is a general-purpose container and takes essentially + // any codec. That is what makes it the natural remux target. + Container.MKV to setOf( + VideoCodec.H264, VideoCodec.H265, VideoCodec.VP8, VideoCodec.VP9, VideoCodec.AV1, + ), + Container.WEBM to setOf(VideoCodec.VP8, VideoCodec.VP9, VideoCodec.AV1), + Container.MPEG_TS to setOf(VideoCodec.H264, VideoCodec.H265), + // AVI predates H.265 and has no standard mapping for it. + Container.AVI to setOf(VideoCodec.H264), + Container.FLV to setOf(VideoCodec.H264), + Container.ASF to setOf(VideoCodec.H264), + Container.GIF to emptySet(), + Container.IMAGE_SEQUENCE to emptySet(), + Container.OGG to emptySet(), + Container.WAV to emptySet(), + Container.AAC_ADTS to emptySet(), + Container.MP3 to emptySet(), + Container.FLAC to emptySet(), + ) + + private val CARRIES_AUDIO: Map> = mapOf( + Container.MP4 to setOf(AudioCodec.AAC, AudioCodec.MP3, AudioCodec.OPUS, AudioCodec.FLAC), + Container.MOV to setOf(AudioCodec.AAC, AudioCodec.MP3, AudioCodec.PCM), + Container.MKV to setOf( + AudioCodec.AAC, AudioCodec.OPUS, AudioCodec.VORBIS, + AudioCodec.MP3, AudioCodec.FLAC, AudioCodec.PCM, + ), + Container.WEBM to setOf(AudioCodec.OPUS, AudioCodec.VORBIS), + Container.MPEG_TS to setOf(AudioCodec.AAC, AudioCodec.MP3), + Container.AVI to setOf(AudioCodec.MP3, AudioCodec.PCM, AudioCodec.AAC), + Container.FLV to setOf(AudioCodec.AAC, AudioCodec.MP3), + Container.ASF to setOf(AudioCodec.AAC, AudioCodec.MP3), + Container.OGG to setOf(AudioCodec.OPUS, AudioCodec.VORBIS, AudioCodec.FLAC), + Container.WAV to setOf(AudioCodec.PCM), + Container.AAC_ADTS to setOf(AudioCodec.AAC), + Container.MP3 to setOf(AudioCodec.MP3), + Container.FLAC to setOf(AudioCodec.FLAC), + Container.GIF to emptySet(), + Container.IMAGE_SEQUENCE to emptySet(), + ) + + /** + * Video codecs this app can encode, via either engine. + * + * VP8 and AV1 are absent deliberately: `FFmpegCommandBuilder` has no encoder branch for either, + * and Media3's `setVideoMimeType` rejects both. They remain copyable. + */ + private val ENCODABLE_VIDEO = setOf(VideoCodec.H264, VideoCodec.H265, VideoCodec.VP9) + + /** Vorbis is absent for the same reason: nothing here emits a Vorbis encoder. */ + private val ENCODABLE_AUDIO = setOf( + AudioCodec.AAC, AudioCodec.OPUS, AudioCodec.MP3, AudioCodec.FLAC, AudioCodec.PCM, + ) + + fun accepts(container: Container, codec: VideoCodec, mode: CodecMode): Boolean = when (codec) { + VideoCodec.NONE -> true + VideoCodec.COPY -> error("Resolve COPY to a concrete codec before asking the matrix") + else -> codec in CARRIES_VIDEO.getValue(container) && + (mode == CodecMode.COPY || codec in ENCODABLE_VIDEO) + } + + fun accepts(container: Container, codec: AudioCodec, mode: CodecMode): Boolean = when (codec) { + AudioCodec.NONE -> true + AudioCodec.COPY -> error("Resolve COPY to a concrete codec before asking the matrix") + else -> codec in CARRIES_AUDIO.getValue(container) && + (mode == CodecMode.COPY || codec in ENCODABLE_AUDIO) + } + + /** Every video codec [container] can be asked to produce, for building the picker. */ + fun encodableVideo(container: Container): List = + CARRIES_VIDEO.getValue(container).filter { it in ENCODABLE_VIDEO } + + fun encodableAudio(container: Container): List = + CARRIES_AUDIO.getValue(container).filter { it in ENCODABLE_AUDIO } + + /** + * Checks a user's choice against the input they picked. + * + * Returns alternatives rather than merely refusing, because the Advanced picker deliberately + * lets an incompatible combination be selected: the error has to explain what would work. + */ + fun validate(spec: OutputSpec, probe: InputProbe): Validation { + if (spec.isImageOutput) { + return if (spec.videoCodec == VideoCodec.NONE && spec.audioCodec == AudioCodec.NONE) { + Validation.Valid + } else { + Validation.Invalid( + "${spec.container.label} is an image format and carries no codecs.", + listOf(spec.copy(videoCodec = VideoCodec.NONE, audioCodec = AudioCodec.NONE)), + ) + } + } + + if (spec.videoCodec == VideoCodec.NONE && spec.audioCodec == AudioCodec.NONE) { + return Validation.Invalid( + "This would produce an empty file — keep at least one track.", + suggestions( + // Ask for both tracks back, then let repair settle what this container and + // this input can actually give. + spec.copy(videoCodec = VideoCodec.COPY, audioCodec = AudioCodec.COPY), + probe, + exclude = spec, + ), + ) + } + + if (spec.videoCodec != VideoCodec.NONE && !spec.container.canHoldVideo) { + return Validation.Invalid( + "${spec.container.label} holds audio only.", + suggestions(spec, probe), + ) + } + + validateVideo(spec, probe)?.let { return it } + validateAudio(spec, probe)?.let { return it } + return Validation.Valid + } + + private fun validateVideo(spec: OutputSpec, probe: InputProbe): Validation.Invalid? { + val codec = spec.videoCodec + if (codec == VideoCodec.NONE) return null + + if (codec == VideoCodec.COPY) { + if (!probe.hasVideo) { + return Validation.Invalid( + "This file has no video track to copy.", + listOf(spec.copy(videoCodec = VideoCodec.NONE)), + ) + } + val source = CodecNames.videoFromName(probe.videoCodec) + ?: return Validation.Invalid( + // Never guess. A copy of an unidentified codec is how you ship a file that + // does not play — the same reasoning ConcatPlanner records for the join flow. + "The source video codec could not be identified, so it cannot be copied.", + suggestions(spec, probe), + ) + if (!accepts(spec.container, source, CodecMode.COPY)) { + return Validation.Invalid( + "${spec.container.label} cannot hold ${source.label} video.", + suggestions(spec, probe), + ) + } + return null + } + + if (codec !in CARRIES_VIDEO.getValue(spec.container)) { + return Validation.Invalid( + "${spec.container.label} cannot hold ${codec.label} video.", + suggestions(spec, probe), + ) + } + if (codec !in ENCODABLE_VIDEO) { + return Validation.Invalid( + "This app cannot encode ${codec.label}. It can still be copied from a " + + "${codec.label} source.", + suggestions(spec, probe), + ) + } + return null + } + + private fun validateAudio(spec: OutputSpec, probe: InputProbe): Validation.Invalid? { + val codec = spec.audioCodec + if (codec == AudioCodec.NONE) return null + + if (codec == AudioCodec.COPY) { + val source = CodecNames.audioFromName(probe.audioCodec) + ?: return Validation.Invalid( + "The source audio codec could not be identified, so it cannot be copied.", + suggestions(spec, probe), + ) + if (!accepts(spec.container, source, CodecMode.COPY)) { + return Validation.Invalid( + "${spec.container.label} cannot hold ${source.label} audio.", + suggestions(spec, probe), + ) + } + return null + } + + if (codec !in CARRIES_AUDIO.getValue(spec.container)) { + return Validation.Invalid( + "${spec.container.label} cannot hold ${codec.label} audio.", + suggestions(spec, probe), + ) + } + if (codec !in ENCODABLE_AUDIO) { + return Validation.Invalid( + "This app cannot encode ${codec.label} audio. It can still be copied from a " + + "${codec.label} source.", + suggestions(spec, probe), + ) + } + return null + } + + /** + * Combinations that would work, closest to what was asked for. + * + * Every entry is repaired on *both* codec axes, not just the one that failed. Fixing only the + * offending axis is the obvious implementation and it is wrong: "H.264 in WebM" swaps the video + * to VP9 and leaves AAC behind, which WebM cannot hold either, so the suggestion is as invalid + * as the thing it was meant to fix. `ContainerCapabilitiesTest` asserts every suggestion + * validates, which is what caught that. + */ + private fun suggestions( + spec: OutputSpec, + probe: InputProbe, + /** What not to suggest. Differs from [spec] when the caller repaired it first. */ + exclude: OutputSpec = spec, + ): List { + val containers = buildList { + add(spec.container) + // A container that can hold what the user actually asked for keeps their intent. + firstContainerHolding(spec.videoCodec, probe)?.let(::add) + CodecNames.videoFromName(probe.videoCodec) + ?.let { firstContainerHolding(it, probe) } + ?.let(::add) + } + + return containers.distinct() + .mapNotNull { repair(spec.copy(container = it), probe) } + .filter { it != exclude } + .distinct() + .filter { validate(it, probe).isValid } + .take(3) + } + + /** Best valid spec for this container, preserving as much of the request as possible. */ + private fun repair(spec: OutputSpec, probe: InputProbe): OutputSpec? { + val container = spec.container + if (container == Container.GIF || container == Container.IMAGE_SEQUENCE) return null + + val video = repairVideo(spec, probe) + val audio = repairAudio(spec, probe) + if (video == VideoCodec.NONE && audio == AudioCodec.NONE) return null + return spec.copy(videoCodec = video, audioCodec = audio) + } + + private fun repairVideo(spec: OutputSpec, probe: InputProbe): VideoCodec { + val container = spec.container + if (spec.videoCodec == VideoCodec.NONE || !container.canHoldVideo) return VideoCodec.NONE + + val source = CodecNames.videoFromName(probe.videoCodec).takeIf { probe.hasVideo } + val copyable = source != null && accepts(container, source, CodecMode.COPY) + + return when { + // An explicit copy that works is exactly what was asked for. + spec.videoCodec == VideoCodec.COPY && copyable -> VideoCodec.COPY + // The requested codec is what the source already is, and we cannot encode it — but we + // can carry it across untouched. That is the useful answer for AV1 and VP8. + copyable && source == spec.videoCodec -> VideoCodec.COPY + spec.videoCodec != VideoCodec.COPY && + accepts(container, spec.videoCodec, CodecMode.ENCODE) -> spec.videoCodec + else -> encodableVideo(container).firstOrNull() ?: VideoCodec.NONE + } + } + + private fun repairAudio(spec: OutputSpec, probe: InputProbe): AudioCodec { + val container = spec.container + if (spec.audioCodec == AudioCodec.NONE) return AudioCodec.NONE + + val source = CodecNames.audioFromName(probe.audioCodec) + val copyable = source != null && accepts(container, source, CodecMode.COPY) + + return when { + spec.audioCodec == AudioCodec.COPY && copyable -> AudioCodec.COPY + copyable && source == spec.audioCodec -> AudioCodec.COPY + spec.audioCodec != AudioCodec.COPY && + accepts(container, spec.audioCodec, CodecMode.ENCODE) -> spec.audioCodec + else -> encodableAudio(container).firstOrNull() ?: AudioCodec.NONE + } + } + + /** A container that can hold [codec], preferring the one the input already uses. */ + private fun firstContainerHolding(codec: VideoCodec, probe: InputProbe): Container? { + if (codec == VideoCodec.NONE || codec == VideoCodec.COPY) return null + val holders = Container.entries.filter { codec in CARRIES_VIDEO.getValue(it) } + return holders.firstOrNull { it == probe.container } ?: holders.firstOrNull() + } +} + +/** Whether a track is being copied through or re-encoded. */ +enum class CodecMode { COPY, ENCODE } + +sealed interface Validation { + data object Valid : Validation + + /** + * @param suggestions combinations that would work. Never empty in practice, and the Advanced + * picker renders them as one-tap fixes; every entry is itself valid, which + * `ContainerCapabilitiesTest` asserts. + */ + data class Invalid(val message: String, val suggestions: List) : Validation + + val isValid: Boolean get() = this is Valid +} diff --git a/app/src/main/java/org/libremediaconverter/model/ConversionRouter.kt b/app/src/main/java/org/libremediaconverter/model/ConversionRouter.kt index 49c3fd3..06afebd 100644 --- a/app/src/main/java/org/libremediaconverter/model/ConversionRouter.kt +++ b/app/src/main/java/org/libremediaconverter/model/ConversionRouter.kt @@ -10,6 +10,15 @@ package org.libremediaconverter.model * * The rules below are capability boundaries, not preferences. Each one exists because * Media3 would fail or silently produce the wrong thing. + * + * ## Every rule asks the plan, not the request + * + * [CopyPlanner] resolves [VideoCodec.COPY] into a concrete per-track decision *before* any rule + * runs, and each rule below consults that plan rather than the user's raw choice. The ordering is + * load-bearing: `COPY` belongs to none of the capability sets, so a rule that tested the request + * directly would send every remux to FFmpeg on the first check and Media3's transmux path would + * never run. Worse, nothing would notice — FFmpeg's `-c copy` produces a perfectly correct file, + * just more slowly and on the CPU. */ object ConversionRouter { @@ -28,15 +37,40 @@ object ConversionRouter { Container.AAC_ADTS, ) - /** WebM is codec-restricted: Media3's WebmMuxer writes only these. */ - private val WEBM_AUDIO = setOf(AudioCodec.OPUS, AudioCodec.VORBIS) - private val WEBM_VIDEO = setOf(VideoCodec.VP8, VideoCodec.VP9, VideoCodec.NONE) + /** + * What Media3's muxers can *carry*, as distinct from what Media3 can encode. + * + * Only stream copy makes the difference visible, and it is not a small one: Media3's MP4 muxer + * accepts AAC, Opus, Vorbis and PCM but neither MP3 nor FLAC, so remuxing an MP3 track into MP4 + * — legal, and something FFmpeg does without complaint — has to leave the hardware path. Before + * remuxing existed nothing could reach that combination, so nothing had to know. + * + * Transcribed from each `Muxer.Factory.getSupportedSampleMimeTypes` in `Media3Muxers`; + * `Media3MuxersTest` asserts the transcription still matches. + */ + internal val MEDIA3_MUXABLE_VIDEO: Map> = mapOf( + Container.MP4 to setOf(VideoCodec.H264, VideoCodec.H265, VideoCodec.VP9, VideoCodec.AV1), + Container.WEBM to setOf(VideoCodec.VP8, VideoCodec.VP9), + Container.OGG to emptySet(), + Container.WAV to emptySet(), + Container.AAC_ADTS to emptySet(), + ) - /** Video codecs Media3 can be asked to encode (`Transformer.setVideoMimeType`). */ - private val MEDIA3_VIDEO = setOf(VideoCodec.H264, VideoCodec.H265, VideoCodec.NONE) + internal val MEDIA3_MUXABLE_AUDIO: Map> = mapOf( + Container.MP4 to setOf( + AudioCodec.AAC, AudioCodec.OPUS, AudioCodec.VORBIS, AudioCodec.PCM, + ), + Container.WEBM to setOf(AudioCodec.OPUS, AudioCodec.VORBIS), + Container.OGG to setOf(AudioCodec.OPUS, AudioCodec.VORBIS), + Container.WAV to setOf(AudioCodec.PCM), + Container.AAC_ADTS to setOf(AudioCodec.AAC), + ) + + /** Video codecs Media3 can encode (`Transformer.setVideoMimeType`). */ + private val MEDIA3_VIDEO = setOf(VideoCodec.H264, VideoCodec.H265) /** Audio codecs Media3 can encode. Notably absent: MP3 and FLAC. */ - private val MEDIA3_AUDIO = setOf(AudioCodec.AAC, AudioCodec.OPUS, AudioCodec.PCM, AudioCodec.NONE) + private val MEDIA3_AUDIO = setOf(AudioCodec.AAC, AudioCodec.OPUS, AudioCodec.PCM) fun route(request: ConversionRequest, device: DeviceCodecs): Decision { when (request.enginePreference) { @@ -45,6 +79,10 @@ object ConversionRouter { else -> Unit } + val plan = CopyPlanner.plan(request.spec, request.probe) + val videoEncode = (plan.video as? VideoPlan.Encode)?.codec + val audioEncode = (plan.audio as? AudioPlan.Encode)?.codec + // Order matters below: the specific reasons are checked before the general // ones, because the reason string is shown to the user. "Android has no // encoder for this format" tells them something actionable about MP3; @@ -52,36 +90,59 @@ object ConversionRouter { // MP3 has no encoder anywhere on Android, at any API level. That is a platform // gap rather than a Media3 limitation, and libmp3lame is the only way the app - // can produce MP3 at all. - if (request.format.audioCodec !in MEDIA3_AUDIO) { + // can produce MP3 at all. Asked only of a track being encoded: copying an MP3 + // stream into a different container needs no encoder and stays on hardware. + if (audioEncode != null && audioEncode !in MEDIA3_AUDIO) { return Decision(Engine.FFMPEG, Reason.NO_PLATFORM_ENCODER) } // GIF and frame sequences are image outputs; Media3 has no muxer for them. - if (request.format.isImageOutput) { + if (request.spec.isImageOutput) { return Decision(Engine.FFMPEG, Reason.IMAGE_OUTPUT) } - // Containers Media3 cannot mux at all, chiefly Matroska. - if (request.format.container !in MEDIA3_CONTAINERS) { + // Containers Media3 cannot mux at all, chiefly Matroska. This is about the *output*: + // Media3 reads Matroska perfectly well, which is what makes MKV -> MP4 a hardware remux. + if (plan.container !in MEDIA3_CONTAINERS) { return Decision(Engine.FFMPEG, Reason.CONTAINER_UNSUPPORTED) } - // WebM is codec-restricted even though the container itself is supported. - if (request.format.container == Container.WEBM && - (request.format.audioCodec !in WEBM_AUDIO || request.format.videoCodec !in WEBM_VIDEO) - ) { - return Decision(Engine.FFMPEG, Reason.WEBM_CODEC_UNSUPPORTED) + // The container is supported but its muxer is codec-restricted. What matters is what ends + // up in the file, so a copied track is judged by its source codec rather than the request. + if (!media3CanMux(plan, request.probe)) { + return Decision( + Engine.FFMPEG, + if (plan.container == Container.WEBM) { + Reason.WEBM_CODEC_UNSUPPORTED + } else { + Reason.CONTAINER_CODEC_UNSUPPORTED + }, + ) + } + + // A file the platform extractor could not open cannot be read at all, copied or not. + if (request.probe.videoCodec == InputProbe.UNPARSEABLE) { + return Decision(Engine.FFMPEG, Reason.NO_PLATFORM_DECODER) } // Media3 does not bundle ExoPlayer's software decoders, so an input codec with // no platform decoder cannot be read at all. The dav1d extension does not // rescue this: Transformer ignores bundled software decoder modules. + // + // Only relevant for a track being re-encoded — a stream copy never decodes, which is + // precisely why remuxing an exotic codec into a new container still works on hardware. val inputCodec = request.probe.videoCodec - if (inputCodec != null && !device.canDecode(inputCodec)) { + if (videoEncode != null && inputCodec != null && !device.canDecode(inputCodec)) { return Decision(Engine.FFMPEG, Reason.NO_PLATFORM_DECODER) } + // Nothing is re-encoded, so none of the encoder rules below apply and neither does the + // quality tier: there is no quality decision to make when no samples are recompressed. + // This is the fast path the remux feature exists for. + if (plan.isPureRemux) { + return Decision(Engine.MEDIA3, Reason.REMUX_NO_REENCODE) + } + // CRF and two-pass are the whole point of the quality tier, and MediaCodec // exposes neither, so BEST always means software encoding. if (request.quality == QualityTier.BEST) { @@ -90,27 +151,52 @@ object ConversionRouter { // Transformer.setVideoMimeType accepts only H.263/H.264/H.265/MP4V, so VP9 and // AV1 targets cannot be encoded by Media3 regardless of what the device can do. - if (request.format.videoCodec !in MEDIA3_VIDEO) { + if (videoEncode != null && videoEncode !in MEDIA3_VIDEO) { return Decision(Engine.FFMPEG, Reason.NO_PLATFORM_ENCODER) } // Finally, the target codec has to be hardware-encodable on this specific // device. HEVC is near-universal; AV1 encode is rare. - if (request.format.videoCodec != VideoCodec.NONE && - !device.canEncode(request.format.videoCodec) - ) { + if (videoEncode != null && !device.canEncode(videoEncode)) { return Decision(Engine.FFMPEG, Reason.NO_HARDWARE_ENCODER) } return Decision(Engine.MEDIA3, Reason.HARDWARE_CAPABLE) } + /** Whether Media3's muxer for this container can carry what the plan will produce. */ + private fun media3CanMux(plan: ConversionPlan, probe: InputProbe): Boolean { + val video = when (val v = plan.video) { + is VideoPlan.Encode -> v.codec + VideoPlan.Copy -> CodecNames.videoFromName(probe.videoCodec) + VideoPlan.Drop -> null + } + val audio = when (val a = plan.audio) { + is AudioPlan.Encode -> a.codec + AudioPlan.Copy -> CodecNames.audioFromName(probe.audioCodec) + AudioPlan.Drop -> null + } + + // A copied track whose codec we could not name is unproven, not permitted. CopyPlanner + // should already have refused it, so this is the second line of defence. + if (plan.video == VideoPlan.Copy && video == null) return false + if (plan.audio == AudioPlan.Copy && audio == null) return false + + val muxableVideo = MEDIA3_MUXABLE_VIDEO[plan.container].orEmpty() + val muxableAudio = MEDIA3_MUXABLE_AUDIO[plan.container].orEmpty() + if (video != null && video !in muxableVideo) return false + if (audio != null && audio !in muxableAudio) return false + return true + } + data class Decision(val engine: Engine, val reason: Reason) enum class Reason(val explanation: String) { HARDWARE_CAPABLE("Hardware accelerated"), + REMUX_NO_REENCODE("Remuxed — streams copied, nothing re-encoded"), CONTAINER_UNSUPPORTED("This container needs FFmpeg"), WEBM_CODEC_UNSUPPORTED("WebM only supports VP8/VP9 with Opus or Vorbis"), + CONTAINER_CODEC_UNSUPPORTED("That codec needs FFmpeg for this container"), NO_PLATFORM_ENCODER("Android has no encoder for this format"), NO_PLATFORM_DECODER("This device cannot decode the input in hardware"), NO_HARDWARE_ENCODER("This device has no hardware encoder for that codec"), diff --git a/app/src/main/java/org/libremediaconverter/model/CopyPlanner.kt b/app/src/main/java/org/libremediaconverter/model/CopyPlanner.kt new file mode 100644 index 0000000..3542944 --- /dev/null +++ b/app/src/main/java/org/libremediaconverter/model/CopyPlanner.kt @@ -0,0 +1,119 @@ +package org.libremediaconverter.model + +/** What happens to the video track. */ +sealed interface VideoPlan { + /** Stream copy — the samples are muxed across untouched. */ + data object Copy : VideoPlan + data class Encode(val codec: VideoCodec) : VideoPlan + data object Drop : VideoPlan +} + +/** What happens to the audio track. */ +sealed interface AudioPlan { + data object Copy : AudioPlan + data class Encode(val codec: AudioCodec) : AudioPlan + data object Drop : AudioPlan +} + +/** + * A resolved conversion: no [VideoCodec.COPY] left to interpret, every track decided. + */ +data class ConversionPlan( + val container: Container, + val video: VideoPlan, + val audio: AudioPlan, +) { + /** No track is re-encoded and at least one is copied: a container change and nothing more. */ + val isPureRemux: Boolean + get() = video !is VideoPlan.Encode && audio !is AudioPlan.Encode && + (video is VideoPlan.Copy || audio is AudioPlan.Copy) + + val hasVideo: Boolean get() = video != VideoPlan.Drop +} + +/** + * Turns a requested [OutputSpec] plus what is known about the input into a concrete plan. + * + * ## Unknown is not a match + * + * When the source codec cannot be identified — [InputProbe.UNPARSEABLE], or FFprobe reported + * something this app does not recognise — a copy is never attempted. It falls back to re-encoding. + * This is the same rule [ConcatPlanner] states for the join flow, for the same reason: a needless + * re-encode costs time, while a wrong stream copy costs the user a file that will not play, and + * they may not find out until long after the source is gone. + * + * ## Why a matching codec is not automatically a copy + * + * When the user asks for H.264 and the source is already H.264, copying is usually what they want + * — but only if the *container* is changing. If both container and codec already match, the only + * reason to run the job at all is to re-encode it, most likely to make it smaller, and silently + * copying would hand back a byte-identical file and call it done. + * + * That rule assumes re-encoding is the only other reason to convert. If bitrate, resolution or + * frame-rate controls are ever added, this is the decision that has to be revisited: at that point + * "same container, same codec" stops implying "compress it". + */ +object CopyPlanner { + + fun plan(spec: OutputSpec, probe: InputProbe): ConversionPlan = ConversionPlan( + container = spec.container, + video = planVideo(spec, probe), + audio = planAudio(spec, probe), + ) + + private fun planVideo(spec: OutputSpec, probe: InputProbe): VideoPlan { + val requested = spec.videoCodec + if (requested == VideoCodec.NONE) return VideoPlan.Drop + if (!probe.hasVideo) return VideoPlan.Drop + + val source = CodecNames.videoFromName(probe.videoCodec) + + if (requested == VideoCodec.COPY) { + val copyable = source != null && + ContainerCapabilities.accepts(spec.container, source, CodecMode.COPY) + if (copyable) return VideoPlan.Copy + // Asked to copy but cannot prove it is safe. Re-encode rather than guess; the picker + // normally refuses this combination before it gets here, so this is the belt to + // validation's braces — a stale queued job must not turn into a corrupt file. + return fallbackVideoEncode(spec.container) + } + + val shouldCopyInstead = source == requested && + probe.container != null && + probe.container != spec.container && + ContainerCapabilities.accepts(spec.container, requested, CodecMode.COPY) + + return if (shouldCopyInstead) VideoPlan.Copy else VideoPlan.Encode(requested) + } + + private fun planAudio(spec: OutputSpec, probe: InputProbe): AudioPlan { + val requested = spec.audioCodec + if (requested == AudioCodec.NONE) return AudioPlan.Drop + + val source = CodecNames.audioFromName(probe.audioCodec) + + if (requested == AudioCodec.COPY) { + val copyable = source != null && + ContainerCapabilities.accepts(spec.container, source, CodecMode.COPY) + if (copyable) return AudioPlan.Copy + return fallbackAudioEncode(spec.container) + } + + val shouldCopyInstead = source == requested && + probe.container != null && + probe.container != spec.container && + ContainerCapabilities.accepts(spec.container, requested, CodecMode.COPY) + + return if (shouldCopyInstead) AudioPlan.Copy else AudioPlan.Encode(requested) + } + + private fun fallbackVideoEncode(container: Container): VideoPlan = + ContainerCapabilities.encodableVideo(container).firstOrNull() + ?.let(VideoPlan::Encode) + ?: VideoPlan.Drop + + private fun fallbackAudioEncode(container: Container): AudioPlan = + ContainerCapabilities.encodableAudio(container).firstOrNull() + ?.let(AudioPlan::Encode) + ?: AudioPlan.Drop +} diff --git a/app/src/main/java/org/libremediaconverter/model/OutputFormat.kt b/app/src/main/java/org/libremediaconverter/model/OutputFormat.kt index c0ce6bd..1ff906e 100644 --- a/app/src/main/java/org/libremediaconverter/model/OutputFormat.kt +++ b/app/src/main/java/org/libremediaconverter/model/OutputFormat.kt @@ -1,45 +1,153 @@ package org.libremediaconverter.model -/** Container families, used by the router to decide which engine can mux the result. */ -enum class Container { MP4, WEBM, MKV, OGG, WAV, AAC_ADTS, MP3, GIF, IMAGE_SEQUENCE } +/** + * Container families. + * + * Each entry owns the three things that used to be duplicated per preset: the FFmpeg muxer name, + * the file extension, and the MIME type handed to the SAF `CreateDocument` contract. Keeping them + * here is what makes an open container × codec matrix possible — a preset no longer has to exist + * for every combination someone might want. + * + * Extension and MIME type depend on whether a video track survives: Matroska with video is `.mkv` + * and without it `.mka`, MP4 is `.mp4` or `.m4a`. That distinction is why they are functions rather + * than properties. + * + * @param ffmpegFormat the `-f` value. Named explicitly rather than left to extension inference, + * which is unreliable for MPEG-TS and ASF. + */ +enum class Container( + val label: String, + val ffmpegFormat: String, + private val videoExtension: String?, + private val audioExtension: String, + private val videoMime: String?, + private val audioMime: String, +) { + MP4("MP4", "mp4", "mp4", "m4a", "video/mp4", "audio/mp4"), + MOV("MOV", "mov", "mov", "m4a", "video/quicktime", "audio/mp4"), + MKV("Matroska", "matroska", "mkv", "mka", "video/x-matroska", "audio/x-matroska"), + WEBM("WebM", "webm", "webm", "weba", "video/webm", "audio/webm"), + MPEG_TS("MPEG-TS", "mpegts", "ts", "ts", "video/mp2t", "video/mp2t"), + AVI("AVI", "avi", "avi", "avi", "video/x-msvideo", "video/x-msvideo"), + FLV("FLV", "flv", "flv", "flv", "video/x-flv", "video/x-flv"), + ASF("WMV/ASF", "asf", "wmv", "wma", "video/x-ms-wmv", "audio/x-ms-wma"), -/** Codecs this app can be asked to produce. */ -enum class VideoCodec { H264, H265, VP9, VP8, AV1, NONE } -enum class AudioCodec { AAC, OPUS, VORBIS, MP3, FLAC, PCM, NONE } + OGG("Ogg", "ogg", null, "opus", null, "audio/ogg"), + WAV("WAV", "wav", null, "wav", null, "audio/wav"), + AAC_ADTS("AAC", "adts", null, "aac", null, "audio/aac"), + MP3("MP3", "mp3", null, "mp3", null, "audio/mpeg"), + FLAC("FLAC", "flac", null, "flac", null, "audio/flac"), + + GIF("GIF", "gif", "gif", "gif", "image/gif", "image/gif"), + IMAGE_SEQUENCE("PNG frames", "image2", "png", "png", "image/png", "image/png"), + ; + + /** Whether this container can hold a video track at all. */ + val canHoldVideo: Boolean get() = videoExtension != null + + fun extensionFor(hasVideo: Boolean): String = + if (hasVideo) videoExtension ?: audioExtension else audioExtension + + fun mimeTypeFor(hasVideo: Boolean): String = + if (hasVideo) videoMime ?: audioMime else audioMime +} /** - * A user-selectable output format. + * Codecs this app can be asked to produce. * - * Deliberately a closed set rather than a free-form codec/container matrix: most - * combinations are either invalid or pointless, and the closed set is what makes the - * routing rules decidable. + * [COPY] is a first-class value rather than a flag alongside them: "keep whatever the source has" + * sits in exactly the same slot as "make it H.264", and modelling it as a codec means every + * exhaustive `when` in the codebase is forced to say what it does about copying. */ -enum class OutputFormat( - val label: String, +enum class VideoCodec(val label: String) { + COPY("Copy"), + H264("H.264"), + H265("H.265"), + VP9("VP9"), + VP8("VP8"), + AV1("AV1"), + NONE("None"), +} + +enum class AudioCodec(val label: String) { + COPY("Copy"), + AAC("AAC"), + OPUS("Opus"), + VORBIS("Vorbis"), + MP3("MP3"), + FLAC("FLAC"), + PCM("PCM"), + NONE("None"), +} + +/** + * A concrete output: one container, one choice per track. + * + * This replaces the closed enum of twelve triples that used to be the only way to describe an + * output. The closed set was defended on the grounds that it made routing decidable; decidability + * now comes from [ContainerCapabilities] instead, which is explicit and unit-tested rather than + * implicit in which combinations someone remembered to enumerate. + */ +data class OutputSpec( val container: Container, val videoCodec: VideoCodec, val audioCodec: AudioCodec, - val extension: String, - val mimeType: String, ) { - MP4_H264("MP4 (H.264)", Container.MP4, VideoCodec.H264, AudioCodec.AAC, "mp4", "video/mp4"), - MP4_H265("MP4 (H.265)", Container.MP4, VideoCodec.H265, AudioCodec.AAC, "mp4", "video/mp4"), - WEBM_VP9("WebM (VP9)", Container.WEBM, VideoCodec.VP9, AudioCodec.OPUS, "webm", "video/webm"), - MKV_H264("MKV (H.264)", Container.MKV, VideoCodec.H264, AudioCodec.AAC, "mkv", "video/x-matroska"), - MKV_H265("MKV (H.265)", Container.MKV, VideoCodec.H265, AudioCodec.AAC, "mkv", "video/x-matroska"), + /** Whether the output keeps a video track — the thing extension and MIME type turn on. */ + val hasVideo: Boolean get() = videoCodec != VideoCodec.NONE - MP3("MP3", Container.MP3, VideoCodec.NONE, AudioCodec.MP3, "mp3", "audio/mpeg"), - M4A_AAC("M4A (AAC)", Container.MP4, VideoCodec.NONE, AudioCodec.AAC, "m4a", "audio/mp4"), - OPUS("Opus", Container.OGG, VideoCodec.NONE, AudioCodec.OPUS, "opus", "audio/opus"), - FLAC("FLAC", Container.MKV, VideoCodec.NONE, AudioCodec.FLAC, "flac", "audio/flac"), - WAV("WAV", Container.WAV, VideoCodec.NONE, AudioCodec.PCM, "wav", "audio/wav"), + val isAudioOnly: Boolean + get() = videoCodec == VideoCodec.NONE && audioCodec != AudioCodec.NONE - GIF("GIF", Container.GIF, VideoCodec.NONE, AudioCodec.NONE, "gif", "image/gif"), - FRAMES_PNG("PNG frames", Container.IMAGE_SEQUENCE, VideoCodec.NONE, AudioCodec.NONE, "png", "image/png"); - - val isAudioOnly: Boolean get() = videoCodec == VideoCodec.NONE && audioCodec != AudioCodec.NONE val isImageOutput: Boolean get() = container == Container.GIF || container == Container.IMAGE_SEQUENCE + + /** True when neither track is re-encoded, i.e. this is a pure container change. */ + val isPureRemux: Boolean + get() = videoCodec.isCopyOrAbsent() && audioCodec.isCopyOrAbsent() && + (videoCodec == VideoCodec.COPY || audioCodec == AudioCodec.COPY) + + val extension: String get() = container.extensionFor(hasVideo) + val mimeType: String get() = container.mimeTypeFor(hasVideo) + + private fun VideoCodec.isCopyOrAbsent() = this == VideoCodec.COPY || this == VideoCodec.NONE + private fun AudioCodec.isCopyOrAbsent() = this == AudioCodec.COPY || this == AudioCodec.NONE +} + +/** + * The one-tap presets. + * + * Still a closed list, but it is now a convenience layer over [OutputSpec] rather than the only + * vocabulary available. Anything not here is reachable through the Advanced picker. + */ +enum class OutputFormat(val label: String, val spec: OutputSpec) { + MP4_H264("MP4 (H.264)", OutputSpec(Container.MP4, VideoCodec.H264, AudioCodec.AAC)), + MP4_H265("MP4 (H.265)", OutputSpec(Container.MP4, VideoCodec.H265, AudioCodec.AAC)), + WEBM_VP9("WebM (VP9)", OutputSpec(Container.WEBM, VideoCodec.VP9, AudioCodec.OPUS)), + MKV_H264("MKV (H.264)", OutputSpec(Container.MKV, VideoCodec.H264, AudioCodec.AAC)), + MKV_H265("MKV (H.265)", OutputSpec(Container.MKV, VideoCodec.H265, AudioCodec.AAC)), + + /** Container change only. The headline of the remux feature, given a preset of its own. */ + REMUX_MP4("Remux to MP4", OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY)), + REMUX_MKV("Remux to MKV", OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY)), + + MP3("MP3", OutputSpec(Container.MP3, VideoCodec.NONE, AudioCodec.MP3)), + M4A_AAC("M4A (AAC)", OutputSpec(Container.MP4, VideoCodec.NONE, AudioCodec.AAC)), + OPUS("Opus", OutputSpec(Container.OGG, VideoCodec.NONE, AudioCodec.OPUS)), + FLAC("FLAC", OutputSpec(Container.FLAC, VideoCodec.NONE, AudioCodec.FLAC)), + WAV("WAV", OutputSpec(Container.WAV, VideoCodec.NONE, AudioCodec.PCM)), + + GIF("GIF", OutputSpec(Container.GIF, VideoCodec.NONE, AudioCodec.NONE)), + FRAMES_PNG("PNG frames", OutputSpec(Container.IMAGE_SEQUENCE, VideoCodec.NONE, AudioCodec.NONE)), + ; + + val container: Container get() = spec.container + val videoCodec: VideoCodec get() = spec.videoCodec + val audioCodec: AudioCodec get() = spec.audioCodec + val extension: String get() = spec.extension + val mimeType: String get() = spec.mimeType + val isAudioOnly: Boolean get() = spec.isAudioOnly + val isImageOutput: Boolean get() = spec.isImageOutput } /** @@ -59,12 +167,33 @@ enum class EnginePreference { AUTO, PREFER_HARDWARE, FORCE_SOFTWARE } enum class Engine { MEDIA3, FFMPEG } -/** What we know about the input, as far as routing is concerned. */ +/** What kind of file the input turned out to be. */ +enum class InputKind { + VIDEO, + AUDIO_ONLY, + IMAGE, + + /** Nothing could read it. Distinct from the others, and never evidence of a codec match. */ + UNPARSEABLE, +} + +/** What we know about the input, as far as routing and stream copy are concerned. */ data class InputProbe( val videoCodec: String? = null, val audioCodec: String? = null, val hasVideo: Boolean = true, val durationMs: Long = 0, + val kind: InputKind = InputKind.VIDEO, + /** + * The source container, when it could be identified. + * + * `MediaExtractor` cannot report this at all, so it comes from FFprobe. It matters because + * [CopyPlanner] only upgrades a matching codec to a stream copy when the container is actually + * changing — see its `plan` KDoc. + */ + val container: Container? = null, + val width: Int = 0, + val height: Int = 0, ) { companion object { /** @@ -80,7 +209,7 @@ data class InputProbe( } data class ConversionRequest( - val format: OutputFormat, + val spec: OutputSpec, val quality: QualityTier = QualityTier.FAST, val enginePreference: EnginePreference = EnginePreference.AUTO, val probe: InputProbe = InputProbe(), @@ -93,4 +222,8 @@ data class ConversionRequest( * genuinely fast software preset instead of a mislabelled slow one. */ val hardwareEncodeAvailable: Boolean = true, -) +) { + val container: Container get() = spec.container + val videoCodec: VideoCodec get() = spec.videoCodec + val audioCodec: AudioCodec get() = spec.audioCodec +} diff --git a/app/src/main/java/org/libremediaconverter/work/ConversionWorker.kt b/app/src/main/java/org/libremediaconverter/work/ConversionWorker.kt index f05a577..ea28f12 100644 --- a/app/src/main/java/org/libremediaconverter/work/ConversionWorker.kt +++ b/app/src/main/java/org/libremediaconverter/work/ConversionWorker.kt @@ -14,12 +14,18 @@ import androidx.work.workDataOf import com.arthenica.ffmpegkit.FFmpegKitConfig import org.libremediaconverter.convert.ConversionDependencies import org.libremediaconverter.convert.MediaProbe +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.Container +import org.libremediaconverter.model.ContainerCapabilities import org.libremediaconverter.model.ConversionRequest import org.libremediaconverter.model.ConversionRouter import org.libremediaconverter.model.Engine import org.libremediaconverter.model.EnginePreference import org.libremediaconverter.model.OutputFormat +import org.libremediaconverter.model.OutputSpec import org.libremediaconverter.model.QualityTier +import org.libremediaconverter.model.Validation +import org.libremediaconverter.model.VideoCodec import java.io.File /** @@ -48,9 +54,7 @@ class ConversionWorker( ?: return Result.failure(workDataOf(KEY_ERROR to "No input file.")) val displayName = inputData.getString(KEY_DISPLAY_NAME) ?: "input" val sizeBytes = inputData.getLong(KEY_SIZE_BYTES, 0L) - val format = OutputFormat.valueOf( - inputData.getString(KEY_FORMAT) ?: OutputFormat.MP4_H265.name - ) + val spec = readSpec() val quality = QualityTier.valueOf( inputData.getString(KEY_QUALITY) ?: QualityTier.FAST.name ) @@ -67,16 +71,26 @@ class ConversionWorker( val probe = MediaProbe.probe(applicationContext, inputUri) val devices = ConversionDependencies.deviceCodecs() val request = ConversionRequest( - format = format, + spec = spec, quality = quality, enginePreference = preference, probe = probe, - hardwareEncodeAvailable = devices.canEncode(format.videoCodec), + hardwareEncodeAvailable = devices.canEncode(spec.videoCodec), ) - val decision = ConversionRouter.route(request, devices) - Log.i(TAG, "Routing $displayName -> ${format.name} via ${decision.engine} (${decision.reason})") + // The picker refuses an impossible combination before Convert is tappable, but a job can + // also arrive from a queued request made before the settings changed, or from a direct + // ConversionWorker.request(...) call. Checking here means an invalid spec fails with the + // reason rather than being silently coerced into something else. + val validation = ContainerCapabilities.validate(spec, probe) + if (validation is Validation.Invalid) { + Log.w(TAG, "Refusing $spec for $displayName: ${validation.message}") + return Result.failure(workDataOf(KEY_ERROR to validation.message)) + } - val staged = publisher.createStagingFile(outputNameFor(displayName, format)) + val decision = ConversionRouter.route(request, devices) + Log.i(TAG, "Routing $displayName -> $spec via ${decision.engine} (${decision.reason})") + + val staged = publisher.createStagingFile(outputNameFor(displayName, spec)) return try { when (decision.engine) { @@ -113,7 +127,7 @@ class ConversionWorker( ) { val engine = ConversionDependencies.hardware(applicationContext) try { - engine.transcode(inputUri, staged, request.format) { percent -> + engine.transcode(inputUri, staged, request) { percent -> publishProgress(displayName, percent) } return @@ -187,6 +201,28 @@ class ConversionWorker( } } + /** + * Reads the output spec out of the worker's input Data. + * + * Carried as three separate strings rather than one preset name: the picker can now produce + * combinations no preset covers, so there is no enum entry to name. Unknown or missing values + * fall back to the default preset rather than throwing — a worker that crashes on malformed + * input reports "conversion failed" with no useful message. + */ + private fun readSpec(): OutputSpec { + val fallback = OutputFormat.MP4_H265.spec + val container = inputData.getString(KEY_CONTAINER) + ?.let { name -> Container.entries.firstOrNull { it.name == name } } + ?: return fallback + val video = inputData.getString(KEY_VIDEO_CODEC) + ?.let { name -> VideoCodec.entries.firstOrNull { it.name == name } } + ?: return fallback + val audio = inputData.getString(KEY_AUDIO_CODEC) + ?.let { name -> AudioCodec.entries.firstOrNull { it.name == name } } + ?: return fallback + return OutputSpec(container, video, audio) + } + override suspend fun getForegroundInfo(): ForegroundInfo = foregroundInfo( inputData.getString(KEY_DISPLAY_NAME) ?: "input", @@ -205,7 +241,9 @@ class ConversionWorker( const val KEY_INPUT_URI = "input_uri" const val KEY_DISPLAY_NAME = "display_name" const val KEY_SIZE_BYTES = "size_bytes" - const val KEY_FORMAT = "format" + const val KEY_CONTAINER = "container" + const val KEY_VIDEO_CODEC = "video_codec" + const val KEY_AUDIO_CODEC = "audio_codec" const val KEY_QUALITY = "quality" const val KEY_ENGINE_PREFERENCE = "engine_preference" const val KEY_PROGRESS = "progress" @@ -218,15 +256,22 @@ class ConversionWorker( private const val NOTIFICATION_INTERVAL_MS = 1_000L private const val TAG = "ConversionWorker" - fun outputNameFor(inputName: String, format: OutputFormat): String = + /** + * The staged and suggested filename. + * + * The extension comes from the container and whether a video track survives, so Matroska + * yields `.mkv` or `.mka` and MP4 yields `.mp4` or `.m4a` without a preset having to + * enumerate both. + */ + fun outputNameFor(inputName: String, spec: OutputSpec): String = inputName.substringBeforeLast('.', inputName) + - "_converted.${format.extension}" + "_converted.${spec.extension}" fun request( inputUri: Uri, displayName: String, sizeBytes: Long, - format: OutputFormat = OutputFormat.MP4_H265, + spec: OutputSpec = OutputFormat.MP4_H265.spec, quality: QualityTier = QualityTier.FAST, enginePreference: EnginePreference = EnginePreference.AUTO, ) = OneTimeWorkRequestBuilder() @@ -235,7 +280,9 @@ class ConversionWorker( .putString(KEY_INPUT_URI, inputUri.toString()) .putString(KEY_DISPLAY_NAME, displayName) .putLong(KEY_SIZE_BYTES, sizeBytes) - .putString(KEY_FORMAT, format.name) + .putString(KEY_CONTAINER, spec.container.name) + .putString(KEY_VIDEO_CODEC, spec.videoCodec.name) + .putString(KEY_AUDIO_CODEC, spec.audioCodec.name) .putString(KEY_QUALITY, quality.name) .putString(KEY_ENGINE_PREFERENCE, enginePreference.name) .build() diff --git a/app/src/test/java/org/libremediaconverter/convert/Media3MuxersTest.kt b/app/src/test/java/org/libremediaconverter/convert/Media3MuxersTest.kt index 4a4b314..73c97a3 100644 --- a/app/src/test/java/org/libremediaconverter/convert/Media3MuxersTest.kt +++ b/app/src/test/java/org/libremediaconverter/convert/Media3MuxersTest.kt @@ -4,12 +4,17 @@ import androidx.media3.common.C import androidx.media3.common.MimeTypes import androidx.media3.common.util.UnstableApi import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse import org.junit.Assert.assertNotNull import org.junit.Assert.assertNull import org.junit.Assert.assertTrue import org.junit.Test +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.CodecMode import org.libremediaconverter.model.Container +import org.libremediaconverter.model.ContainerCapabilities import org.libremediaconverter.model.ConversionRouter +import org.libremediaconverter.model.VideoCodec /** * Guards the agreement between the router's container set and the muxers that back it. @@ -95,4 +100,92 @@ class Media3MuxersTest { .getSupportedSampleMimeTypes(C.TRACK_TYPE_AUDIO), ) } + + /** + * The router's transcription of what Media3 can carry must match the factories themselves. + * + * `ConversionRouter` cannot import a `Muxer.Factory` — `model/` is deliberately free of Android + * and Media3 types so the routing rules stay JVM-testable — so it restates these sets by hand. + * Two hand-written answers to one question drift; this is what stops them. + * + * The check runs against the factory's own `getSupportedSampleMimeTypes`, which is what + * `Transformer` consults, so it is ground truth rather than another transcription. + */ + @Test + fun `the router's muxable sets match what the factories report`() { + ConversionRouter.MEDIA3_CONTAINERS.forEach { container -> + val factory = requireNotNull(Media3Muxers.factoryFor(container)) + + val reportedVideo = factory.getSupportedSampleMimeTypes(C.TRACK_TYPE_VIDEO) + .mapNotNull(::videoCodecOf).toSet() + val reportedAudio = factory.getSupportedSampleMimeTypes(C.TRACK_TYPE_AUDIO) + .mapNotNull(::audioCodecOf).toSet() + + assertEquals( + "$container video", + reportedVideo, + ConversionRouter.MEDIA3_MUXABLE_VIDEO[container].orEmpty(), + ) + assertEquals( + "$container audio", + reportedAudio, + ConversionRouter.MEDIA3_MUXABLE_AUDIO[container].orEmpty(), + ) + } + } + + /** + * Records the two places Media3's muxers and the app-wide matrix deliberately disagree. + * + * Neither is a bug, and pinning them is the point: if either moves, one of the two documents + * has changed its mind and somebody should say so on purpose. + * + * - Media3's MP4 muxer accepts Vorbis; [ContainerCapabilities] declines to offer it, because + * Vorbis-in-MP4 is poorly supported by players. + * - The matrix offers MP3 and FLAC in MP4, which is legal and which FFmpeg writes happily, but + * Media3's MP4 muxer carries neither — so those jobs route to FFmpeg rather than failing. + */ + @Test + fun `the documented divergences between Media3 and the matrix still hold`() { + val media3Mp4Audio = ConversionRouter.MEDIA3_MUXABLE_AUDIO.getValue(Container.MP4) + + assertTrue( + "Media3 still takes Vorbis in MP4", + AudioCodec.VORBIS in media3Mp4Audio, + ) + assertFalse( + "the matrix still declines to offer it", + ContainerCapabilities.accepts(Container.MP4, AudioCodec.VORBIS, CodecMode.COPY), + ) + + listOf(AudioCodec.MP3, AudioCodec.FLAC).forEach { codec -> + assertTrue( + "the matrix still allows $codec in MP4", + ContainerCapabilities.accepts(Container.MP4, codec, CodecMode.COPY), + ) + assertFalse( + "Media3's MP4 muxer still cannot carry $codec", + codec in media3Mp4Audio, + ) + } + } + + private fun videoCodecOf(mime: String): VideoCodec? = when (mime) { + MimeTypes.VIDEO_H264 -> VideoCodec.H264 + MimeTypes.VIDEO_H265 -> VideoCodec.H265 + MimeTypes.VIDEO_VP8 -> VideoCodec.VP8 + MimeTypes.VIDEO_VP9 -> VideoCodec.VP9 + MimeTypes.VIDEO_AV1 -> VideoCodec.AV1 + else -> null + } + + private fun audioCodecOf(mime: String): AudioCodec? = when (mime) { + MimeTypes.AUDIO_AAC -> AudioCodec.AAC + MimeTypes.AUDIO_OPUS -> AudioCodec.OPUS + MimeTypes.AUDIO_VORBIS -> AudioCodec.VORBIS + MimeTypes.AUDIO_RAW -> AudioCodec.PCM + MimeTypes.AUDIO_MPEG -> AudioCodec.MP3 + MimeTypes.AUDIO_FLAC -> AudioCodec.FLAC + else -> null + } } diff --git a/app/src/test/java/org/libremediaconverter/convert/MediaProbeFormatTest.kt b/app/src/test/java/org/libremediaconverter/convert/MediaProbeFormatTest.kt new file mode 100644 index 0000000..8fa8a2c --- /dev/null +++ b/app/src/test/java/org/libremediaconverter/convert/MediaProbeFormatTest.kt @@ -0,0 +1,86 @@ +package org.libremediaconverter.convert + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Test +import org.libremediaconverter.model.Container + +/** + * Reading FFprobe's `format_name`, which is the only source of the source container. + * + * The trap is that FFprobe names every format sharing the demuxer, not the one the file actually + * is: a plain MP4 comes back as `mov,mp4,m4a,3gp,3g2,mj2`. Matching the whole string against a + * container name never succeeds, and taking the first entry reports every MP4 as a MOV. + * + * Getting this wrong fails silently rather than loudly — a null container just means `CopyPlanner` + * cannot tell the container is changing, so it declines to upgrade to a stream copy and everything + * is re-encoded. The feature would look like it simply did not work. + */ +class MediaProbeFormatTest { + + @Test + fun `an MP4 is not mistaken for a MOV`() { + assertEquals( + Container.MP4, + MediaProbe.containerFrom("mov,mp4,m4a,3gp,3g2,mj2"), + ) + } + + @Test + fun `a real QuickTime file is recognised`() { + assertEquals(Container.MOV, MediaProbe.containerFrom("mov")) + } + + /** + * WebM and Matroska share a demuxer and report the same format name. + * + * Both committed fixtures — an H.264 `.mkv` and a VP9 `.webm` — come back as `matroska,webm`, + * so the string alone cannot separate them. The codec is the only remaining signal: WebM + * permits VP8/VP9/AV1 and nothing else. + */ + @Test + fun `a Matroska file carrying H264 is not reported as WebM`() { + assertEquals(Container.MKV, MediaProbe.containerFrom("matroska,webm", "h264")) + } + + @Test + fun `a WebM-legal codec in that container is reported as WebM`() { + assertEquals(Container.WEBM, MediaProbe.containerFrom("matroska,webm", "vp9")) + assertEquals(Container.WEBM, MediaProbe.containerFrom("matroska,webm", "av1")) + } + + /** With no video track there is nothing to disambiguate on; Matroska is the general case. */ + @Test + fun `an audio-only Matroska file is reported as Matroska`() { + assertEquals(Container.MKV, MediaProbe.containerFrom("matroska,webm", null)) + } + + @Test + fun `the remaining containers map to themselves`() { + assertEquals(Container.MPEG_TS, MediaProbe.containerFrom("mpegts")) + assertEquals(Container.AVI, MediaProbe.containerFrom("avi")) + assertEquals(Container.FLV, MediaProbe.containerFrom("flv")) + assertEquals(Container.ASF, MediaProbe.containerFrom("asf")) + assertEquals(Container.WAV, MediaProbe.containerFrom("wav")) + assertEquals(Container.MP3, MediaProbe.containerFrom("mp3")) + assertEquals(Container.FLAC, MediaProbe.containerFrom("flac")) + assertEquals(Container.OGG, MediaProbe.containerFrom("ogg")) + } + + @Test + fun `whitespace and case do not matter`() { + assertEquals(Container.MP4, MediaProbe.containerFrom("MOV, MP4, M4A")) + } + + /** + * Unknown means unknown, not a guess. + * + * A wrong container would let the copy planner believe the container is changing when it is + * not, or the reverse — and the planner's whole discipline is that it never acts on a guess. + */ + @Test + fun `an unrecognised format name yields null rather than a guess`() { + assertNull(MediaProbe.containerFrom("some_new_format")) + assertNull(MediaProbe.containerFrom("")) + } +} diff --git a/app/src/test/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilderTest.kt b/app/src/test/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilderTest.kt index ccd3197..f119133 100644 --- a/app/src/test/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilderTest.kt +++ b/app/src/test/java/org/libremediaconverter/ffmpeg/FFmpegCommandBuilderTest.kt @@ -1,7 +1,12 @@ package org.libremediaconverter.ffmpeg +import org.libremediaconverter.model.Container import org.libremediaconverter.model.ConversionRequest +import org.libremediaconverter.model.InputProbe import org.libremediaconverter.model.OutputFormat +import org.libremediaconverter.model.AudioCodec +import org.libremediaconverter.model.OutputSpec +import org.libremediaconverter.model.VideoCodec import org.libremediaconverter.model.QualityTier import org.junit.Assert.assertEquals import org.junit.Assert.assertFalse @@ -14,14 +19,22 @@ class FFmpegCommandBuilderTest { format: OutputFormat, quality: QualityTier = QualityTier.BEST, hardwareEncodeAvailable: Boolean = true, + ): List = cmd(format.spec, quality, hardwareEncodeAvailable) + + private fun cmd( + spec: OutputSpec, + quality: QualityTier = QualityTier.BEST, + hardwareEncodeAvailable: Boolean = true, + probe: InputProbe = InputProbe(videoCodec = "h264", audioCodec = "aac"), ): List = FFmpegCommandBuilder.build( ConversionRequest( - format = format, + spec = spec, quality = quality, + probe = probe, hardwareEncodeAvailable = hardwareEncodeAvailable, ), inputPath = "/cache/in.mp4", - outputPath = "/cache/out.${format.extension}", + outputPath = "/cache/out.${spec.extension}", ) /** Asserts `flag` is present and immediately followed by `value`. */ @@ -183,7 +196,7 @@ class FFmpegCommandBuilderTest { assertPair(args, "-vf", "fps=1") assertEquals( "shot_%04d.png", - FFmpegCommandBuilder.outputPattern(OutputFormat.FRAMES_PNG, "shot"), + FFmpegCommandBuilder.outputPattern(Container.IMAGE_SEQUENCE, "shot"), ) } @@ -191,7 +204,7 @@ class FFmpegCommandBuilderTest { fun `single file formats keep their plain name`() { assertEquals( "clip.mp4", - FFmpegCommandBuilder.outputPattern(OutputFormat.MP4_H264, "clip.mp4"), + FFmpegCommandBuilder.outputPattern(Container.MP4, "clip.mp4"), ) } @@ -228,4 +241,154 @@ class FFmpegCommandBuilderTest { } } } + + // --- stream copy -------------------------------------------------------- + + @Test + fun `a pure remux copies both tracks and encodes neither`() { + val args = cmd(OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY)) + + assertPair(args, "-c:v", "copy") + assertPair(args, "-c:a", "copy") + assertFalse("a copy must not name an encoder: $args", args.contains("libx264")) + assertFalse(args.contains("libx265")) + assertFalse(args.contains("aac")) + } + + /** + * A stream copy decodes nothing, so there are no frames to convert. + * + * Every encode path pins yuv420p for good reason, but carrying the flag onto the copy path + * would be meaningless at best and would force a decode at worst. + */ + @Test + fun `a stream copy does not pin a pixel format`() { + val args = cmd(OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY)) + assertFalse("copying needs no -pix_fmt: $args", args.contains("-pix_fmt")) + } + + @Test + fun `copying video while re-encoding audio does both`() { + val args = cmd(OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.OPUS)) + assertPair(args, "-c:v", "copy") + assertPair(args, "-c:a", "libopus") + } + + /** + * The brand matters on the copy path too. + * + * Remuxing HEVC out of Matroska into MP4 moves byte-identical samples, but without the hvc1 + * brand Apple devices and many hardware players refuse the result. + */ + @Test + fun `copying HEVC into MP4 still tags it hvc1`() { + val args = cmd( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "hevc", audioCodec = "aac", container = Container.MKV), + ) + assertPair(args, "-c:v", "copy") + assertPair(args, "-tag:v", "hvc1") + } + + @Test + fun `copying HEVC into Matroska does not tag it`() { + val args = cmd( + OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "hevc", audioCodec = "aac", container = Container.MP4), + ) + assertFalse("hvc1 is an ISO-BMFF brand, not a Matroska one: $args", args.contains("-tag:v")) + } + + @Test + fun `copying H264 into MP4 does not tag it hvc1`() { + val args = cmd( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "h264", audioCodec = "aac", container = Container.MKV), + ) + assertFalse(args.contains("hvc1")) + } + + // --- explicit muxer selection ------------------------------------------- + + /** + * Every container names its muxer. + * + * The builder used to rely on FFmpeg inferring the format from the output extension, which is + * unreliable for MPEG-TS and ASF and became untenable once the container stopped being implied + * by the preset. + */ + @Test + fun `every container passes an explicit -f`() { + Container.entries.forEach { container -> + val spec = when { + container == Container.GIF || container == Container.IMAGE_SEQUENCE -> + OutputSpec(container, VideoCodec.NONE, AudioCodec.NONE) + + container.canHoldVideo -> OutputSpec(container, VideoCodec.COPY, AudioCodec.COPY) + else -> OutputSpec(container, VideoCodec.NONE, AudioCodec.COPY) + } + assertPair(cmd(spec), "-f", container.ffmpegFormat) + } + } + + @Test + fun `the new containers name the muxers FFmpeg actually uses`() { + assertEquals("matroska", Container.MKV.ffmpegFormat) + assertEquals("mpegts", Container.MPEG_TS.ffmpegFormat) + assertEquals("asf", Container.ASF.ffmpegFormat) + assertEquals("mov", Container.MOV.ffmpegFormat) + assertEquals("adts", Container.AAC_ADTS.ffmpegFormat) + } + + @Test + fun `faststart applies to the MP4 family only`() { + assertTrue(cmd(OutputFormat.MP4_H264).contains("+faststart")) + assertTrue( + cmd(OutputSpec(Container.MOV, VideoCodec.COPY, AudioCodec.COPY)) + .contains("+faststart"), + ) + assertFalse( + cmd(OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY)) + .contains("+faststart"), + ) + } + + // --- no silent substitution --------------------------------------------- + + /** + * Asking for a codec this app cannot encode must fail, not quietly become H.264. + * + * The builder used to end its codec `when` with `else -> libx264`, which is the same shape as + * the `media3MimeType()` bug that put an HEVC video track in a file named `.m4a`: the user asks + * for one thing, gets another, and nothing reports it. VP8 and AV1 are copyable but not + * encodable, so they are exactly the requests that would land in that branch. + */ + @Test + fun `requesting an unencodable codec fails instead of substituting H264`() { + listOf(VideoCodec.VP8, VideoCodec.AV1).forEach { codec -> + val failure = runCatching { + cmd(OutputSpec(Container.MKV, codec, AudioCodec.AAC)) + }.exceptionOrNull() + + assertTrue( + "$codec should be refused, not silently encoded as something else", + failure != null, + ) + assertTrue( + "the message should name the codec, got: ${failure?.message}", + failure?.message?.contains(codec.label) == true, + ) + } + } + + @Test + fun `the codecs this app does encode still produce their own encoder`() { + assertPair(cmd(OutputSpec(Container.MP4, VideoCodec.H264, AudioCodec.AAC)), "-c:v", "libx264") + assertPair(cmd(OutputSpec(Container.MP4, VideoCodec.H265, AudioCodec.AAC)), "-c:v", "libx265") + assertPair( + cmd(OutputSpec(Container.WEBM, VideoCodec.VP9, AudioCodec.OPUS)), + "-c:v", + "libvpx-vp9", + ) + } } diff --git a/app/src/test/java/org/libremediaconverter/model/CodecNamesTest.kt b/app/src/test/java/org/libremediaconverter/model/CodecNamesTest.kt new file mode 100644 index 0000000..6f21931 --- /dev/null +++ b/app/src/test/java/org/libremediaconverter/model/CodecNamesTest.kt @@ -0,0 +1,51 @@ +package org.libremediaconverter.model + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Test + +/** + * The bridge between what probing reports and what the app's enums mean. + * + * Three vocabularies meet: `MediaExtractor` MIME types, FFprobe `codec_name` strings, and the + * enums. Stream copy depends on the round trip, so a missing alias here shows up as "we could not + * identify the source codec" and silently costs the user a re-encode. + */ +class CodecNamesTest { + + @Test + fun `the aliases both probes emit resolve to the same codec`() { + listOf("h264", "avc", "avc1", "AVC1").forEach { + assertEquals("$it should be H.264", VideoCodec.H264, CodecNames.videoFromName(it)) + } + listOf("hevc", "h265", "hvc1", "hev1").forEach { + assertEquals("$it should be H.265", VideoCodec.H265, CodecNames.videoFromName(it)) + } + assertEquals(VideoCodec.AV1, CodecNames.videoFromName("av01")) + assertEquals(AudioCodec.AAC, CodecNames.audioFromName("mp4a")) + assertEquals(AudioCodec.PCM, CodecNames.audioFromName("pcm_s16le")) + } + + /** The sentinel must never look like a codec — that is the whole point of it. */ + @Test + fun `the unparseable sentinel resolves to nothing`() { + assertNull(CodecNames.videoFromName(InputProbe.UNPARSEABLE)) + } + + @Test + fun `an unknown name resolves to nothing rather than a default`() { + assertNull(CodecNames.videoFromName("cinepak")) + assertNull(CodecNames.audioFromName("qdm2")) + assertNull(CodecNames.videoFromName(null)) + } + + /** The source-info card shows these, so they must never be blank or a raw sentinel. */ + @Test + fun `descriptions stay readable for unknown and missing codecs`() { + assertEquals("H.264", CodecNames.describeVideo("h264")) + assertEquals("Unknown", CodecNames.describeVideo(null)) + assertEquals("Unrecognised", CodecNames.describeVideo(InputProbe.UNPARSEABLE)) + // An unrecognised but real codec name is more useful shown than hidden. + assertEquals("cinepak", CodecNames.describeVideo("cinepak")) + } +} diff --git a/app/src/test/java/org/libremediaconverter/model/ContainerCapabilitiesTest.kt b/app/src/test/java/org/libremediaconverter/model/ContainerCapabilitiesTest.kt new file mode 100644 index 0000000..ff88663 --- /dev/null +++ b/app/src/test/java/org/libremediaconverter/model/ContainerCapabilitiesTest.kt @@ -0,0 +1,171 @@ +package org.libremediaconverter.model + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +/** + * The matrix that replaced the closed format enum. + * + * `OutputFormat` used to be twelve hand-picked triples, and its KDoc defended that on the grounds + * that a closed set was what made routing decidable. Opening it up moves that burden here, so this + * is where decidability now has to be proven. + */ +class ContainerCapabilitiesTest { + + private val h264Source = InputProbe( + videoCodec = "h264", + audioCodec = "aac", + container = Container.MP4, + ) + + // --- copy and encode are different questions ---------------------------- + + /** + * The case that makes the mode axis necessary. + * + * A single boolean would have to answer one way or the other, and either answer is wrong half + * the time: refusing AV1 in MP4 blocks a legitimate remux, allowing it promises an encode + * neither engine can deliver. + */ + @Test + fun `MP4 carries AV1 on copy but cannot encode it`() { + assertTrue(ContainerCapabilities.accepts(Container.MP4, VideoCodec.AV1, CodecMode.COPY)) + assertFalse(ContainerCapabilities.accepts(Container.MP4, VideoCodec.AV1, CodecMode.ENCODE)) + } + + @Test + fun `Matroska carries Vorbis on copy but nothing here encodes it`() { + assertTrue(ContainerCapabilities.accepts(Container.MKV, AudioCodec.VORBIS, CodecMode.COPY)) + assertFalse( + ContainerCapabilities.accepts(Container.MKV, AudioCodec.VORBIS, CodecMode.ENCODE), + ) + } + + @Test + fun `a codec the container cannot hold is refused in both modes`() { + listOf(CodecMode.COPY, CodecMode.ENCODE).forEach { mode -> + assertFalse( + "WebM should never accept H.264 ($mode)", + ContainerCapabilities.accepts(Container.WEBM, VideoCodec.H264, mode), + ) + assertFalse( + "WAV should never accept AAC ($mode)", + ContainerCapabilities.accepts(Container.WAV, AudioCodec.AAC, mode), + ) + } + } + + @Test + fun `H265 in AVI is refused — AVI predates it`() { + assertFalse(ContainerCapabilities.accepts(Container.AVI, VideoCodec.H265, CodecMode.COPY)) + assertTrue(ContainerCapabilities.accepts(Container.AVI, VideoCodec.H264, CodecMode.COPY)) + } + + @Test + fun `resolving COPY before asking the matrix is required`() { + // The matrix cannot answer for COPY; the caller has to resolve it against the probe first. + // Failing loudly is what stops a caller from silently getting "false" and refusing a + // perfectly good remux. + runCatching { ContainerCapabilities.accepts(Container.MP4, VideoCodec.COPY, CodecMode.COPY) } + .onSuccess { throw AssertionError("expected COPY to be rejected by the matrix") } + } + + // --- validation --------------------------------------------------------- + + @Test + fun `every preset is a valid spec`() { + OutputFormat.entries.forEach { preset -> + val result = ContainerCapabilities.validate(preset.spec, h264Source) + assertTrue("${preset.name} is not valid: $result", result.isValid) + } + } + + /** A suggestion that is itself invalid is worse than no suggestion. */ + @Test + fun `every suggestion is itself valid`() { + val broken = OutputSpec(Container.WEBM, VideoCodec.H264, AudioCodec.AAC) + val result = ContainerCapabilities.validate(broken, h264Source) + + val invalid = result as? Validation.Invalid + ?: throw AssertionError("expected H.264 in WebM to be rejected") + assertTrue("no alternatives offered", invalid.suggestions.isNotEmpty()) + invalid.suggestions.forEach { suggestion -> + assertTrue( + "suggested $suggestion is itself invalid", + ContainerCapabilities.validate(suggestion, h264Source).isValid, + ) + } + } + + @Test + fun `an unidentifiable source codec cannot be copied`() { + val unknown = InputProbe(videoCodec = InputProbe.UNPARSEABLE, audioCodec = null) + val spec = OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.NONE) + + val result = ContainerCapabilities.validate(spec, unknown) + assertFalse("copying an unknown codec must be refused", result.isValid) + } + + @Test + fun `copying a video track into an audio-only container is refused`() { + val spec = OutputSpec(Container.MP3, VideoCodec.COPY, AudioCodec.MP3) + val result = ContainerCapabilities.validate(spec, h264Source) + + val invalid = result as? Validation.Invalid + ?: throw AssertionError("expected video in MP3 to be rejected") + assertTrue(invalid.message.contains("audio only")) + } + + @Test + fun `dropping both tracks is refused rather than producing an empty file`() { + val spec = OutputSpec(Container.MP4, VideoCodec.NONE, AudioCodec.NONE) + val result = ContainerCapabilities.validate(spec, h264Source) + + assertFalse(result.isValid) + assertTrue((result as Validation.Invalid).suggestions.isNotEmpty()) + } + + @Test + fun `copying is offered as the fix when the codec is right but unencodable`() { + val av1Source = InputProbe(videoCodec = "av1", audioCodec = "aac", container = Container.MKV) + val spec = OutputSpec(Container.MP4, VideoCodec.AV1, AudioCodec.AAC) + + val invalid = ContainerCapabilities.validate(spec, av1Source) as? Validation.Invalid + ?: throw AssertionError("expected an AV1 encode to be rejected") + assertTrue( + "should offer to copy the AV1 track instead, got ${invalid.suggestions}", + invalid.suggestions.any { it.videoCodec == VideoCodec.COPY }, + ) + } + + // --- the picker reads these --------------------------------------------- + + @Test + fun `encodable lists never contain a codec the container cannot hold`() { + Container.entries.forEach { container -> + ContainerCapabilities.encodableVideo(container).forEach { + assertTrue( + "$container claims to encode $it but cannot hold it", + ContainerCapabilities.accepts(container, it, CodecMode.COPY), + ) + } + ContainerCapabilities.encodableAudio(container).forEach { + assertTrue( + "$container claims to encode $it but cannot hold it", + ContainerCapabilities.accepts(container, it, CodecMode.COPY), + ) + } + } + } + + @Test + fun `audio-only containers offer no video codecs`() { + listOf(Container.MP3, Container.WAV, Container.FLAC, Container.OGG, Container.AAC_ADTS) + .forEach { container -> + assertFalse("$container claims to hold video", container.canHoldVideo) + assertEquals(emptyList(), ContainerCapabilities.encodableVideo(container)) + } + } +} diff --git a/app/src/test/java/org/libremediaconverter/model/ConversionRouterTest.kt b/app/src/test/java/org/libremediaconverter/model/ConversionRouterTest.kt index 36ab736..3d1a81f 100644 --- a/app/src/test/java/org/libremediaconverter/model/ConversionRouterTest.kt +++ b/app/src/test/java/org/libremediaconverter/model/ConversionRouterTest.kt @@ -20,7 +20,18 @@ class ConversionRouterTest { probe: InputProbe = InputProbe(videoCodec = "h264"), device: DeviceCodecs = DeviceCodecs.PERMISSIVE, ) = ConversionRouter.route( - ConversionRequest(format, quality, preference, probe), + ConversionRequest(format.spec, quality, preference, probe), + device, + ) + + private fun route( + spec: OutputSpec, + quality: QualityTier = QualityTier.FAST, + preference: EnginePreference = EnginePreference.AUTO, + probe: InputProbe = InputProbe(videoCodec = "h264"), + device: DeviceCodecs = DeviceCodecs.PERMISSIVE, + ) = ConversionRouter.route( + ConversionRequest(spec, quality, preference, probe), device, ) @@ -173,6 +184,126 @@ class ConversionRouterTest { assertEquals(Reason.USER_FORCED_SOFTWARE, d.reason) } + // --- remux -------------------------------------------------------------- + + /** + * The test that keeps the Media3 transmux path alive. + * + * `COPY` belongs to none of the router's capability sets, so the obvious implementation — test + * the request's codecs directly — sends every remux to FFmpeg on the very first check. Nothing + * would fail: FFmpeg's `-c copy` produces a correct file, just on the CPU. Only an assertion + * about the *engine* catches it, which is why this one exists. + */ + @Test + fun `a full stream copy into a Media3 container stays on hardware`() { + val d = route( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "h264", audioCodec = "aac", container = Container.MKV), + ) + assertEquals(Engine.MEDIA3, d.engine) + assertEquals(Reason.REMUX_NO_REENCODE, d.reason) + } + + /** A copy needs no encoder at all, so a device with none must not push it to software. */ + @Test + fun `a stream copy ignores the device encoder capabilities`() { + val noEncoders = object : DeviceCodecs { + override fun canEncode(codec: VideoCodec) = false + override fun canDecode(codecName: String) = true + } + val d = route( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "h264", audioCodec = "aac", container = Container.MKV), + device = noEncoders, + ) + assertEquals(Engine.MEDIA3, d.engine) + } + + /** Nor does it decode, so a codec the device cannot decode is still copyable. */ + @Test + fun `a stream copy ignores the device decoder capabilities`() { + val noDecoders = object : DeviceCodecs { + override fun canEncode(codec: VideoCodec) = true + override fun canDecode(codecName: String) = false + } + val d = route( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "av1", audioCodec = "aac", container = Container.MKV), + device = noDecoders, + ) + assertEquals(Engine.MEDIA3, d.engine) + } + + /** + * Copying MP3 audio into MP4 needs no encoder — but Media3's MP4 muxer cannot carry MP3. + * + * Two separate limits that used to be indistinguishable because nothing could reach this + * combination. The encoder gap is real and permanent (Android has no MP3 encoder); the muxer + * gap is what actually decides this job, and the reason has to say so rather than blaming an + * encoder nobody asked for. + */ + @Test + fun `copying MP3 audio into MP4 needs FFmpeg because Media3 cannot mux it`() { + val d = route( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "h264", audioCodec = "mp3", container = Container.MKV), + ) + assertEquals(Engine.FFMPEG, d.engine) + assertEquals(Reason.CONTAINER_CODEC_UNSUPPORTED, d.reason) + } + + /** Opus, by contrast, Media3's MP4 muxer does carry. */ + @Test + fun `copying Opus audio into MP4 stays on hardware`() { + val d = route( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = "h264", audioCodec = "opus", container = Container.MKV), + ) + assertEquals(Engine.MEDIA3, d.engine) + assertEquals(Reason.REMUX_NO_REENCODE, d.reason) + } + + /** But Best quality is meaningless for a copy, so it must not force software. */ + @Test + fun `best quality does not force software for a pure remux`() { + val d = route( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + quality = QualityTier.BEST, + probe = InputProbe(videoCodec = "h264", audioCodec = "aac", container = Container.MKV), + ) + assertEquals(Engine.MEDIA3, d.engine) + } + + /** An unreadable file cannot be copied either — there is nothing to demux. */ + @Test + fun `an unparseable input still goes to FFmpeg even when a copy was asked for`() { + val d = route( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe(videoCodec = InputProbe.UNPARSEABLE), + ) + assertEquals(Engine.FFMPEG, d.engine) + assertEquals(Reason.NO_PLATFORM_DECODER, d.reason) + } + + @Test + fun `the new containers are all FFmpeg-only`() { + listOf( + Container.MOV, Container.MKV, Container.MPEG_TS, + Container.AVI, Container.FLV, Container.ASF, + ).forEach { container -> + val d = route( + OutputSpec(container, VideoCodec.COPY, AudioCodec.COPY), + probe = InputProbe( + videoCodec = "h264", + audioCodec = "aac", + container = Container.MP4, + ), + ) + assertEquals("$container should need FFmpeg", Engine.FFMPEG, d.engine) + assertEquals(Reason.CONTAINER_UNSUPPORTED, d.reason) + } + } + // --- exhaustiveness ---------------------------------------------------- @Test diff --git a/app/src/test/java/org/libremediaconverter/model/CopyPlannerTest.kt b/app/src/test/java/org/libremediaconverter/model/CopyPlannerTest.kt new file mode 100644 index 0000000..58fb1cd --- /dev/null +++ b/app/src/test/java/org/libremediaconverter/model/CopyPlannerTest.kt @@ -0,0 +1,159 @@ +package org.libremediaconverter.model + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +/** + * The stream-copy decision for a single file. + * + * Deliberately mirrors [ConcatPlannerTest]'s shape, because it enforces the same rule for the same + * reason: an unproven match must never become a stream copy. A needless re-encode costs time; a + * wrong copy costs the user a file that will not play, and they may not notice until the source is + * gone. + */ +class CopyPlannerTest { + + private val mp4H264 = InputProbe( + videoCodec = "h264", + audioCodec = "aac", + container = Container.MP4, + ) + + // --- explicit copy ------------------------------------------------------ + + @Test + fun `asking to copy a known codec the container accepts copies it`() { + val plan = CopyPlanner.plan( + OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.COPY), + mp4H264, + ) + assertEquals(VideoPlan.Copy, plan.video) + assertEquals(AudioPlan.Copy, plan.audio) + assertTrue("this is a pure container change", plan.isPureRemux) + } + + @Test + fun `an unparseable source is never copied`() { + val plan = CopyPlanner.plan( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + InputProbe(videoCodec = InputProbe.UNPARSEABLE, audioCodec = null), + ) + assertTrue("must not copy an unidentified stream", plan.video !is VideoPlan.Copy) + assertTrue(plan.audio !is AudioPlan.Copy) + } + + /** Two unknowns are not evidence of agreement — the rule ConcatPlanner already states. */ + @Test + fun `an unrecognised codec name falls back to re-encoding`() { + val exotic = InputProbe(videoCodec = "cinepak", audioCodec = "qdm2", container = Container.MOV) + val plan = CopyPlanner.plan( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.COPY), + exotic, + ) + assertTrue(plan.video is VideoPlan.Encode) + assertTrue(plan.audio is AudioPlan.Encode) + } + + @Test + fun `copying into a container that cannot hold the codec re-encodes instead`() { + // H.264 cannot go in WebM. Asking to copy anyway must not produce a broken file. + val plan = CopyPlanner.plan( + OutputSpec(Container.WEBM, VideoCodec.COPY, AudioCodec.NONE), + mp4H264, + ) + assertEquals(VideoPlan.Encode(VideoCodec.VP9), plan.video) + } + + // --- the auto-upgrade rule ---------------------------------------------- + + @Test + fun `a matching codec with a changing container is upgraded to a copy`() { + val plan = CopyPlanner.plan( + OutputSpec(Container.MKV, VideoCodec.H264, AudioCodec.AAC), + mp4H264, + ) + assertEquals("MP4 h264 -> MKV h264 is a remux", VideoPlan.Copy, plan.video) + assertEquals(AudioPlan.Copy, plan.audio) + } + + /** + * The case that stops auto-upgrade from breaking compression. + * + * Same container, same codec means the only reason to run the job is to re-encode it — almost + * always to make it smaller. Copying would hand back a byte-identical file and call it done. + */ + @Test + fun `a matching codec in the same container still re-encodes`() { + val plan = CopyPlanner.plan( + OutputSpec(Container.MP4, VideoCodec.H264, AudioCodec.AAC), + mp4H264, + ) + assertEquals(VideoPlan.Encode(VideoCodec.H264), plan.video) + assertEquals(AudioPlan.Encode(AudioCodec.AAC), plan.audio) + } + + @Test + fun `an unknown source container never triggers auto-upgrade`() { + // Without a source container there is no way to know the container is changing, so the + // conservative answer is to encode. + val noContainer = InputProbe(videoCodec = "h264", audioCodec = "aac", container = null) + val plan = CopyPlanner.plan( + OutputSpec(Container.MKV, VideoCodec.H264, AudioCodec.AAC), + noContainer, + ) + assertEquals(VideoPlan.Encode(VideoCodec.H264), plan.video) + } + + @Test + fun `a different codec is always re-encoded`() { + val plan = CopyPlanner.plan( + OutputSpec(Container.MKV, VideoCodec.H265, AudioCodec.OPUS), + mp4H264, + ) + assertEquals(VideoPlan.Encode(VideoCodec.H265), plan.video) + assertEquals(AudioPlan.Encode(AudioCodec.OPUS), plan.audio) + } + + // --- track removal ------------------------------------------------------ + + @Test + fun `audio-only output drops the video track`() { + val plan = CopyPlanner.plan( + OutputSpec(Container.MP4, VideoCodec.NONE, AudioCodec.AAC), + mp4H264, + ) + assertEquals(VideoPlan.Drop, plan.video) + assertTrue(plan.audio is AudioPlan.Encode) + assertTrue("dropping plus encoding is not a remux", !plan.isPureRemux) + } + + @Test + fun `a source with no video track drops video even when video was requested`() { + val audioOnly = InputProbe( + videoCodec = null, + audioCodec = "aac", + hasVideo = false, + container = Container.MP4, + kind = InputKind.AUDIO_ONLY, + ) + val plan = CopyPlanner.plan( + OutputSpec(Container.MKV, VideoCodec.H264, AudioCodec.COPY), + audioOnly, + ) + assertEquals(VideoPlan.Drop, plan.video) + assertEquals(AudioPlan.Copy, plan.audio) + assertTrue("copying the only track is still a remux", plan.isPureRemux) + } + + @Test + fun `copying one track and encoding the other is not a pure remux`() { + val plan = CopyPlanner.plan( + OutputSpec(Container.MKV, VideoCodec.COPY, AudioCodec.OPUS), + mp4H264, + ) + assertEquals(VideoPlan.Copy, plan.video) + assertEquals(AudioPlan.Encode(AudioCodec.OPUS), plan.audio) + assertTrue(!plan.isPureRemux) + } +} diff --git a/app/src/test/java/org/libremediaconverter/model/OutputFormatTest.kt b/app/src/test/java/org/libremediaconverter/model/OutputFormatTest.kt index 911d413..25fb4a0 100644 --- a/app/src/test/java/org/libremediaconverter/model/OutputFormatTest.kt +++ b/app/src/test/java/org/libremediaconverter/model/OutputFormatTest.kt @@ -61,4 +61,66 @@ class OutputFormatTest { assertTrue(it.description.isNotBlank()) } } + + // --- containers are now load-bearing ------------------------------------ + + /** + * Extension and MIME type moved from the preset onto the container. + * + * They used to be per-preset literals, which is why `OutputFormat.FLAC` could declare + * `Container.MKV` with extension `flac` and nobody noticed — nothing read the container. Now + * `-f`, the filename and the SAF create-document MIME all derive from it. + */ + @Test + fun `every container names an extension, a mime type and an ffmpeg muxer`() { + Container.entries.forEach { container -> + listOf(true, false).forEach { hasVideo -> + val ext = container.extensionFor(hasVideo) + assertTrue("$container has no extension", ext.isNotBlank()) + assertFalse("$container extension has a dot", ext.startsWith(".")) + assertTrue( + "$container has no mime type", + container.mimeTypeFor(hasVideo).contains('/'), + ) + } + assertTrue("$container names no muxer", container.ffmpegFormat.isNotBlank()) + } + } + + @Test + fun `audio-only variants of a container get their own extension`() { + assertEquals("mp4", Container.MP4.extensionFor(hasVideo = true)) + assertEquals("m4a", Container.MP4.extensionFor(hasVideo = false)) + assertEquals("mkv", Container.MKV.extensionFor(hasVideo = true)) + assertEquals("mka", Container.MKV.extensionFor(hasVideo = false)) + } + + /** Regression guard: FLAC used to be declared as Matroska with a `.flac` extension. */ + @Test + fun `FLAC is its own container, not Matroska`() { + assertEquals(Container.FLAC, OutputFormat.FLAC.container) + assertEquals("flac", OutputFormat.FLAC.extension) + assertEquals("flac", Container.FLAC.ffmpegFormat) + } + + @Test + fun `the remux presets copy both tracks`() { + listOf(OutputFormat.REMUX_MP4, OutputFormat.REMUX_MKV).forEach { + assertEquals(VideoCodec.COPY, it.videoCodec) + assertEquals(AudioCodec.COPY, it.audioCodec) + assertTrue("${it.name} should be a pure remux", it.spec.isPureRemux) + } + } + + @Test + fun `a spec that encodes anything is not a remux`() { + assertFalse(OutputFormat.MP4_H265.spec.isPureRemux) + assertFalse( + OutputSpec(Container.MP4, VideoCodec.COPY, AudioCodec.AAC).isPureRemux, + ) + assertFalse( + "dropping both tracks copies nothing", + OutputSpec(Container.MP4, VideoCodec.NONE, AudioCodec.NONE).isPureRemux, + ) + } }