From 66069bbbd94e0aa396759e6c6905e3fa33b567c1 Mon Sep 17 00:00:00 2001 From: Martin Donadieu Date: Mon, 13 Apr 2026 17:44:29 +0200 Subject: [PATCH 1/6] Add convertAudio API with iOS m4a support --- FEATURE_MATRIX.md | 19 ++- README.md | 51 ++++++- ROADMAP.md | 29 ++-- .../CapacitorFFmpegPlugin.java | 8 + .../CapacitorFFmpegPluginTest.java | 4 + example-app/src/js/app.js | 2 + example-app/src/js/app.vitest.js | 7 + .../CapacitorFFmpeg.swift | 117 ++++++++++++++- .../CapacitorFFmpegPlugin.swift | 28 ++++ .../CapacitorFFmpegPluginTests.swift | 138 +++++++++++++++++- src/definitions.ts | 21 +++ src/web.ts | 15 +- test/TEST_INVENTORY.md | 6 +- test/web.test.ts | 12 ++ 14 files changed, 418 insertions(+), 39 deletions(-) diff --git a/FEATURE_MATRIX.md b/FEATURE_MATRIX.md index caf89be..a77299e 100644 --- a/FEATURE_MATRIX.md +++ b/FEATURE_MATRIX.md @@ -1,6 +1,6 @@ # Feature Matrix -Last updated: 2026-04-04 +Last updated: 2026-04-13 This file tracks the public plugin contract and the current platform support level for each capability. @@ -12,19 +12,20 @@ This file tracks the public plugin contract and the current platform support lev | `getPluginVersion()` | Resolves `{ version: string }` on iOS, Android, and web | Web returns `"web"` as a platform marker. | | `reencodeVideo(options)` | Resolves when the job is accepted by iOS and returns `{ jobId, status: "queued" }` when available | Android and web reject with `UNIMPLEMENTED`. | | `convertImage(options)` | Resolves `{ outputPath, format }` on iOS and Android | Web rejects with `UNIMPLEMENTED`. | +| `convertAudio(options)` | Resolves `{ outputPath, format }` on iOS for `m4a` output | Android and web reject with `UNIMPLEMENTED`. | | `progress` listener | Emits `{ jobId, progress, state, message?, outputPath? }` | `fileId` is kept as a compatibility alias for `jobId`. | ## Failure contract Use these codes as the shared error vocabulary for JS consumers: -| Code | Meaning | Current producers | -| ------------------------ | ------------------------------------------------------------ | ----------------------------------------------- | -| `UNIMPLEMENTED` | The API is not implemented on the current platform | Android/web `reencodeVideo`, web `convertImage` | -| `UNAVAILABLE` | The API exists but cannot be used in the current environment | Reserved for future capabilities | -| `INVALID_ARGUMENT` | Caller input failed local validation | iOS `reencodeVideo` validation | -| `PLUGIN_NOT_INITIALIZED` | Native core could not be initialized | iOS native wrapper | -| `TRANSCODE_FAILED` | The media pipeline failed after acceptance or during setup | iOS native wrapper and Rust bridge | +| Code | Meaning | Current producers | +| ------------------------ | ------------------------------------------------------------ | --------------------------------------------------------------------------- | +| `UNIMPLEMENTED` | The API is not implemented on the current platform | Android/web `reencodeVideo`, Android/web `convertAudio`, web `convertImage` | +| `UNAVAILABLE` | The API exists but cannot be used in the current environment | Reserved for future capabilities | +| `INVALID_ARGUMENT` | Caller input failed local validation | iOS media wrapper validation | +| `PLUGIN_NOT_INITIALIZED` | Native core could not be initialized | iOS native wrapper | +| `TRANSCODE_FAILED` | The media pipeline failed after acceptance or during setup | iOS native wrappers and Rust bridge | ## Platform support @@ -35,6 +36,7 @@ Use these codes as the shared error vocabulary for JS consumers: | `reencodeVideo` acceptance contract | ✅ | ❌ `UNIMPLEMENTED` | ❌ `UNIMPLEMENTED` | Covered for iOS helpers and web/Android unsupported behavior | | `reencodeVideo` media execution | ⚠️ Experimental | ❌ | ❌ | Wrapper contract covered; media regression fixtures still pending | | `convertImage` | ✅ `jpeg`/`png` | ✅ `webp`/`jpeg`/`png` | ❌ `UNIMPLEMENTED` | Covered by iOS native tests, Android unit tests, and Maestro flows | +| `convertAudio` | ✅ `m4a` | ❌ `UNIMPLEMENTED` | ❌ `UNIMPLEMENTED` | Covered by iOS native tests plus web and Android unsupported tests | | `progress` listener contract | ✅ | ❌ | ❌ | Covered by iOS helper tests | | `probeMedia` | ❌ Planned | ❌ Planned | ❌ Planned | Not started | | `generateThumbnail` | ❌ Planned | ❌ Planned | ❌ Planned | Not started | @@ -46,6 +48,7 @@ Use these codes as the shared error vocabulary for JS consumers: - iOS is still the reference platform for media work. - `convertImage()` does not depend on the Rust FFmpeg core and remains available even when `reencodeVideo()` is unavailable in SwiftPM builds. +- `convertAudio()` currently relies on `AVAssetExportSession` on iOS and is limited to `m4a` output. - Android currently implements image conversion without the broader FFmpeg job pipeline. - `getCapabilities()` is the machine-readable source of truth for app-side feature gating. - Android and web should reject unsupported media APIs explicitly instead of failing implicitly. diff --git a/README.md b/README.md index c53276a..d2b9990 100644 --- a/README.md +++ b/README.md @@ -53,14 +53,15 @@ bunx cap sync | `getPluginVersion` | ✅ | ✅ | ✅ | Returns a `{ version }` payload on every platform; use `getCapabilities().platform` for platform detection. | | `reencodeVideo` | ⚠️ Experimental | ❌ | ❌ | iOS accepts a queued job and reports lifecycle via `progress`; Android and web reject with `UNIMPLEMENTED`. | | `convertImage` | ✅ | ✅ | ❌ | iOS converts still images to `jpeg` or `png`; Android converts to `webp`, `jpeg`, or `png`; web rejects. | +| `convertAudio` | ✅ | ❌ | ❌ | iOS converts audio to `m4a`; Android and web reject with `UNIMPLEMENTED`. | ## Platform status -| Platform | Status | Notes | -| -------- | ----------------------------- | ---------------------------------------------------------------------------------------- | -| iOS | Early implementation | Current reference platform for media work. | -| Android | Partial native implementation | `convertImage` is native; the broader FFmpeg media engine still needs to be implemented. | -| Web | Stub only | Media operations are intentionally unsupported right now. | +| Platform | Status | Notes | +| -------- | ----------------------------- | ---------------------------------------------------------------------------------------------------------- | +| iOS | Early implementation | Current reference platform for media work. | +| Android | Partial native implementation | `convertImage` is native; the broader FFmpeg media engine and `convertAudio` still need to be implemented. | +| Web | Stub only | Media operations are intentionally unsupported right now. | ## API @@ -69,6 +70,7 @@ bunx cap sync - [`getCapabilities()`](#getcapabilities) - [`reencodeVideo(...)`](#reencodevideo) - [`convertImage(...)`](#convertimage) +- [`convertAudio(...)`](#convertaudio) - [`addListener('progress', ...)`](#addlistenerprogress-) - [`getPluginVersion()`](#getpluginversion) - [Interfaces](#interfaces) @@ -132,6 +134,25 @@ Web currently rejects with `UNIMPLEMENTED`. --- +### convertAudio(...) + +```typescript +convertAudio(options: ConvertAudioOptions) => Promise +``` + +Convert audio into another container or codec. + +iOS currently supports `m4a`. +Android and web currently reject with `UNIMPLEMENTED`. + +| Param | Type | +| ------------- | ------------------------------------------------------------------- | +| **`options`** | ConvertAudioOptions | + +**Returns:** Promise<ConvertAudioResult> + +--- + ### addListener('progress', ...) ```typescript @@ -178,6 +199,7 @@ Get the plugin package version reported by the current platform implementation. | **`getCapabilities`** | FFmpegCapability | | **`reencodeVideo`** | FFmpegCapability | | **`convertImage`** | FFmpegCapability | +| **`convertAudio`** | FFmpegCapability | | **`progressEvents`** | FFmpegCapability | | **`probeMedia`** | FFmpegCapability | | **`generateThumbnail`** | FFmpegCapability | @@ -225,6 +247,21 @@ Get the plugin package version reported by the current platform implementation. | **`format`** | ImageOutputFormat | | | **`quality`** | number | Compression quality in the inclusive range `0.0..1.0`. Native platforms reject values outside that range. | +#### ConvertAudioResult + +| Prop | Type | +| ---------------- | --------------------------------------------------------------- | +| **`outputPath`** | string | +| **`format`** | AudioOutputFormat | + +#### ConvertAudioOptions + +| Prop | Type | +| ---------------- | --------------------------------------------------------------- | +| **`inputPath`** | string | +| **`outputPath`** | string | +| **`format`** | AudioOutputFormat | + #### PluginListenerHandle | Prop | Type | @@ -258,6 +295,10 @@ Get the plugin package version reported by the current platform implementation. 'webp' | 'jpeg' | 'png' +#### AudioOutputFormat + +'m4a' + #### FFmpegProgressState 'running' | 'completed' | 'failed' diff --git a/ROADMAP.md b/ROADMAP.md index 9fbc5fb..01ea9c4 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,18 +1,18 @@ # Capacitor FFmpeg Roadmap -Last updated: 2026-04-04 +Last updated: 2026-04-13 Status: active planning ## Why this file exists This plugin is currently much smaller than FFmpeg itself: -- The TypeScript API exposes `getCapabilities()`, `reencodeVideo()`, `convertImage()`, a `progress` event, and `getPluginVersion()`. +- The TypeScript API exposes `getCapabilities()`, `reencodeVideo()`, `convertImage()`, `convertAudio()`, a `progress` event, and `getPluginVersion()`. - The Rust core currently focuses on one video re-encode path: decode video, encode H.264, and copy non-video streams. -- iOS has the only real native media implementation today. -- Android exposes `getPluginVersion()`, `getCapabilities()`, and `convertImage()`. +- iOS has the only real native media implementation today, including still-image conversion and a first audio-conversion path. +- Android exposes `getPluginVersion()`, `getCapabilities()`, `convertImage()`, and an explicit `convertAudio()` contract that still rejects with `UNIMPLEMENTED`. - Web is a stub. -- The test suite now covers basic wrapper contracts, but not media regressions yet. +- The test suite now covers basic wrapper contracts plus iOS image and audio helper flows, but not full media regressions yet. This roadmap is the living source of truth for how we expand the plugin safely, how we test it, and how we keep its scope realistic. @@ -47,15 +47,15 @@ The right goal is a well-documented, well-tested mobile subset of FFmpeg, with a ## Current baseline -| Area | Current state | Main gap | -| --------- | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- | -| JS API | `getCapabilities`, `reencodeVideo`, `progress`, `getPluginVersion` | No probe, trim, remux, thumbnails, extract-audio, cancel, or cross-platform job execution | -| Rust core | One re-encode flow with progress callback | No unit tests, no fixture suite, no stable result model, no broader operation set | -| iOS | Real implementation exists | Tied to a narrow workflow and limited test coverage | -| Android | Version method only | No media engine, no API parity | -| Web | Stub | No documented support story | -| Tests | Basic web/iOS/Android contract tests | No media fixture coverage, no Rust regressions, and no end-to-end flows | -| Docs | README documents support and roadmap links | No generated API docs yet for planned operations and no capability matrix by phase | +| Area | Current state | Main gap | +| --------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | +| JS API | `getCapabilities`, `reencodeVideo`, `convertImage`, `convertAudio`, `progress`, `getPluginVersion` | No probe, trim, remux, thumbnails, extract-audio, cancel, or cross-platform media parity | +| Rust core | One re-encode flow with progress callback | No unit tests, no fixture suite, no stable result model, no broader operation set | +| iOS | Real implementation exists for video, image, and limited audio conversion | Still tied to narrow workflows and lacks real end-to-end regression fixtures | +| Android | Native image conversion plus explicit unsupported contracts for broader media APIs | No media engine parity for audio or video workflows | +| Web | Stub | No documented support story | +| Tests | Basic web/iOS/Android contract tests plus iOS image/audio helper tests | No media fixture coverage, no Rust regressions, and no end-to-end flows | +| Docs | README documents support and roadmap links | No capability matrix by release phase and limited examples for new media operations | ## Scope model @@ -160,6 +160,7 @@ Exit criteria: Status: in progress - Stabilize the current re-encode path. +- Harden the initial `convertAudio()` path and keep its scope explicit until broader format support is real. - Add structured progress and cancellation semantics. - Clarify background execution behavior and platform minimums. - Add real iOS native tests using media fixtures. diff --git a/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java b/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java index f333b83..349365d 100644 --- a/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java +++ b/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java @@ -90,6 +90,11 @@ public void convertImage(final PluginCall call) { } } + @PluginMethod + public void convertAudio(final PluginCall call) { + call.unimplemented(getUnsupportedOperationMessage("convertAudio")); + } + @PluginMethod public void getPluginVersion(final PluginCall call) { try { @@ -118,6 +123,7 @@ JSObject createCapabilitiesFeaturesPayload() { features.put("getCapabilities", createCapabilityPayload("getCapabilities")); features.put("reencodeVideo", createCapabilityPayload("reencodeVideo")); features.put("convertImage", createCapabilityPayload("convertImage")); + features.put("convertAudio", createCapabilityPayload("convertAudio")); features.put("progressEvents", createCapabilityPayload("progressEvents")); features.put("probeMedia", createCapabilityPayload("probeMedia")); features.put("generateThumbnail", createCapabilityPayload("generateThumbnail")); @@ -150,6 +156,7 @@ String getCapabilityStatus(final String feature) { case "getPluginVersion", "getCapabilities" -> "available"; case "reencodeVideo" -> "unimplemented"; case "convertImage" -> "available"; + case "convertAudio" -> "unimplemented"; case "progressEvents" -> "unavailable"; case "probeMedia", "generateThumbnail", "extractAudio", "remux", "trim" -> "unimplemented"; default -> "unimplemented"; @@ -160,6 +167,7 @@ String getCapabilityReason(final String feature) { return switch (feature) { case "reencodeVideo" -> getUnsupportedOperationMessage("reencodeVideo"); case "convertImage" -> "Still-image conversion is available on Android for webp, jpeg, and png outputs."; + case "convertAudio" -> getUnsupportedOperationMessage("convertAudio"); case "progressEvents" -> "No media jobs are available on Android today."; case "probeMedia" -> "probeMedia is not implemented on Android."; case "generateThumbnail" -> "generateThumbnail is not implemented on Android."; diff --git a/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java b/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java index 0132c91..36efe6b 100644 --- a/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java +++ b/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java @@ -43,11 +43,13 @@ public void capabilityHelpersDescribeTheCurrentAndroidScope() { assertEquals("available", plugin.getCapabilityStatus("getCapabilities")); assertEquals("unimplemented", plugin.getCapabilityStatus("reencodeVideo")); assertEquals("available", plugin.getCapabilityStatus("convertImage")); + assertEquals("unimplemented", plugin.getCapabilityStatus("convertAudio")); assertEquals("reencodeVideo is currently only available on iOS.", plugin.getCapabilityReason("reencodeVideo")); assertEquals( "Still-image conversion is available on Android for webp, jpeg, and png outputs.", plugin.getCapabilityReason("convertImage") ); + assertEquals("convertAudio is currently only available on iOS.", plugin.getCapabilityReason("convertAudio")); } @Test @@ -60,6 +62,8 @@ public void capabilitiesPayloadExposesTheJsVisibleContract() throws Exception { assertEquals("available", features.getJSONObject("getCapabilities").getString("status")); assertEquals("unimplemented", features.getJSONObject("reencodeVideo").getString("status")); assertEquals("reencodeVideo is currently only available on iOS.", features.getJSONObject("reencodeVideo").getString("reason")); + assertEquals("unimplemented", features.getJSONObject("convertAudio").getString("status")); + assertEquals("convertAudio is currently only available on iOS.", features.getJSONObject("convertAudio").getString("reason")); } @Test diff --git a/example-app/src/js/app.js b/example-app/src/js/app.js index 75eabe7..b9ee2c3 100644 --- a/example-app/src/js/app.js +++ b/example-app/src/js/app.js @@ -6,6 +6,7 @@ const FEATURE_ORDER = [ 'getCapabilities', 'reencodeVideo', 'convertImage', + 'convertAudio', 'progressEvents', 'probeMedia', 'generateThumbnail', @@ -19,6 +20,7 @@ const FEATURE_LABELS = { getCapabilities: 'Capability matrix', reencodeVideo: 'Re-encode video', convertImage: 'Convert image', + convertAudio: 'Convert audio', progressEvents: 'Progress events', probeMedia: 'Probe media', generateThumbnail: 'Generate thumbnail', diff --git a/example-app/src/js/app.vitest.js b/example-app/src/js/app.vitest.js index e318a5a..daf020e 100644 --- a/example-app/src/js/app.vitest.js +++ b/example-app/src/js/app.vitest.js @@ -24,6 +24,13 @@ function createCapabilities(platform, overrides = {}) { ? 'Still-image conversion is available on iOS.' : 'Image conversion is currently only available on iOS and Android.', }, + convertAudio: { + status: overrides.convertAudio ?? 'unimplemented', + reason: + overrides.convertAudio === 'available' + ? 'Audio conversion is available on iOS.' + : 'Audio conversion is currently only available on iOS.', + }, progressEvents: { status: overrides.reencodeVideo === 'experimental' ? 'available' : 'unavailable', reason: diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift index ac76024..9dfe3bd 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift @@ -1,4 +1,5 @@ import CapacitorFFmpegNativeCore +import AVFoundation import Foundation import ImageIO import UIKit @@ -40,6 +41,19 @@ protocol FFmpegNativeBinding { ) -> UnsafeMutablePointer? } +protocol AudioExportSessioning: AnyObject { + var outputURL: URL? { get set } + var outputFileType: AVFileType? { get set } + var status: AVAssetExportSession.Status { get } + var error: Error? { get } + + func exportAsynchronously(completionHandler handler: @escaping @Sendable () -> Void) +} + +extension AVAssetExportSession: AudioExportSessioning {} + +typealias AudioExportSessionFactory = (_ asset: AVAsset, _ presetName: String) -> (any AudioExportSessioning)? + private struct LinkedFFmpegNativeBindings: FFmpegNativeBinding { func initPlugin() -> UnsafeMutableRawPointer? { init_ffmpeg_plugin() @@ -100,6 +114,18 @@ struct FFmpegConvertedImage { } } +struct FFmpegConvertedAudio { + let outputPath: String + let format: String + + var asDictionary: [String: Any] { + [ + "outputPath": outputPath, + "format": format + ] + } +} + struct FFmpegCapabilityPayload { let status: String let reason: String? @@ -144,6 +170,10 @@ struct FFmpegCapabilitiesPayload { status: "available", reason: "Still-image conversion is available on iOS for jpeg and png outputs." ), + "convertAudio": FFmpegCapabilityPayload( + status: "available", + reason: "Audio conversion is available on iOS for m4a output." + ), "progressEvents": FFmpegCapabilityPayload( status: nativeCoreAvailable ? "available" : "unavailable", reason: nativeCoreAvailable @@ -232,17 +262,29 @@ private final class SelfForReencodeVideo { @objc public class CapacitorFFmpeg: NSObject { var pointerToRustPlugin: UnsafeMutableRawPointer? private let nativeBindings: any FFmpegNativeBinding + private let audioExportSessionFactory: AudioExportSessionFactory private let nativeCoreReason: String? /// Progress callback closure that can be set from outside var onProgress: ((FFmpegProgressPayload) -> Void)? - public override convenience init() { + override public convenience init() { self.init(nativeBindings: LinkedFFmpegNativeBindings()) } - init(nativeBindings: any FFmpegNativeBinding) { + convenience init(audioExportSessionFactory: @escaping AudioExportSessionFactory) { + self.init( + nativeBindings: LinkedFFmpegNativeBindings(), + audioExportSessionFactory: audioExportSessionFactory + ) + } + + init( + nativeBindings: any FFmpegNativeBinding, + audioExportSessionFactory: @escaping AudioExportSessionFactory = CapacitorFFmpeg.defaultAudioExportSessionFactory + ) { self.nativeBindings = nativeBindings + self.audioExportSessionFactory = audioExportSessionFactory let resolvedPlugin = nativeBindings.initPlugin() self.pointerToRustPlugin = resolvedPlugin self.nativeCoreReason = resolvedPlugin == nil ? "The native FFmpeg core could not be initialized." : nil @@ -254,6 +296,13 @@ private final class SelfForReencodeVideo { } } + private static func defaultAudioExportSessionFactory( + asset: AVAsset, + presetName: String + ) -> (any AudioExportSessioning)? { + AVAssetExportSession(asset: asset, presetName: presetName) + } + deinit { if let plugin = self.pointerToRustPlugin { nativeBindings.deinitPlugin(plugin) @@ -469,6 +518,65 @@ private final class SelfForReencodeVideo { ) } + func convertAudio( + inputPath: String, + outputPath: String, + format: String + ) throws -> FFmpegConvertedAudio { + let inputURL = try resolveFileURL(from: inputPath).standardizedFileURL + let outputURL = try resolveFileURL(from: outputPath).standardizedFileURL + let normalizedFormat = format.lowercased() + + guard inputURL.path != outputURL.path else { + throw FFmpegError.invalidArgument("In-place conversion is not allowed. Choose a different output path.") + } + + guard normalizedFormat == "m4a" else { + throw FFmpegError.invalidArgument("Unsupported audio format: \(format)") + } + + let asset = AVURLAsset(url: inputURL) + guard let audioTrack = asset.tracks(withMediaType: .audio).first else { + throw FFmpegError.invalidArgument("The input media does not contain an audio track.") + } + _ = audioTrack + + let fileManager = FileManager.default + let outputDirectory = outputURL.deletingLastPathComponent() + try fileManager.createDirectory( + at: outputDirectory, + withIntermediateDirectories: true + ) + + if fileManager.fileExists(atPath: outputURL.path) { + try fileManager.removeItem(at: outputURL) + } + guard let exportSession = audioExportSessionFactory(asset, AVAssetExportPresetAppleM4A) else { + throw FFmpegError.transcodeFailed("Could not create the audio export session.") + } + + exportSession.outputURL = outputURL + exportSession.outputFileType = .m4a + + let semaphore = DispatchSemaphore(value: 0) + exportSession.exportAsynchronously { + semaphore.signal() + } + semaphore.wait() + + guard exportSession.status == .completed else { + try? fileManager.removeItem(at: outputURL) + throw FFmpegError.transcodeFailed( + exportSession.error?.localizedDescription ?? "Could not export the output audio." + ) + } + + return FFmpegConvertedAudio( + outputPath: outputURL.absoluteString, + format: normalizedFormat + ) + } + func getCapabilities() -> FFmpegCapabilitiesPayload { FFmpegCapabilitiesPayload.iosCurrent( nativeCoreAvailable: pointerToRustPlugin != nil, @@ -481,6 +589,7 @@ private final class SelfForReencodeVideo { public enum FFmpegError: LocalizedError { case pluginNotInitialized case reencodingFailed(String) + case transcodeFailed(String) case invalidPath(String) case invalidArgument(String) @@ -488,7 +597,7 @@ public enum FFmpegError: LocalizedError { switch self { case .pluginNotInitialized: return "PLUGIN_NOT_INITIALIZED" - case .reencodingFailed: + case .reencodingFailed, .transcodeFailed: return "TRANSCODE_FAILED" case .invalidPath, .invalidArgument: return "INVALID_ARGUMENT" @@ -501,6 +610,8 @@ public enum FFmpegError: LocalizedError { return "FFmpeg plugin was not properly initialized" case .reencodingFailed(let message): return "Video re-encoding failed: \(message)" + case .transcodeFailed(let message): + return "Media transcode failed: \(message)" case .invalidPath(let path): return "Invalid file path: \(path)" case .invalidArgument(let message): diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift index 74e0b9d..e07632c 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift @@ -13,6 +13,7 @@ public class CapacitorFFmpegPlugin: CAPPlugin, CAPBridgedPlugin { CAPPluginMethod(name: "getCapabilities", returnType: CAPPluginReturnPromise), CAPPluginMethod(name: "reencodeVideo", returnType: CAPPluginReturnPromise), CAPPluginMethod(name: "convertImage", returnType: CAPPluginReturnPromise), + CAPPluginMethod(name: "convertAudio", returnType: CAPPluginReturnPromise), CAPPluginMethod(name: "getPluginVersion", returnType: CAPPluginReturnPromise) ] @@ -108,6 +109,33 @@ public class CapacitorFFmpegPlugin: CAPPlugin, CAPBridgedPlugin { } } + @objc func convertAudio(_ call: CAPPluginCall) { + guard let inputPath = call.getString("inputPath") else { + call.reject("Input path is required", "INVALID_ARGUMENT") + return + } + guard let outputPath = call.getString("outputPath") else { + call.reject("Output path is required", "INVALID_ARGUMENT") + return + } + guard let format = call.getString("format") else { + call.reject("Output format is required", "INVALID_ARGUMENT") + return + } + + do { + let result = try implementation.convertAudio( + inputPath: inputPath, + outputPath: outputPath, + format: format + ) + + call.resolve(result.asDictionary) + } catch { + reject(call, with: error) + } + } + @objc func getPluginVersion(_ call: CAPPluginCall) { call.resolve(["version": CapacitorFFmpegPluginVersion.value]) } diff --git a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift index 53769f6..39f3669 100644 --- a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift +++ b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift @@ -1,3 +1,4 @@ +import AVFoundation import CapacitorFFmpegNativeCore import Darwin import UIKit @@ -39,6 +40,42 @@ private struct FailingFFmpegNativeBindings: FFmpegNativeBinding { } } +private final class MockAudioExportSession: AudioExportSessioning { + var outputURL: URL? + var outputFileType: AVFileType? + var status: AVAssetExportSession.Status + var error: Error? + + private let onExport: (MockAudioExportSession) -> Void + + init( + status: AVAssetExportSession.Status = .completed, + error: Error? = nil, + onExport: @escaping (MockAudioExportSession) -> Void = { _ in } + ) { + self.status = status + self.error = error + self.onExport = onExport + } + + func exportAsynchronously(completionHandler handler: @escaping @Sendable () -> Void) { + onExport(self) + handler() + } +} + +private func writeToneWav(to url: URL) throws { + let toneFormat = try XCTUnwrap(AVAudioFormat(standardFormatWithSampleRate: 44_100, channels: 1)) + let toneFile = try AVAudioFile(forWriting: url, settings: toneFormat.settings) + let toneBuffer = try XCTUnwrap(AVAudioPCMBuffer(pcmFormat: toneFormat, frameCapacity: 44_100)) + toneBuffer.frameLength = 44_100 + let samples = try XCTUnwrap(toneBuffer.floatChannelData?[0]) + for index in 0..; + /** + * Convert audio into another container or codec. + * + * iOS currently supports `m4a`. + * Android and web currently reject with `UNIMPLEMENTED`. + */ + convertAudio(options: ConvertAudioOptions): Promise; + /** * Listen for media job progress. */ diff --git a/src/web.ts b/src/web.ts index dbf2f5d..0968086 100644 --- a/src/web.ts +++ b/src/web.ts @@ -2,10 +2,12 @@ import { WebPlugin } from '@capacitor/core'; import type { CapacitorFFmpegPlugin, - FFmpegAcceptedJob, - FFmpegCapabilitiesResult, + ConvertAudioOptions, + ConvertAudioResult, ConvertImageOptions, ConvertImageResult, + FFmpegAcceptedJob, + FFmpegCapabilitiesResult, PluginVersionResult, ReencodeVideoOptions, } from './definitions'; @@ -26,6 +28,10 @@ export class CapacitorFFmpegWeb extends WebPlugin implements CapacitorFFmpegPlug status: 'unimplemented', reason: 'Image conversion is currently only available on iOS and Android.', }, + convertAudio: { + status: 'unimplemented', + reason: 'Audio conversion is currently only available on iOS.', + }, progressEvents: { status: 'unavailable', reason: 'No media jobs are available on web today.', @@ -64,6 +70,11 @@ export class CapacitorFFmpegWeb extends WebPlugin implements CapacitorFFmpegPlug throw this.unimplemented('convertImage is currently only available on iOS and Android.'); } + async convertAudio(options: ConvertAudioOptions): Promise { + void options; + throw this.unimplemented('convertAudio is currently only available on iOS.'); + } + async getPluginVersion(): Promise { return { version: PLUGIN_VERSION }; } diff --git a/test/TEST_INVENTORY.md b/test/TEST_INVENTORY.md index 41c682b..77d3e68 100644 --- a/test/TEST_INVENTORY.md +++ b/test/TEST_INVENTORY.md @@ -1,6 +1,6 @@ # Test Inventory -Last updated: 2026-04-04 +Last updated: 2026-04-13 This file maps the public plugin contract to the tests that exist now and the next regression coverage we still need. @@ -13,6 +13,8 @@ This file maps the public plugin contract to the tests that exist now and the ne | `reencodeVideo` unsupported behavior | Plugin-only contract | Web contract test, Android unit test helper | None for current scope | | `convertImage` unsupported behavior | Plugin-only contract | Web contract test | None for current scope | | `convertImage` native execution | Still-image transcode | Android unit test helper, iOS native test for fixture generation and output write, Maestro Android+iOS flows | Add wrapper-level test with `CAPPluginCall` and output file metadata validation | +| `convertAudio` unsupported behavior | Plugin-only contract | Web contract test, Android unit test helper | None for current scope | +| `convertAudio` native execution | Audio transcode | iOS native tests for format validation, successful export wrapper behavior, and failed export cleanup | Add end-to-end regression against a real exported file when simulator export is stable | | `reencodeVideo` acceptance result | Plugin job contract | iOS helper tests for queued job payload | iOS plugin wrapper test | | `progress` event payload | Plugin job/progress contract | iOS helper tests | End-to-end event assertion during a real encode | | Example app runtime harness | App-level contract smoke | Vitest example-app checks plus Maestro iOS/Android flows for runtime checks, image conversion, and iOS video flow | Add native device smoke flow: pick arbitrary gallery media -> transcode -> verify metadata | @@ -43,5 +45,5 @@ We are not importing upstream FATE directly. We are using it as a taxonomy: ## Immediate next media tests 1. Generate a tiny MP4 input fixture for the current iOS re-encode flow. -2. Add an iOS-native regression that checks output dimensions and duration tolerance. +2. Add a real exported-audio regression once simulator-side `AVAssetExportSession` output is stable in CI. 3. Add a broken-input fixture and assert structured failure reporting. diff --git a/test/web.test.ts b/test/web.test.ts index 5a58631..a23ed0d 100644 --- a/test/web.test.ts +++ b/test/web.test.ts @@ -20,6 +20,7 @@ describe('CapacitorFFmpegWeb', () => { getCapabilities: { status: 'available' }, reencodeVideo: { status: 'unimplemented' }, convertImage: { status: 'unimplemented' }, + convertAudio: { status: 'unimplemented' }, }, }); }); @@ -49,5 +50,16 @@ describe('CapacitorFFmpegWeb', () => { code: ExceptionCode.Unimplemented, message: 'convertImage is currently only available on iOS and Android.', }); + + await expect( + plugin.convertAudio({ + inputPath: 'file:///input.wav', + outputPath: 'file:///output.m4a', + format: 'm4a', + }), + ).rejects.toMatchObject({ + code: ExceptionCode.Unimplemented, + message: 'convertAudio is currently only available on iOS.', + }); }); }); From 57bb738bf19044cb2a727ffdcd1582dbf49c705e Mon Sep 17 00:00:00 2001 From: Martin Donadieu Date: Mon, 13 Apr 2026 17:54:16 +0200 Subject: [PATCH 2/6] Keep convertAudio optional in v8 typings --- README.md | 51 ++++++++++++++++++++++++++++++++------------ src/definitions.ts | 4 ++-- src/pluginVersion.ts | 2 +- 3 files changed, 40 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index d2b9990..9286fbb 100644 --- a/README.md +++ b/README.md @@ -67,14 +67,14 @@ bunx cap sync -- [`getCapabilities()`](#getcapabilities) -- [`reencodeVideo(...)`](#reencodevideo) -- [`convertImage(...)`](#convertimage) -- [`convertAudio(...)`](#convertaudio) -- [`addListener('progress', ...)`](#addlistenerprogress-) -- [`getPluginVersion()`](#getpluginversion) -- [Interfaces](#interfaces) -- [Type Aliases](#type-aliases) +* [`getCapabilities()`](#getcapabilities) +* [`reencodeVideo(...)`](#reencodevideo) +* [`convertImage(...)`](#convertimage) +* [`convertAudio(...)`](#convertaudio) +* [`addListener('progress', ...)`](#addlistenerprogress-) +* [`getPluginVersion()`](#getpluginversion) +* [Interfaces](#interfaces) +* [Type Aliases](#type-aliases) @@ -91,7 +91,8 @@ Return the machine-readable capability matrix for the current platform. **Returns:** Promise<FFmpegCapabilitiesResult> ---- +-------------------- + ### reencodeVideo(...) @@ -112,7 +113,8 @@ Android and web currently reject with `UNIMPLEMENTED`. **Returns:** Promise<FFmpegAcceptedJob> ---- +-------------------- + ### convertImage(...) @@ -132,7 +134,8 @@ Web currently rejects with `UNIMPLEMENTED`. **Returns:** Promise<ConvertImageResult> ---- +-------------------- + ### convertAudio(...) @@ -151,7 +154,8 @@ Android and web currently reject with `UNIMPLEMENTED`. **Returns:** Promise<ConvertAudioResult> ---- +-------------------- + ### addListener('progress', ...) @@ -168,7 +172,8 @@ Listen for media job progress. **Returns:** Promise<PluginListenerHandle> ---- +-------------------- + ### getPluginVersion() @@ -180,10 +185,12 @@ Get the plugin package version reported by the current platform implementation. **Returns:** Promise<PluginVersionResult> ---- +-------------------- + ### Interfaces + #### FFmpegCapabilitiesResult | Prop | Type | @@ -191,6 +198,7 @@ Get the plugin package version reported by the current platform implementation. | **`platform`** | string | | **`features`** | FFmpegCapabilitiesFeatures | + #### FFmpegCapabilitiesFeatures | Prop | Type | @@ -207,6 +215,7 @@ Get the plugin package version reported by the current platform implementation. | **`remux`** | FFmpegCapability | | **`trim`** | FFmpegCapability | + #### FFmpegCapability | Prop | Type | @@ -214,6 +223,7 @@ Get the plugin package version reported by the current platform implementation. | **`status`** | FFmpegCapabilityStatus | | **`reason`** | string | + #### FFmpegAcceptedJob | Prop | Type | @@ -221,6 +231,7 @@ Get the plugin package version reported by the current platform implementation. | **`jobId`** | string | | **`status`** | 'queued' | + #### ReencodeVideoOptions | Prop | Type | @@ -231,6 +242,7 @@ Get the plugin package version reported by the current platform implementation. | **`height`** | number | | **`bitrate`** | number | + #### ConvertImageResult | Prop | Type | @@ -238,6 +250,7 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | **`format`** | ImageOutputFormat | + #### ConvertImageOptions | Prop | Type | Description | @@ -247,6 +260,7 @@ Get the plugin package version reported by the current platform implementation. | **`format`** | ImageOutputFormat | | | **`quality`** | number | Compression quality in the inclusive range `0.0..1.0`. Native platforms reject values outside that range. | + #### ConvertAudioResult | Prop | Type | @@ -254,6 +268,7 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | **`format`** | AudioOutputFormat | + #### ConvertAudioOptions | Prop | Type | @@ -262,12 +277,14 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | **`format`** | AudioOutputFormat | + #### PluginListenerHandle | Prop | Type | | ------------ | ----------------------------------------- | | **`remove`** | () => Promise<void> | + #### FFmpegProgressEvent | Prop | Type | Description | @@ -279,26 +296,32 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | | **`fileId`** | string | Legacy alias kept for compatibility while callers migrate to `jobId`. | + #### PluginVersionResult | Prop | Type | | ------------- | ------------------- | | **`version`** | string | + ### Type Aliases + #### FFmpegCapabilityStatus 'available' | 'experimental' | 'unimplemented' | 'unavailable' + #### ImageOutputFormat 'webp' | 'jpeg' | 'png' + #### AudioOutputFormat 'm4a' + #### FFmpegProgressState 'running' | 'completed' | 'failed' diff --git a/src/definitions.ts b/src/definitions.ts index a75a2dc..9a6f0ac 100644 --- a/src/definitions.ts +++ b/src/definitions.ts @@ -23,7 +23,7 @@ export interface FFmpegCapabilitiesFeatures { getCapabilities: FFmpegCapability; reencodeVideo: FFmpegCapability; convertImage: FFmpegCapability; - convertAudio: FFmpegCapability; + convertAudio?: FFmpegCapability; progressEvents: FFmpegCapability; probeMedia: FFmpegCapability; generateThumbnail: FFmpegCapability; @@ -128,7 +128,7 @@ export interface CapacitorFFmpegPlugin extends Plugin { * iOS currently supports `m4a`. * Android and web currently reject with `UNIMPLEMENTED`. */ - convertAudio(options: ConvertAudioOptions): Promise; + convertAudio?(options: ConvertAudioOptions): Promise; /** * Listen for media job progress. diff --git a/src/pluginVersion.ts b/src/pluginVersion.ts index 6f21a3b..de80c0b 100644 --- a/src/pluginVersion.ts +++ b/src/pluginVersion.ts @@ -1 +1 @@ -export const PLUGIN_VERSION = '0.0.8'; +export const PLUGIN_VERSION = "0.0.8"; From 15bd0c8a1a865f1cb6a3b3972357ebffa323260c Mon Sep 17 00:00:00 2001 From: Martin Donadieu Date: Mon, 13 Apr 2026 17:59:13 +0200 Subject: [PATCH 3/6] Preserve audio output on failed export --- .../CapacitorFFmpeg.swift | 18 +++++--- .../CapacitorFFmpegPluginTests.swift | 42 +++++++++++++++++-- 2 files changed, 51 insertions(+), 9 deletions(-) diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift index 9dfe3bd..0ca4c37 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift @@ -547,15 +547,18 @@ private final class SelfForReencodeVideo { at: outputDirectory, withIntermediateDirectories: true ) - - if fileManager.fileExists(atPath: outputURL.path) { - try fileManager.removeItem(at: outputURL) + let temporaryOutputURL = outputDirectory + .appendingPathComponent(".ffmpeg-audio-\(UUID().uuidString)") + .appendingPathExtension(outputURL.pathExtension.isEmpty ? "m4a" : outputURL.pathExtension) + defer { + try? fileManager.removeItem(at: temporaryOutputURL) } + guard let exportSession = audioExportSessionFactory(asset, AVAssetExportPresetAppleM4A) else { throw FFmpegError.transcodeFailed("Could not create the audio export session.") } - exportSession.outputURL = outputURL + exportSession.outputURL = temporaryOutputURL exportSession.outputFileType = .m4a let semaphore = DispatchSemaphore(value: 0) @@ -565,12 +568,17 @@ private final class SelfForReencodeVideo { semaphore.wait() guard exportSession.status == .completed else { - try? fileManager.removeItem(at: outputURL) throw FFmpegError.transcodeFailed( exportSession.error?.localizedDescription ?? "Could not export the output audio." ) } + if fileManager.fileExists(atPath: outputURL.path) { + _ = try fileManager.replaceItemAt(outputURL, withItemAt: temporaryOutputURL) + } else { + try fileManager.moveItem(at: temporaryOutputURL, to: outputURL) + } + return FFmpegConvertedAudio( outputPath: outputURL.absoluteString, format: normalizedFormat diff --git a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift index 39f3669..71de220 100644 --- a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift +++ b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift @@ -239,9 +239,10 @@ final class CapacitorFFmpegPluginTests: XCTestCase { try Data("stale".utf8).write(to: outputURL) let exportSession = MockAudioExportSession { session in - XCTAssertEqual(session.outputURL, outputURL) + XCTAssertNotEqual(session.outputURL, outputURL) + XCTAssertEqual(session.outputURL?.deletingLastPathComponent(), outputURL.deletingLastPathComponent()) XCTAssertEqual(session.outputFileType, .m4a) - try? Data("converted-audio".utf8).write(to: outputURL) + try? Data("converted-audio".utf8).write(to: XCTUnwrap(session.outputURL)) session.status = .completed } @@ -269,6 +270,7 @@ final class CapacitorFFmpegPluginTests: XCTestCase { let inputURL = baseURL.appendingPathComponent("input.wav") let outputURL = baseURL.appendingPathComponent("output.m4a") try writeToneWav(to: inputURL) + try Data("keep-existing-output".utf8).write(to: outputURL) let exportError = NSError( domain: "CapacitorFFmpegPluginTests", @@ -276,8 +278,8 @@ final class CapacitorFFmpegPluginTests: XCTestCase { userInfo: [NSLocalizedDescriptionKey: "simulated export failure"] ) let exportSession = MockAudioExportSession(status: .failed, error: exportError) { session in - XCTAssertEqual(session.outputURL, outputURL) - try? Data("partial-output".utf8).write(to: outputURL) + XCTAssertNotEqual(session.outputURL, outputURL) + try? Data("partial-output".utf8).write(to: XCTUnwrap(session.outputURL)) } XCTAssertThrowsError( @@ -291,6 +293,38 @@ final class CapacitorFFmpegPluginTests: XCTestCase { XCTAssertEqual(error.localizedDescription, "Media transcode failed: simulated export failure") } + XCTAssertEqual(try String(contentsOf: outputURL), "keep-existing-output") + } + + func testConvertAudioRemovesPartialTemporaryOutputWhenNoDestinationExists() throws { + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } + + let inputURL = baseURL.appendingPathComponent("input.wav") + let outputURL = baseURL.appendingPathComponent("output.m4a") + try writeToneWav(to: inputURL) + + let exportError = NSError( + domain: "CapacitorFFmpegPluginTests", + code: 2, + userInfo: [NSLocalizedDescriptionKey: "simulated export failure"] + ) + let exportSession = MockAudioExportSession(status: .failed, error: exportError) { session in + try? Data("partial-output".utf8).write(to: XCTUnwrap(session.outputURL)) + } + + XCTAssertThrowsError( + try CapacitorFFmpeg(audioExportSessionFactory: { _, _ in exportSession }).convertAudio( + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) + ) { error in + XCTAssertEqual((error as? FFmpegError)?.code, "TRANSCODE_FAILED") + } + XCTAssertFalse(fileManager.fileExists(atPath: outputURL.path)) } } From dd73e3d060684028497008f03f2c5d32740a2494 Mon Sep 17 00:00:00 2001 From: Martin Donadieu Date: Mon, 13 Apr 2026 18:07:39 +0200 Subject: [PATCH 4/6] Validate audio output path and async export --- .../CapacitorFFmpeg.swift | 46 ++++---- .../CapacitorFFmpegPlugin.swift | 19 +++- .../CapacitorFFmpegPluginTests.swift | 103 ++++++++++++++---- 3 files changed, 122 insertions(+), 46 deletions(-) diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift index 0ca4c37..6bccedb 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift @@ -126,6 +126,8 @@ struct FFmpegConvertedAudio { } } +typealias AudioConversionCompletion = (Result) -> Void + struct FFmpegCapabilityPayload { let status: String let reason: String? @@ -521,8 +523,9 @@ private final class SelfForReencodeVideo { func convertAudio( inputPath: String, outputPath: String, - format: String - ) throws -> FFmpegConvertedAudio { + format: String, + completion: @escaping AudioConversionCompletion + ) throws { let inputURL = try resolveFileURL(from: inputPath).standardizedFileURL let outputURL = try resolveFileURL(from: outputPath).standardizedFileURL let normalizedFormat = format.lowercased() @@ -534,6 +537,9 @@ private final class SelfForReencodeVideo { guard normalizedFormat == "m4a" else { throw FFmpegError.invalidArgument("Unsupported audio format: \(format)") } + guard outputURL.pathExtension.lowercased() == normalizedFormat else { + throw FFmpegError.invalidArgument("Output path extension must be .\(normalizedFormat).") + } let asset = AVURLAsset(url: inputURL) guard let audioTrack = asset.tracks(withMediaType: .audio).first else { @@ -561,28 +567,28 @@ private final class SelfForReencodeVideo { exportSession.outputURL = temporaryOutputURL exportSession.outputFileType = .m4a - let semaphore = DispatchSemaphore(value: 0) exportSession.exportAsynchronously { - semaphore.signal() - } - semaphore.wait() + do { + guard exportSession.status == .completed else { + throw FFmpegError.transcodeFailed( + exportSession.error?.localizedDescription ?? "Could not export the output audio." + ) + } - guard exportSession.status == .completed else { - throw FFmpegError.transcodeFailed( - exportSession.error?.localizedDescription ?? "Could not export the output audio." - ) - } + if fileManager.fileExists(atPath: outputURL.path) { + _ = try fileManager.replaceItemAt(outputURL, withItemAt: temporaryOutputURL) + } else { + try fileManager.moveItem(at: temporaryOutputURL, to: outputURL) + } - if fileManager.fileExists(atPath: outputURL.path) { - _ = try fileManager.replaceItemAt(outputURL, withItemAt: temporaryOutputURL) - } else { - try fileManager.moveItem(at: temporaryOutputURL, to: outputURL) + completion(.success(FFmpegConvertedAudio( + outputPath: outputURL.absoluteString, + format: normalizedFormat + ))) + } catch { + completion(.failure(error)) + } } - - return FFmpegConvertedAudio( - outputPath: outputURL.absoluteString, - format: normalizedFormat - ) } func getCapabilities() -> FFmpegCapabilitiesPayload { diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift index e07632c..08e38fc 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift @@ -124,13 +124,24 @@ public class CapacitorFFmpegPlugin: CAPPlugin, CAPBridgedPlugin { } do { - let result = try implementation.convertAudio( + try implementation.convertAudio( inputPath: inputPath, outputPath: outputPath, format: format - ) - - call.resolve(result.asDictionary) + ) { [weak self] result in + DispatchQueue.main.async { + switch result { + case .success(let convertedAudio): + call.resolve(convertedAudio.asDictionary) + case .failure(let error): + guard let self else { + call.reject(error.localizedDescription, "TRANSCODE_FAILED") + return + } + self.reject(call, with: error) + } + } + } } catch { reject(call, with: error) } diff --git a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift index 71de220..7b5e677 100644 --- a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift +++ b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift @@ -77,6 +77,29 @@ private func writeToneWav(to url: URL) throws { } final class CapacitorFFmpegPluginTests: XCTestCase { + private func waitForAudioConversion( + using plugin: CapacitorFFmpeg, + inputPath: String, + outputPath: String, + format: String, + timeout: TimeInterval = 1.0 + ) throws -> Result { + let expectation = expectation(description: "convertAudio") + var conversionResult: Result? + + try plugin.convertAudio( + inputPath: inputPath, + outputPath: outputPath, + format: format + ) { result in + conversionResult = result + expectation.fulfill() + } + + wait(for: [expectation], timeout: timeout) + return try XCTUnwrap(conversionResult) + } + func testCapabilitiesPayloadDescribesTheCurrentIosScope() { let payload = CapacitorFFmpeg().getCapabilities().asDictionary let features = payload["features"] as? [String: [String: Any]] @@ -220,13 +243,36 @@ final class CapacitorFFmpegPluginTests: XCTestCase { inputPath: inputURL.path, outputPath: outputURL.path, format: "wav" - ) + ) { _ in } ) { error in XCTAssertEqual((error as? FFmpegError)?.code, "INVALID_ARGUMENT") XCTAssertEqual(error.localizedDescription, "Unsupported audio format: wav") } } + func testConvertAudioRejectsMismatchedOutputExtension() throws { + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } + + let inputURL = baseURL.appendingPathComponent("input.wav") + let outputURL = baseURL.appendingPathComponent("output.wav") + + try writeToneWav(to: inputURL) + + XCTAssertThrowsError( + try CapacitorFFmpeg().convertAudio( + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) { _ in } + ) { error in + XCTAssertEqual((error as? FFmpegError)?.code, "INVALID_ARGUMENT") + XCTAssertEqual(error.localizedDescription, "Output path extension must be .m4a.") + } + } + func testConvertAudioWritesM4AOutputWhenExportSucceeds() throws { let fileManager = FileManager.default let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) @@ -246,18 +292,23 @@ final class CapacitorFFmpegPluginTests: XCTestCase { session.status = .completed } - let result = try CapacitorFFmpeg(audioExportSessionFactory: { asset, presetName in + let result = try waitForAudioConversion(using: CapacitorFFmpeg(audioExportSessionFactory: { asset, presetName in XCTAssertEqual(asset.tracks(withMediaType: .audio).count, 1) XCTAssertEqual(presetName, AVAssetExportPresetAppleM4A) return exportSession - }).convertAudio( - inputPath: inputURL.path, - outputPath: outputURL.path, - format: "m4a" + }), + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" ) - XCTAssertEqual(result.format, "m4a") - XCTAssertEqual(result.outputPath, outputURL.absoluteString) + switch result { + case .success(let convertedAudio): + XCTAssertEqual(convertedAudio.format, "m4a") + XCTAssertEqual(convertedAudio.outputPath, outputURL.absoluteString) + case .failure(let error): + XCTFail("Expected successful audio conversion, got \(error)") + } XCTAssertEqual(try String(contentsOf: outputURL), "converted-audio") } @@ -282,13 +333,17 @@ final class CapacitorFFmpegPluginTests: XCTestCase { try? Data("partial-output".utf8).write(to: XCTUnwrap(session.outputURL)) } - XCTAssertThrowsError( - try CapacitorFFmpeg(audioExportSessionFactory: { _, _ in exportSession }).convertAudio( - inputPath: inputURL.path, - outputPath: outputURL.path, - format: "m4a" - ) - ) { error in + let result = try waitForAudioConversion( + using: CapacitorFFmpeg(audioExportSessionFactory: { _, _ in exportSession }), + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) + + switch result { + case .success: + XCTFail("Expected audio conversion failure") + case .failure(let error): XCTAssertEqual((error as? FFmpegError)?.code, "TRANSCODE_FAILED") XCTAssertEqual(error.localizedDescription, "Media transcode failed: simulated export failure") } @@ -315,13 +370,17 @@ final class CapacitorFFmpegPluginTests: XCTestCase { try? Data("partial-output".utf8).write(to: XCTUnwrap(session.outputURL)) } - XCTAssertThrowsError( - try CapacitorFFmpeg(audioExportSessionFactory: { _, _ in exportSession }).convertAudio( - inputPath: inputURL.path, - outputPath: outputURL.path, - format: "m4a" - ) - ) { error in + let result = try waitForAudioConversion( + using: CapacitorFFmpeg(audioExportSessionFactory: { _, _ in exportSession }), + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) + + switch result { + case .success: + XCTFail("Expected audio conversion failure") + case .failure(let error): XCTAssertEqual((error as? FFmpegError)?.code, "TRANSCODE_FAILED") } From 7b2698fe2df589a747eba142a8773de70c0bccd4 Mon Sep 17 00:00:00 2001 From: Martin Donadieu Date: Mon, 13 Apr 2026 18:13:25 +0200 Subject: [PATCH 5/6] Clean up audio temp files after export --- .../CapacitorFFmpegPlugin/CapacitorFFmpeg.swift | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift index 6bccedb..c68707f 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift @@ -542,10 +542,9 @@ private final class SelfForReencodeVideo { } let asset = AVURLAsset(url: inputURL) - guard let audioTrack = asset.tracks(withMediaType: .audio).first else { + guard !asset.tracks(withMediaType: .audio).isEmpty else { throw FFmpegError.invalidArgument("The input media does not contain an audio track.") } - _ = audioTrack let fileManager = FileManager.default let outputDirectory = outputURL.deletingLastPathComponent() @@ -556,9 +555,6 @@ private final class SelfForReencodeVideo { let temporaryOutputURL = outputDirectory .appendingPathComponent(".ffmpeg-audio-\(UUID().uuidString)") .appendingPathExtension(outputURL.pathExtension.isEmpty ? "m4a" : outputURL.pathExtension) - defer { - try? fileManager.removeItem(at: temporaryOutputURL) - } guard let exportSession = audioExportSessionFactory(asset, AVAssetExportPresetAppleM4A) else { throw FFmpegError.transcodeFailed("Could not create the audio export session.") @@ -568,6 +564,10 @@ private final class SelfForReencodeVideo { exportSession.outputFileType = .m4a exportSession.exportAsynchronously { + defer { + try? fileManager.removeItem(at: temporaryOutputURL) + } + do { guard exportSession.status == .completed else { throw FFmpegError.transcodeFailed( From c0b5e5c831291219ad6403ec02737c0745ea9819 Mon Sep 17 00:00:00 2001 From: Martin Donadieu Date: Mon, 13 Apr 2026 19:40:36 +0200 Subject: [PATCH 6/6] Fix lint formatting after merging main --- README.md | 51 ++++++++++++-------------------------------- src/pluginVersion.ts | 2 +- 2 files changed, 15 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index 9286fbb..d2b9990 100644 --- a/README.md +++ b/README.md @@ -67,14 +67,14 @@ bunx cap sync -* [`getCapabilities()`](#getcapabilities) -* [`reencodeVideo(...)`](#reencodevideo) -* [`convertImage(...)`](#convertimage) -* [`convertAudio(...)`](#convertaudio) -* [`addListener('progress', ...)`](#addlistenerprogress-) -* [`getPluginVersion()`](#getpluginversion) -* [Interfaces](#interfaces) -* [Type Aliases](#type-aliases) +- [`getCapabilities()`](#getcapabilities) +- [`reencodeVideo(...)`](#reencodevideo) +- [`convertImage(...)`](#convertimage) +- [`convertAudio(...)`](#convertaudio) +- [`addListener('progress', ...)`](#addlistenerprogress-) +- [`getPluginVersion()`](#getpluginversion) +- [Interfaces](#interfaces) +- [Type Aliases](#type-aliases) @@ -91,8 +91,7 @@ Return the machine-readable capability matrix for the current platform. **Returns:** Promise<FFmpegCapabilitiesResult> --------------------- - +--- ### reencodeVideo(...) @@ -113,8 +112,7 @@ Android and web currently reject with `UNIMPLEMENTED`. **Returns:** Promise<FFmpegAcceptedJob> --------------------- - +--- ### convertImage(...) @@ -134,8 +132,7 @@ Web currently rejects with `UNIMPLEMENTED`. **Returns:** Promise<ConvertImageResult> --------------------- - +--- ### convertAudio(...) @@ -154,8 +151,7 @@ Android and web currently reject with `UNIMPLEMENTED`. **Returns:** Promise<ConvertAudioResult> --------------------- - +--- ### addListener('progress', ...) @@ -172,8 +168,7 @@ Listen for media job progress. **Returns:** Promise<PluginListenerHandle> --------------------- - +--- ### getPluginVersion() @@ -185,12 +180,10 @@ Get the plugin package version reported by the current platform implementation. **Returns:** Promise<PluginVersionResult> --------------------- - +--- ### Interfaces - #### FFmpegCapabilitiesResult | Prop | Type | @@ -198,7 +191,6 @@ Get the plugin package version reported by the current platform implementation. | **`platform`** | string | | **`features`** | FFmpegCapabilitiesFeatures | - #### FFmpegCapabilitiesFeatures | Prop | Type | @@ -215,7 +207,6 @@ Get the plugin package version reported by the current platform implementation. | **`remux`** | FFmpegCapability | | **`trim`** | FFmpegCapability | - #### FFmpegCapability | Prop | Type | @@ -223,7 +214,6 @@ Get the plugin package version reported by the current platform implementation. | **`status`** | FFmpegCapabilityStatus | | **`reason`** | string | - #### FFmpegAcceptedJob | Prop | Type | @@ -231,7 +221,6 @@ Get the plugin package version reported by the current platform implementation. | **`jobId`** | string | | **`status`** | 'queued' | - #### ReencodeVideoOptions | Prop | Type | @@ -242,7 +231,6 @@ Get the plugin package version reported by the current platform implementation. | **`height`** | number | | **`bitrate`** | number | - #### ConvertImageResult | Prop | Type | @@ -250,7 +238,6 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | **`format`** | ImageOutputFormat | - #### ConvertImageOptions | Prop | Type | Description | @@ -260,7 +247,6 @@ Get the plugin package version reported by the current platform implementation. | **`format`** | ImageOutputFormat | | | **`quality`** | number | Compression quality in the inclusive range `0.0..1.0`. Native platforms reject values outside that range. | - #### ConvertAudioResult | Prop | Type | @@ -268,7 +254,6 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | **`format`** | AudioOutputFormat | - #### ConvertAudioOptions | Prop | Type | @@ -277,14 +262,12 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | **`format`** | AudioOutputFormat | - #### PluginListenerHandle | Prop | Type | | ------------ | ----------------------------------------- | | **`remove`** | () => Promise<void> | - #### FFmpegProgressEvent | Prop | Type | Description | @@ -296,32 +279,26 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | | **`fileId`** | string | Legacy alias kept for compatibility while callers migrate to `jobId`. | - #### PluginVersionResult | Prop | Type | | ------------- | ------------------- | | **`version`** | string | - ### Type Aliases - #### FFmpegCapabilityStatus 'available' | 'experimental' | 'unimplemented' | 'unavailable' - #### ImageOutputFormat 'webp' | 'jpeg' | 'png' - #### AudioOutputFormat 'm4a' - #### FFmpegProgressState 'running' | 'completed' | 'failed' diff --git a/src/pluginVersion.ts b/src/pluginVersion.ts index 7149838..0273c9d 100644 --- a/src/pluginVersion.ts +++ b/src/pluginVersion.ts @@ -1 +1 @@ -export const PLUGIN_VERSION = "0.0.9"; +export const PLUGIN_VERSION = '0.0.9';