diff --git a/FEATURE_MATRIX.md b/FEATURE_MATRIX.md index caf89be..a77299e 100644 --- a/FEATURE_MATRIX.md +++ b/FEATURE_MATRIX.md @@ -1,6 +1,6 @@ # Feature Matrix -Last updated: 2026-04-04 +Last updated: 2026-04-13 This file tracks the public plugin contract and the current platform support level for each capability. @@ -12,19 +12,20 @@ This file tracks the public plugin contract and the current platform support lev | `getPluginVersion()` | Resolves `{ version: string }` on iOS, Android, and web | Web returns `"web"` as a platform marker. | | `reencodeVideo(options)` | Resolves when the job is accepted by iOS and returns `{ jobId, status: "queued" }` when available | Android and web reject with `UNIMPLEMENTED`. | | `convertImage(options)` | Resolves `{ outputPath, format }` on iOS and Android | Web rejects with `UNIMPLEMENTED`. | +| `convertAudio(options)` | Resolves `{ outputPath, format }` on iOS for `m4a` output | Android and web reject with `UNIMPLEMENTED`. | | `progress` listener | Emits `{ jobId, progress, state, message?, outputPath? }` | `fileId` is kept as a compatibility alias for `jobId`. | ## Failure contract Use these codes as the shared error vocabulary for JS consumers: -| Code | Meaning | Current producers | -| ------------------------ | ------------------------------------------------------------ | ----------------------------------------------- | -| `UNIMPLEMENTED` | The API is not implemented on the current platform | Android/web `reencodeVideo`, web `convertImage` | -| `UNAVAILABLE` | The API exists but cannot be used in the current environment | Reserved for future capabilities | -| `INVALID_ARGUMENT` | Caller input failed local validation | iOS `reencodeVideo` validation | -| `PLUGIN_NOT_INITIALIZED` | Native core could not be initialized | iOS native wrapper | -| `TRANSCODE_FAILED` | The media pipeline failed after acceptance or during setup | iOS native wrapper and Rust bridge | +| Code | Meaning | Current producers | +| ------------------------ | ------------------------------------------------------------ | --------------------------------------------------------------------------- | +| `UNIMPLEMENTED` | The API is not implemented on the current platform | Android/web `reencodeVideo`, Android/web `convertAudio`, web `convertImage` | +| `UNAVAILABLE` | The API exists but cannot be used in the current environment | Reserved for future capabilities | +| `INVALID_ARGUMENT` | Caller input failed local validation | iOS media wrapper validation | +| `PLUGIN_NOT_INITIALIZED` | Native core could not be initialized | iOS native wrapper | +| `TRANSCODE_FAILED` | The media pipeline failed after acceptance or during setup | iOS native wrappers and Rust bridge | ## Platform support @@ -35,6 +36,7 @@ Use these codes as the shared error vocabulary for JS consumers: | `reencodeVideo` acceptance contract | ✅ | ❌ `UNIMPLEMENTED` | ❌ `UNIMPLEMENTED` | Covered for iOS helpers and web/Android unsupported behavior | | `reencodeVideo` media execution | ⚠️ Experimental | ❌ | ❌ | Wrapper contract covered; media regression fixtures still pending | | `convertImage` | ✅ `jpeg`/`png` | ✅ `webp`/`jpeg`/`png` | ❌ `UNIMPLEMENTED` | Covered by iOS native tests, Android unit tests, and Maestro flows | +| `convertAudio` | ✅ `m4a` | ❌ `UNIMPLEMENTED` | ❌ `UNIMPLEMENTED` | Covered by iOS native tests plus web and Android unsupported tests | | `progress` listener contract | ✅ | ❌ | ❌ | Covered by iOS helper tests | | `probeMedia` | ❌ Planned | ❌ Planned | ❌ Planned | Not started | | `generateThumbnail` | ❌ Planned | ❌ Planned | ❌ Planned | Not started | @@ -46,6 +48,7 @@ Use these codes as the shared error vocabulary for JS consumers: - iOS is still the reference platform for media work. - `convertImage()` does not depend on the Rust FFmpeg core and remains available even when `reencodeVideo()` is unavailable in SwiftPM builds. +- `convertAudio()` currently relies on `AVAssetExportSession` on iOS and is limited to `m4a` output. - Android currently implements image conversion without the broader FFmpeg job pipeline. - `getCapabilities()` is the machine-readable source of truth for app-side feature gating. - Android and web should reject unsupported media APIs explicitly instead of failing implicitly. diff --git a/README.md b/README.md index 5c6f8b5..d2b9990 100644 --- a/README.md +++ b/README.md @@ -53,26 +53,28 @@ bunx cap sync | `getPluginVersion` | ✅ | ✅ | ✅ | Returns a `{ version }` payload on every platform; use `getCapabilities().platform` for platform detection. | | `reencodeVideo` | ⚠️ Experimental | ❌ | ❌ | iOS accepts a queued job and reports lifecycle via `progress`; Android and web reject with `UNIMPLEMENTED`. | | `convertImage` | ✅ | ✅ | ❌ | iOS converts still images to `jpeg` or `png`; Android converts to `webp`, `jpeg`, or `png`; web rejects. | +| `convertAudio` | ✅ | ❌ | ❌ | iOS converts audio to `m4a`; Android and web reject with `UNIMPLEMENTED`. | ## Platform status -| Platform | Status | Notes | -| -------- | ----------------------------- | ---------------------------------------------------------------------------------------- | -| iOS | Early implementation | Current reference platform for media work. | -| Android | Partial native implementation | `convertImage` is native; the broader FFmpeg media engine still needs to be implemented. | -| Web | Stub only | Media operations are intentionally unsupported right now. | +| Platform | Status | Notes | +| -------- | ----------------------------- | ---------------------------------------------------------------------------------------------------------- | +| iOS | Early implementation | Current reference platform for media work. | +| Android | Partial native implementation | `convertImage` is native; the broader FFmpeg media engine and `convertAudio` still need to be implemented. | +| Web | Stub only | Media operations are intentionally unsupported right now. | ## API -* [`getCapabilities()`](#getcapabilities) -* [`reencodeVideo(...)`](#reencodevideo) -* [`convertImage(...)`](#convertimage) -* [`addListener('progress', ...)`](#addlistenerprogress-) -* [`getPluginVersion()`](#getpluginversion) -* [Interfaces](#interfaces) -* [Type Aliases](#type-aliases) +- [`getCapabilities()`](#getcapabilities) +- [`reencodeVideo(...)`](#reencodevideo) +- [`convertImage(...)`](#convertimage) +- [`convertAudio(...)`](#convertaudio) +- [`addListener('progress', ...)`](#addlistenerprogress-) +- [`getPluginVersion()`](#getpluginversion) +- [Interfaces](#interfaces) +- [Type Aliases](#type-aliases) @@ -89,8 +91,7 @@ Return the machine-readable capability matrix for the current platform. **Returns:** Promise<FFmpegCapabilitiesResult> --------------------- - +--- ### reencodeVideo(...) @@ -111,8 +112,7 @@ Android and web currently reject with `UNIMPLEMENTED`. **Returns:** Promise<FFmpegAcceptedJob> --------------------- - +--- ### convertImage(...) @@ -132,8 +132,26 @@ Web currently rejects with `UNIMPLEMENTED`. **Returns:** Promise<ConvertImageResult> --------------------- +--- + +### convertAudio(...) + +```typescript +convertAudio(options: ConvertAudioOptions) => Promise +``` + +Convert audio into another container or codec. +iOS currently supports `m4a`. +Android and web currently reject with `UNIMPLEMENTED`. + +| Param | Type | +| ------------- | ------------------------------------------------------------------- | +| **`options`** | ConvertAudioOptions | + +**Returns:** Promise<ConvertAudioResult> + +--- ### addListener('progress', ...) @@ -150,8 +168,7 @@ Listen for media job progress. **Returns:** Promise<PluginListenerHandle> --------------------- - +--- ### getPluginVersion() @@ -163,12 +180,10 @@ Get the plugin package version reported by the current platform implementation. **Returns:** Promise<PluginVersionResult> --------------------- - +--- ### Interfaces - #### FFmpegCapabilitiesResult | Prop | Type | @@ -176,7 +191,6 @@ Get the plugin package version reported by the current platform implementation. | **`platform`** | string | | **`features`** | FFmpegCapabilitiesFeatures | - #### FFmpegCapabilitiesFeatures | Prop | Type | @@ -185,6 +199,7 @@ Get the plugin package version reported by the current platform implementation. | **`getCapabilities`** | FFmpegCapability | | **`reencodeVideo`** | FFmpegCapability | | **`convertImage`** | FFmpegCapability | +| **`convertAudio`** | FFmpegCapability | | **`progressEvents`** | FFmpegCapability | | **`probeMedia`** | FFmpegCapability | | **`generateThumbnail`** | FFmpegCapability | @@ -192,7 +207,6 @@ Get the plugin package version reported by the current platform implementation. | **`remux`** | FFmpegCapability | | **`trim`** | FFmpegCapability | - #### FFmpegCapability | Prop | Type | @@ -200,7 +214,6 @@ Get the plugin package version reported by the current platform implementation. | **`status`** | FFmpegCapabilityStatus | | **`reason`** | string | - #### FFmpegAcceptedJob | Prop | Type | @@ -208,7 +221,6 @@ Get the plugin package version reported by the current platform implementation. | **`jobId`** | string | | **`status`** | 'queued' | - #### ReencodeVideoOptions | Prop | Type | @@ -219,7 +231,6 @@ Get the plugin package version reported by the current platform implementation. | **`height`** | number | | **`bitrate`** | number | - #### ConvertImageResult | Prop | Type | @@ -227,7 +238,6 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | **`format`** | ImageOutputFormat | - #### ConvertImageOptions | Prop | Type | Description | @@ -237,6 +247,20 @@ Get the plugin package version reported by the current platform implementation. | **`format`** | ImageOutputFormat | | | **`quality`** | number | Compression quality in the inclusive range `0.0..1.0`. Native platforms reject values outside that range. | +#### ConvertAudioResult + +| Prop | Type | +| ---------------- | --------------------------------------------------------------- | +| **`outputPath`** | string | +| **`format`** | AudioOutputFormat | + +#### ConvertAudioOptions + +| Prop | Type | +| ---------------- | --------------------------------------------------------------- | +| **`inputPath`** | string | +| **`outputPath`** | string | +| **`format`** | AudioOutputFormat | #### PluginListenerHandle @@ -244,7 +268,6 @@ Get the plugin package version reported by the current platform implementation. | ------------ | ----------------------------------------- | | **`remove`** | () => Promise<void> | - #### FFmpegProgressEvent | Prop | Type | Description | @@ -256,26 +279,25 @@ Get the plugin package version reported by the current platform implementation. | **`outputPath`** | string | | | **`fileId`** | string | Legacy alias kept for compatibility while callers migrate to `jobId`. | - #### PluginVersionResult | Prop | Type | | ------------- | ------------------- | | **`version`** | string | - ### Type Aliases - #### FFmpegCapabilityStatus 'available' | 'experimental' | 'unimplemented' | 'unavailable' - #### ImageOutputFormat 'webp' | 'jpeg' | 'png' +#### AudioOutputFormat + +'m4a' #### FFmpegProgressState diff --git a/ROADMAP.md b/ROADMAP.md index 9fbc5fb..01ea9c4 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,18 +1,18 @@ # Capacitor FFmpeg Roadmap -Last updated: 2026-04-04 +Last updated: 2026-04-13 Status: active planning ## Why this file exists This plugin is currently much smaller than FFmpeg itself: -- The TypeScript API exposes `getCapabilities()`, `reencodeVideo()`, `convertImage()`, a `progress` event, and `getPluginVersion()`. +- The TypeScript API exposes `getCapabilities()`, `reencodeVideo()`, `convertImage()`, `convertAudio()`, a `progress` event, and `getPluginVersion()`. - The Rust core currently focuses on one video re-encode path: decode video, encode H.264, and copy non-video streams. -- iOS has the only real native media implementation today. -- Android exposes `getPluginVersion()`, `getCapabilities()`, and `convertImage()`. +- iOS has the only real native media implementation today, including still-image conversion and a first audio-conversion path. +- Android exposes `getPluginVersion()`, `getCapabilities()`, `convertImage()`, and an explicit `convertAudio()` contract that still rejects with `UNIMPLEMENTED`. - Web is a stub. -- The test suite now covers basic wrapper contracts, but not media regressions yet. +- The test suite now covers basic wrapper contracts plus iOS image and audio helper flows, but not full media regressions yet. This roadmap is the living source of truth for how we expand the plugin safely, how we test it, and how we keep its scope realistic. @@ -47,15 +47,15 @@ The right goal is a well-documented, well-tested mobile subset of FFmpeg, with a ## Current baseline -| Area | Current state | Main gap | -| --------- | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------- | -| JS API | `getCapabilities`, `reencodeVideo`, `progress`, `getPluginVersion` | No probe, trim, remux, thumbnails, extract-audio, cancel, or cross-platform job execution | -| Rust core | One re-encode flow with progress callback | No unit tests, no fixture suite, no stable result model, no broader operation set | -| iOS | Real implementation exists | Tied to a narrow workflow and limited test coverage | -| Android | Version method only | No media engine, no API parity | -| Web | Stub | No documented support story | -| Tests | Basic web/iOS/Android contract tests | No media fixture coverage, no Rust regressions, and no end-to-end flows | -| Docs | README documents support and roadmap links | No generated API docs yet for planned operations and no capability matrix by phase | +| Area | Current state | Main gap | +| --------- | -------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | +| JS API | `getCapabilities`, `reencodeVideo`, `convertImage`, `convertAudio`, `progress`, `getPluginVersion` | No probe, trim, remux, thumbnails, extract-audio, cancel, or cross-platform media parity | +| Rust core | One re-encode flow with progress callback | No unit tests, no fixture suite, no stable result model, no broader operation set | +| iOS | Real implementation exists for video, image, and limited audio conversion | Still tied to narrow workflows and lacks real end-to-end regression fixtures | +| Android | Native image conversion plus explicit unsupported contracts for broader media APIs | No media engine parity for audio or video workflows | +| Web | Stub | No documented support story | +| Tests | Basic web/iOS/Android contract tests plus iOS image/audio helper tests | No media fixture coverage, no Rust regressions, and no end-to-end flows | +| Docs | README documents support and roadmap links | No capability matrix by release phase and limited examples for new media operations | ## Scope model @@ -160,6 +160,7 @@ Exit criteria: Status: in progress - Stabilize the current re-encode path. +- Harden the initial `convertAudio()` path and keep its scope explicit until broader format support is real. - Add structured progress and cancellation semantics. - Clarify background execution behavior and platform minimums. - Add real iOS native tests using media fixtures. diff --git a/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java b/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java index f333b83..349365d 100644 --- a/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java +++ b/android/src/main/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPlugin.java @@ -90,6 +90,11 @@ public void convertImage(final PluginCall call) { } } + @PluginMethod + public void convertAudio(final PluginCall call) { + call.unimplemented(getUnsupportedOperationMessage("convertAudio")); + } + @PluginMethod public void getPluginVersion(final PluginCall call) { try { @@ -118,6 +123,7 @@ JSObject createCapabilitiesFeaturesPayload() { features.put("getCapabilities", createCapabilityPayload("getCapabilities")); features.put("reencodeVideo", createCapabilityPayload("reencodeVideo")); features.put("convertImage", createCapabilityPayload("convertImage")); + features.put("convertAudio", createCapabilityPayload("convertAudio")); features.put("progressEvents", createCapabilityPayload("progressEvents")); features.put("probeMedia", createCapabilityPayload("probeMedia")); features.put("generateThumbnail", createCapabilityPayload("generateThumbnail")); @@ -150,6 +156,7 @@ String getCapabilityStatus(final String feature) { case "getPluginVersion", "getCapabilities" -> "available"; case "reencodeVideo" -> "unimplemented"; case "convertImage" -> "available"; + case "convertAudio" -> "unimplemented"; case "progressEvents" -> "unavailable"; case "probeMedia", "generateThumbnail", "extractAudio", "remux", "trim" -> "unimplemented"; default -> "unimplemented"; @@ -160,6 +167,7 @@ String getCapabilityReason(final String feature) { return switch (feature) { case "reencodeVideo" -> getUnsupportedOperationMessage("reencodeVideo"); case "convertImage" -> "Still-image conversion is available on Android for webp, jpeg, and png outputs."; + case "convertAudio" -> getUnsupportedOperationMessage("convertAudio"); case "progressEvents" -> "No media jobs are available on Android today."; case "probeMedia" -> "probeMedia is not implemented on Android."; case "generateThumbnail" -> "generateThumbnail is not implemented on Android."; diff --git a/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java b/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java index 0132c91..36efe6b 100644 --- a/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java +++ b/android/src/test/java/ee/forgr/capacitor_ffmpeg/CapacitorFFmpegPluginTest.java @@ -43,11 +43,13 @@ public void capabilityHelpersDescribeTheCurrentAndroidScope() { assertEquals("available", plugin.getCapabilityStatus("getCapabilities")); assertEquals("unimplemented", plugin.getCapabilityStatus("reencodeVideo")); assertEquals("available", plugin.getCapabilityStatus("convertImage")); + assertEquals("unimplemented", plugin.getCapabilityStatus("convertAudio")); assertEquals("reencodeVideo is currently only available on iOS.", plugin.getCapabilityReason("reencodeVideo")); assertEquals( "Still-image conversion is available on Android for webp, jpeg, and png outputs.", plugin.getCapabilityReason("convertImage") ); + assertEquals("convertAudio is currently only available on iOS.", plugin.getCapabilityReason("convertAudio")); } @Test @@ -60,6 +62,8 @@ public void capabilitiesPayloadExposesTheJsVisibleContract() throws Exception { assertEquals("available", features.getJSONObject("getCapabilities").getString("status")); assertEquals("unimplemented", features.getJSONObject("reencodeVideo").getString("status")); assertEquals("reencodeVideo is currently only available on iOS.", features.getJSONObject("reencodeVideo").getString("reason")); + assertEquals("unimplemented", features.getJSONObject("convertAudio").getString("status")); + assertEquals("convertAudio is currently only available on iOS.", features.getJSONObject("convertAudio").getString("reason")); } @Test diff --git a/example-app/src/js/app.js b/example-app/src/js/app.js index 75eabe7..b9ee2c3 100644 --- a/example-app/src/js/app.js +++ b/example-app/src/js/app.js @@ -6,6 +6,7 @@ const FEATURE_ORDER = [ 'getCapabilities', 'reencodeVideo', 'convertImage', + 'convertAudio', 'progressEvents', 'probeMedia', 'generateThumbnail', @@ -19,6 +20,7 @@ const FEATURE_LABELS = { getCapabilities: 'Capability matrix', reencodeVideo: 'Re-encode video', convertImage: 'Convert image', + convertAudio: 'Convert audio', progressEvents: 'Progress events', probeMedia: 'Probe media', generateThumbnail: 'Generate thumbnail', diff --git a/example-app/src/js/app.vitest.js b/example-app/src/js/app.vitest.js index e318a5a..daf020e 100644 --- a/example-app/src/js/app.vitest.js +++ b/example-app/src/js/app.vitest.js @@ -24,6 +24,13 @@ function createCapabilities(platform, overrides = {}) { ? 'Still-image conversion is available on iOS.' : 'Image conversion is currently only available on iOS and Android.', }, + convertAudio: { + status: overrides.convertAudio ?? 'unimplemented', + reason: + overrides.convertAudio === 'available' + ? 'Audio conversion is available on iOS.' + : 'Audio conversion is currently only available on iOS.', + }, progressEvents: { status: overrides.reencodeVideo === 'experimental' ? 'available' : 'unavailable', reason: diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift index ac76024..c68707f 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpeg.swift @@ -1,4 +1,5 @@ import CapacitorFFmpegNativeCore +import AVFoundation import Foundation import ImageIO import UIKit @@ -40,6 +41,19 @@ protocol FFmpegNativeBinding { ) -> UnsafeMutablePointer? } +protocol AudioExportSessioning: AnyObject { + var outputURL: URL? { get set } + var outputFileType: AVFileType? { get set } + var status: AVAssetExportSession.Status { get } + var error: Error? { get } + + func exportAsynchronously(completionHandler handler: @escaping @Sendable () -> Void) +} + +extension AVAssetExportSession: AudioExportSessioning {} + +typealias AudioExportSessionFactory = (_ asset: AVAsset, _ presetName: String) -> (any AudioExportSessioning)? + private struct LinkedFFmpegNativeBindings: FFmpegNativeBinding { func initPlugin() -> UnsafeMutableRawPointer? { init_ffmpeg_plugin() @@ -100,6 +114,20 @@ struct FFmpegConvertedImage { } } +struct FFmpegConvertedAudio { + let outputPath: String + let format: String + + var asDictionary: [String: Any] { + [ + "outputPath": outputPath, + "format": format + ] + } +} + +typealias AudioConversionCompletion = (Result) -> Void + struct FFmpegCapabilityPayload { let status: String let reason: String? @@ -144,6 +172,10 @@ struct FFmpegCapabilitiesPayload { status: "available", reason: "Still-image conversion is available on iOS for jpeg and png outputs." ), + "convertAudio": FFmpegCapabilityPayload( + status: "available", + reason: "Audio conversion is available on iOS for m4a output." + ), "progressEvents": FFmpegCapabilityPayload( status: nativeCoreAvailable ? "available" : "unavailable", reason: nativeCoreAvailable @@ -232,17 +264,29 @@ private final class SelfForReencodeVideo { @objc public class CapacitorFFmpeg: NSObject { var pointerToRustPlugin: UnsafeMutableRawPointer? private let nativeBindings: any FFmpegNativeBinding + private let audioExportSessionFactory: AudioExportSessionFactory private let nativeCoreReason: String? /// Progress callback closure that can be set from outside var onProgress: ((FFmpegProgressPayload) -> Void)? - public override convenience init() { + override public convenience init() { self.init(nativeBindings: LinkedFFmpegNativeBindings()) } - init(nativeBindings: any FFmpegNativeBinding) { + convenience init(audioExportSessionFactory: @escaping AudioExportSessionFactory) { + self.init( + nativeBindings: LinkedFFmpegNativeBindings(), + audioExportSessionFactory: audioExportSessionFactory + ) + } + + init( + nativeBindings: any FFmpegNativeBinding, + audioExportSessionFactory: @escaping AudioExportSessionFactory = CapacitorFFmpeg.defaultAudioExportSessionFactory + ) { self.nativeBindings = nativeBindings + self.audioExportSessionFactory = audioExportSessionFactory let resolvedPlugin = nativeBindings.initPlugin() self.pointerToRustPlugin = resolvedPlugin self.nativeCoreReason = resolvedPlugin == nil ? "The native FFmpeg core could not be initialized." : nil @@ -254,6 +298,13 @@ private final class SelfForReencodeVideo { } } + private static func defaultAudioExportSessionFactory( + asset: AVAsset, + presetName: String + ) -> (any AudioExportSessioning)? { + AVAssetExportSession(asset: asset, presetName: presetName) + } + deinit { if let plugin = self.pointerToRustPlugin { nativeBindings.deinitPlugin(plugin) @@ -469,6 +520,77 @@ private final class SelfForReencodeVideo { ) } + func convertAudio( + inputPath: String, + outputPath: String, + format: String, + completion: @escaping AudioConversionCompletion + ) throws { + let inputURL = try resolveFileURL(from: inputPath).standardizedFileURL + let outputURL = try resolveFileURL(from: outputPath).standardizedFileURL + let normalizedFormat = format.lowercased() + + guard inputURL.path != outputURL.path else { + throw FFmpegError.invalidArgument("In-place conversion is not allowed. Choose a different output path.") + } + + guard normalizedFormat == "m4a" else { + throw FFmpegError.invalidArgument("Unsupported audio format: \(format)") + } + guard outputURL.pathExtension.lowercased() == normalizedFormat else { + throw FFmpegError.invalidArgument("Output path extension must be .\(normalizedFormat).") + } + + let asset = AVURLAsset(url: inputURL) + guard !asset.tracks(withMediaType: .audio).isEmpty else { + throw FFmpegError.invalidArgument("The input media does not contain an audio track.") + } + + let fileManager = FileManager.default + let outputDirectory = outputURL.deletingLastPathComponent() + try fileManager.createDirectory( + at: outputDirectory, + withIntermediateDirectories: true + ) + let temporaryOutputURL = outputDirectory + .appendingPathComponent(".ffmpeg-audio-\(UUID().uuidString)") + .appendingPathExtension(outputURL.pathExtension.isEmpty ? "m4a" : outputURL.pathExtension) + + guard let exportSession = audioExportSessionFactory(asset, AVAssetExportPresetAppleM4A) else { + throw FFmpegError.transcodeFailed("Could not create the audio export session.") + } + + exportSession.outputURL = temporaryOutputURL + exportSession.outputFileType = .m4a + + exportSession.exportAsynchronously { + defer { + try? fileManager.removeItem(at: temporaryOutputURL) + } + + do { + guard exportSession.status == .completed else { + throw FFmpegError.transcodeFailed( + exportSession.error?.localizedDescription ?? "Could not export the output audio." + ) + } + + if fileManager.fileExists(atPath: outputURL.path) { + _ = try fileManager.replaceItemAt(outputURL, withItemAt: temporaryOutputURL) + } else { + try fileManager.moveItem(at: temporaryOutputURL, to: outputURL) + } + + completion(.success(FFmpegConvertedAudio( + outputPath: outputURL.absoluteString, + format: normalizedFormat + ))) + } catch { + completion(.failure(error)) + } + } + } + func getCapabilities() -> FFmpegCapabilitiesPayload { FFmpegCapabilitiesPayload.iosCurrent( nativeCoreAvailable: pointerToRustPlugin != nil, @@ -481,6 +603,7 @@ private final class SelfForReencodeVideo { public enum FFmpegError: LocalizedError { case pluginNotInitialized case reencodingFailed(String) + case transcodeFailed(String) case invalidPath(String) case invalidArgument(String) @@ -488,7 +611,7 @@ public enum FFmpegError: LocalizedError { switch self { case .pluginNotInitialized: return "PLUGIN_NOT_INITIALIZED" - case .reencodingFailed: + case .reencodingFailed, .transcodeFailed: return "TRANSCODE_FAILED" case .invalidPath, .invalidArgument: return "INVALID_ARGUMENT" @@ -501,6 +624,8 @@ public enum FFmpegError: LocalizedError { return "FFmpeg plugin was not properly initialized" case .reencodingFailed(let message): return "Video re-encoding failed: \(message)" + case .transcodeFailed(let message): + return "Media transcode failed: \(message)" case .invalidPath(let path): return "Invalid file path: \(path)" case .invalidArgument(let message): diff --git a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift index 74e0b9d..08e38fc 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/CapacitorFFmpegPlugin.swift @@ -13,6 +13,7 @@ public class CapacitorFFmpegPlugin: CAPPlugin, CAPBridgedPlugin { CAPPluginMethod(name: "getCapabilities", returnType: CAPPluginReturnPromise), CAPPluginMethod(name: "reencodeVideo", returnType: CAPPluginReturnPromise), CAPPluginMethod(name: "convertImage", returnType: CAPPluginReturnPromise), + CAPPluginMethod(name: "convertAudio", returnType: CAPPluginReturnPromise), CAPPluginMethod(name: "getPluginVersion", returnType: CAPPluginReturnPromise) ] @@ -108,6 +109,44 @@ public class CapacitorFFmpegPlugin: CAPPlugin, CAPBridgedPlugin { } } + @objc func convertAudio(_ call: CAPPluginCall) { + guard let inputPath = call.getString("inputPath") else { + call.reject("Input path is required", "INVALID_ARGUMENT") + return + } + guard let outputPath = call.getString("outputPath") else { + call.reject("Output path is required", "INVALID_ARGUMENT") + return + } + guard let format = call.getString("format") else { + call.reject("Output format is required", "INVALID_ARGUMENT") + return + } + + do { + try implementation.convertAudio( + inputPath: inputPath, + outputPath: outputPath, + format: format + ) { [weak self] result in + DispatchQueue.main.async { + switch result { + case .success(let convertedAudio): + call.resolve(convertedAudio.asDictionary) + case .failure(let error): + guard let self else { + call.reject(error.localizedDescription, "TRANSCODE_FAILED") + return + } + self.reject(call, with: error) + } + } + } + } catch { + reject(call, with: error) + } + } + @objc func getPluginVersion(_ call: CAPPluginCall) { call.resolve(["version": CapacitorFFmpegPluginVersion.value]) } diff --git a/ios/Sources/CapacitorFFmpegPlugin/PluginVersion.generated.swift b/ios/Sources/CapacitorFFmpegPlugin/PluginVersion.generated.swift index 2ab09d7..03e502d 100644 --- a/ios/Sources/CapacitorFFmpegPlugin/PluginVersion.generated.swift +++ b/ios/Sources/CapacitorFFmpegPlugin/PluginVersion.generated.swift @@ -1,3 +1,3 @@ enum CapacitorFFmpegPluginVersion { - static let value = "0.0.8" + static let value = "0.0.9" } diff --git a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift index 53769f6..7b5e677 100644 --- a/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift +++ b/ios/Tests/CapacitorFFmpegPluginTests/CapacitorFFmpegPluginTests.swift @@ -1,3 +1,4 @@ +import AVFoundation import CapacitorFFmpegNativeCore import Darwin import UIKit @@ -39,7 +40,66 @@ private struct FailingFFmpegNativeBindings: FFmpegNativeBinding { } } +private final class MockAudioExportSession: AudioExportSessioning { + var outputURL: URL? + var outputFileType: AVFileType? + var status: AVAssetExportSession.Status + var error: Error? + + private let onExport: (MockAudioExportSession) -> Void + + init( + status: AVAssetExportSession.Status = .completed, + error: Error? = nil, + onExport: @escaping (MockAudioExportSession) -> Void = { _ in } + ) { + self.status = status + self.error = error + self.onExport = onExport + } + + func exportAsynchronously(completionHandler handler: @escaping @Sendable () -> Void) { + onExport(self) + handler() + } +} + +private func writeToneWav(to url: URL) throws { + let toneFormat = try XCTUnwrap(AVAudioFormat(standardFormatWithSampleRate: 44_100, channels: 1)) + let toneFile = try AVAudioFile(forWriting: url, settings: toneFormat.settings) + let toneBuffer = try XCTUnwrap(AVAudioPCMBuffer(pcmFormat: toneFormat, frameCapacity: 44_100)) + toneBuffer.frameLength = 44_100 + let samples = try XCTUnwrap(toneBuffer.floatChannelData?[0]) + for index in 0.. Result { + let expectation = expectation(description: "convertAudio") + var conversionResult: Result? + + try plugin.convertAudio( + inputPath: inputPath, + outputPath: outputPath, + format: format + ) { result in + conversionResult = result + expectation.fulfill() + } + + wait(for: [expectation], timeout: timeout) + return try XCTUnwrap(conversionResult) + } + func testCapabilitiesPayloadDescribesTheCurrentIosScope() { let payload = CapacitorFFmpeg().getCapabilities().asDictionary let features = payload["features"] as? [String: [String: Any]] @@ -48,6 +108,7 @@ final class CapacitorFFmpegPluginTests: XCTestCase { XCTAssertEqual(features?["getCapabilities"]?["status"] as? String, "available") XCTAssertEqual(features?["reencodeVideo"]?["status"] as? String, "experimental") XCTAssertEqual(features?["convertImage"]?["status"] as? String, "available") + XCTAssertEqual(features?["convertAudio"]?["status"] as? String, "available") } func testCapabilitiesPayloadExplainsNativeCoreInitializationFailure() { @@ -139,10 +200,10 @@ final class CapacitorFFmpegPluginTests: XCTestCase { } func testConvertImageWritesAnOutputFile() throws { - let fm = FileManager.default - let baseURL = fm.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) - try fm.createDirectory(at: baseURL, withIntermediateDirectories: true) - defer { try? fm.removeItem(at: baseURL) } + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } let inputURL = baseURL.appendingPathComponent("input.png") let outputURL = baseURL.appendingPathComponent("output.png") @@ -163,6 +224,166 @@ final class CapacitorFFmpegPluginTests: XCTestCase { XCTAssertEqual(result.format, "png") XCTAssertEqual(result.outputPath, outputURL.absoluteString) - XCTAssertTrue(fm.fileExists(atPath: outputURL.path)) + XCTAssertTrue(fileManager.fileExists(atPath: outputURL.path)) + } + + func testConvertAudioRejectsUnsupportedFormats() throws { + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } + + let inputURL = baseURL.appendingPathComponent("input.wav") + let outputURL = baseURL.appendingPathComponent("output.wav") + + try writeToneWav(to: inputURL) + + XCTAssertThrowsError( + try CapacitorFFmpeg().convertAudio( + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "wav" + ) { _ in } + ) { error in + XCTAssertEqual((error as? FFmpegError)?.code, "INVALID_ARGUMENT") + XCTAssertEqual(error.localizedDescription, "Unsupported audio format: wav") + } + } + + func testConvertAudioRejectsMismatchedOutputExtension() throws { + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } + + let inputURL = baseURL.appendingPathComponent("input.wav") + let outputURL = baseURL.appendingPathComponent("output.wav") + + try writeToneWav(to: inputURL) + + XCTAssertThrowsError( + try CapacitorFFmpeg().convertAudio( + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) { _ in } + ) { error in + XCTAssertEqual((error as? FFmpegError)?.code, "INVALID_ARGUMENT") + XCTAssertEqual(error.localizedDescription, "Output path extension must be .m4a.") + } + } + + func testConvertAudioWritesM4AOutputWhenExportSucceeds() throws { + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } + + let inputURL = baseURL.appendingPathComponent("input.wav") + let outputURL = baseURL.appendingPathComponent("output.m4a") + try writeToneWav(to: inputURL) + try Data("stale".utf8).write(to: outputURL) + + let exportSession = MockAudioExportSession { session in + XCTAssertNotEqual(session.outputURL, outputURL) + XCTAssertEqual(session.outputURL?.deletingLastPathComponent(), outputURL.deletingLastPathComponent()) + XCTAssertEqual(session.outputFileType, .m4a) + try? Data("converted-audio".utf8).write(to: XCTUnwrap(session.outputURL)) + session.status = .completed + } + + let result = try waitForAudioConversion(using: CapacitorFFmpeg(audioExportSessionFactory: { asset, presetName in + XCTAssertEqual(asset.tracks(withMediaType: .audio).count, 1) + XCTAssertEqual(presetName, AVAssetExportPresetAppleM4A) + return exportSession + }), + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) + + switch result { + case .success(let convertedAudio): + XCTAssertEqual(convertedAudio.format, "m4a") + XCTAssertEqual(convertedAudio.outputPath, outputURL.absoluteString) + case .failure(let error): + XCTFail("Expected successful audio conversion, got \(error)") + } + XCTAssertEqual(try String(contentsOf: outputURL), "converted-audio") + } + + func testConvertAudioRemovesPartialOutputWhenExportFails() throws { + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } + + let inputURL = baseURL.appendingPathComponent("input.wav") + let outputURL = baseURL.appendingPathComponent("output.m4a") + try writeToneWav(to: inputURL) + try Data("keep-existing-output".utf8).write(to: outputURL) + + let exportError = NSError( + domain: "CapacitorFFmpegPluginTests", + code: 1, + userInfo: [NSLocalizedDescriptionKey: "simulated export failure"] + ) + let exportSession = MockAudioExportSession(status: .failed, error: exportError) { session in + XCTAssertNotEqual(session.outputURL, outputURL) + try? Data("partial-output".utf8).write(to: XCTUnwrap(session.outputURL)) + } + + let result = try waitForAudioConversion( + using: CapacitorFFmpeg(audioExportSessionFactory: { _, _ in exportSession }), + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) + + switch result { + case .success: + XCTFail("Expected audio conversion failure") + case .failure(let error): + XCTAssertEqual((error as? FFmpegError)?.code, "TRANSCODE_FAILED") + XCTAssertEqual(error.localizedDescription, "Media transcode failed: simulated export failure") + } + + XCTAssertEqual(try String(contentsOf: outputURL), "keep-existing-output") + } + + func testConvertAudioRemovesPartialTemporaryOutputWhenNoDestinationExists() throws { + let fileManager = FileManager.default + let baseURL = fileManager.temporaryDirectory.appendingPathComponent(UUID().uuidString, isDirectory: true) + try fileManager.createDirectory(at: baseURL, withIntermediateDirectories: true) + defer { try? fileManager.removeItem(at: baseURL) } + + let inputURL = baseURL.appendingPathComponent("input.wav") + let outputURL = baseURL.appendingPathComponent("output.m4a") + try writeToneWav(to: inputURL) + + let exportError = NSError( + domain: "CapacitorFFmpegPluginTests", + code: 2, + userInfo: [NSLocalizedDescriptionKey: "simulated export failure"] + ) + let exportSession = MockAudioExportSession(status: .failed, error: exportError) { session in + try? Data("partial-output".utf8).write(to: XCTUnwrap(session.outputURL)) + } + + let result = try waitForAudioConversion( + using: CapacitorFFmpeg(audioExportSessionFactory: { _, _ in exportSession }), + inputPath: inputURL.path, + outputPath: outputURL.path, + format: "m4a" + ) + + switch result { + case .success: + XCTFail("Expected audio conversion failure") + case .failure(let error): + XCTAssertEqual((error as? FFmpegError)?.code, "TRANSCODE_FAILED") + } + + XCTAssertFalse(fileManager.fileExists(atPath: outputURL.path)) } } diff --git a/src/definitions.ts b/src/definitions.ts index 463e41c..9a6f0ac 100644 --- a/src/definitions.ts +++ b/src/definitions.ts @@ -11,6 +11,7 @@ export type FFmpegCapabilityStatus = 'available' | 'experimental' | 'unimplement export type FFmpegJobState = 'queued' | 'running' | 'completed' | 'failed'; export type FFmpegProgressState = 'running' | 'completed' | 'failed'; export type ImageOutputFormat = 'webp' | 'jpeg' | 'png'; +export type AudioOutputFormat = 'm4a'; export interface FFmpegCapability { status: FFmpegCapabilityStatus; @@ -22,6 +23,7 @@ export interface FFmpegCapabilitiesFeatures { getCapabilities: FFmpegCapability; reencodeVideo: FFmpegCapability; convertImage: FFmpegCapability; + convertAudio?: FFmpegCapability; progressEvents: FFmpegCapability; probeMedia: FFmpegCapability; generateThumbnail: FFmpegCapability; @@ -65,6 +67,17 @@ export interface ConvertImageResult { format: ImageOutputFormat; } +export interface ConvertAudioOptions { + inputPath: string; + outputPath: string; + format: AudioOutputFormat; +} + +export interface ConvertAudioResult { + outputPath: string; + format: AudioOutputFormat; +} + export interface FFmpegProgressEvent { jobId: string; /** @@ -109,6 +122,14 @@ export interface CapacitorFFmpegPlugin extends Plugin { */ convertImage(options: ConvertImageOptions): Promise; + /** + * Convert audio into another container or codec. + * + * iOS currently supports `m4a`. + * Android and web currently reject with `UNIMPLEMENTED`. + */ + convertAudio?(options: ConvertAudioOptions): Promise; + /** * Listen for media job progress. */ diff --git a/src/pluginVersion.ts b/src/pluginVersion.ts index 6f21a3b..0273c9d 100644 --- a/src/pluginVersion.ts +++ b/src/pluginVersion.ts @@ -1 +1 @@ -export const PLUGIN_VERSION = '0.0.8'; +export const PLUGIN_VERSION = '0.0.9'; diff --git a/src/web.ts b/src/web.ts index dbf2f5d..0968086 100644 --- a/src/web.ts +++ b/src/web.ts @@ -2,10 +2,12 @@ import { WebPlugin } from '@capacitor/core'; import type { CapacitorFFmpegPlugin, - FFmpegAcceptedJob, - FFmpegCapabilitiesResult, + ConvertAudioOptions, + ConvertAudioResult, ConvertImageOptions, ConvertImageResult, + FFmpegAcceptedJob, + FFmpegCapabilitiesResult, PluginVersionResult, ReencodeVideoOptions, } from './definitions'; @@ -26,6 +28,10 @@ export class CapacitorFFmpegWeb extends WebPlugin implements CapacitorFFmpegPlug status: 'unimplemented', reason: 'Image conversion is currently only available on iOS and Android.', }, + convertAudio: { + status: 'unimplemented', + reason: 'Audio conversion is currently only available on iOS.', + }, progressEvents: { status: 'unavailable', reason: 'No media jobs are available on web today.', @@ -64,6 +70,11 @@ export class CapacitorFFmpegWeb extends WebPlugin implements CapacitorFFmpegPlug throw this.unimplemented('convertImage is currently only available on iOS and Android.'); } + async convertAudio(options: ConvertAudioOptions): Promise { + void options; + throw this.unimplemented('convertAudio is currently only available on iOS.'); + } + async getPluginVersion(): Promise { return { version: PLUGIN_VERSION }; } diff --git a/test/TEST_INVENTORY.md b/test/TEST_INVENTORY.md index 41c682b..77d3e68 100644 --- a/test/TEST_INVENTORY.md +++ b/test/TEST_INVENTORY.md @@ -1,6 +1,6 @@ # Test Inventory -Last updated: 2026-04-04 +Last updated: 2026-04-13 This file maps the public plugin contract to the tests that exist now and the next regression coverage we still need. @@ -13,6 +13,8 @@ This file maps the public plugin contract to the tests that exist now and the ne | `reencodeVideo` unsupported behavior | Plugin-only contract | Web contract test, Android unit test helper | None for current scope | | `convertImage` unsupported behavior | Plugin-only contract | Web contract test | None for current scope | | `convertImage` native execution | Still-image transcode | Android unit test helper, iOS native test for fixture generation and output write, Maestro Android+iOS flows | Add wrapper-level test with `CAPPluginCall` and output file metadata validation | +| `convertAudio` unsupported behavior | Plugin-only contract | Web contract test, Android unit test helper | None for current scope | +| `convertAudio` native execution | Audio transcode | iOS native tests for format validation, successful export wrapper behavior, and failed export cleanup | Add end-to-end regression against a real exported file when simulator export is stable | | `reencodeVideo` acceptance result | Plugin job contract | iOS helper tests for queued job payload | iOS plugin wrapper test | | `progress` event payload | Plugin job/progress contract | iOS helper tests | End-to-end event assertion during a real encode | | Example app runtime harness | App-level contract smoke | Vitest example-app checks plus Maestro iOS/Android flows for runtime checks, image conversion, and iOS video flow | Add native device smoke flow: pick arbitrary gallery media -> transcode -> verify metadata | @@ -43,5 +45,5 @@ We are not importing upstream FATE directly. We are using it as a taxonomy: ## Immediate next media tests 1. Generate a tiny MP4 input fixture for the current iOS re-encode flow. -2. Add an iOS-native regression that checks output dimensions and duration tolerance. +2. Add a real exported-audio regression once simulator-side `AVAssetExportSession` output is stable in CI. 3. Add a broken-input fixture and assert structured failure reporting. diff --git a/test/web.test.ts b/test/web.test.ts index 5a58631..a23ed0d 100644 --- a/test/web.test.ts +++ b/test/web.test.ts @@ -20,6 +20,7 @@ describe('CapacitorFFmpegWeb', () => { getCapabilities: { status: 'available' }, reencodeVideo: { status: 'unimplemented' }, convertImage: { status: 'unimplemented' }, + convertAudio: { status: 'unimplemented' }, }, }); }); @@ -49,5 +50,16 @@ describe('CapacitorFFmpegWeb', () => { code: ExceptionCode.Unimplemented, message: 'convertImage is currently only available on iOS and Android.', }); + + await expect( + plugin.convertAudio({ + inputPath: 'file:///input.wav', + outputPath: 'file:///output.m4a', + format: 'm4a', + }), + ).rejects.toMatchObject({ + code: ExceptionCode.Unimplemented, + message: 'convertAudio is currently only available on iOS.', + }); }); });