Integrating FFmpeg for Multimedia Processing in Mobile Application
FFmpeg on a mobile device is when you need to transcode video, extract audio, merge multiple tracks, convert format or split media files directly on the phone without sending to server. Native APIs (AVFoundation, MediaCodec) don't cover all these scenarios — FFmpeg does.
Connecting FFmpeg to Project
iOS. ffmpeg-kit — mobile FFmpeg build with Swift/Objective-C wrapper:
// Podfile
pod 'ffmpeg-kit-ios-full', '~> 6.0'
// Usage
FFmpegKit.executeAsync("-i input.mp4 -vn -acodec copy output.aac") { session in
guard let returnCode = session?.getReturnCode() else { return }
if ReturnCode.isSuccess(returnCode) {
print("Done: \(session?.getOutput() ?? "")")
} else {
print("Error: \(session?.getLogsAsString() ?? "")")
}
}
Android. Same ffmpeg-kit-android (Maven):
implementation("com.arthenica:ffmpeg-kit-android:6.0.LTS")
FFmpegKit.executeAsync(
"-i input.mp4 -vf scale=1280:720 -c:v libx264 -preset ultrafast -crf 23 output.mp4"
) { session ->
if (ReturnCode.isSuccess(session.returnCode)) {
// process result on main thread
}
}
ffmpeg-kit comes in several build variants: min (minimal codec set), min-gpl, https, full, full-gpl. For most tasks https is sufficient (H.264, AAC, MP3, HTTPS streams). full-gpl includes x264, x265, libvpx — needed if you want to encode H.264 via libx264 instead of system codec.
Flutter. ffmpeg_kit_flutter (pub.dev) — same library with Dart wrapper.
Typical Tasks and Commands
Video Transcoding
# H.264, 720p, CRF 23 (quality/size balance)
-i input.mov -c:v libx264 -preset ultrafast -crf 23 -c:a aac -b:a 128k -vf scale=1280:720 output.mp4
-preset ultrafast — encoding speed matters more than file size. CPU on phone is slow, ultrafast or superfast is reasonable choice for UGC content. veryfast gives noticeably smaller size but encodes 2–3x longer.
Trimming Without Re-encoding
# Stream copy — fast, no quality loss, but accuracy to keyframe
-i input.mp4 -ss 00:00:10 -to 00:00:30 -c copy output.mp4
# Exact trimming — slower but frame-accurate
-i input.mp4 -ss 00:00:10 -to 00:00:30 output.mp4
-c copy doesn't re-encode — works in seconds. Without -c copy FFmpeg decodes and re-encodes every frame. For UI: show "Fast trim" button (with copy) and "Accurate trim" (with re-encoding).
Merging Files
# Concat via file list (single format)
-f concat -safe 0 -i filelist.txt -c copy output.mp4
filelist.txt contains paths: file '/path/to/clip1.mp4'\nfile '/path/to/clip2.mp4'. Create file programmatically in temp directory before call.
Extracting Audio, Replacing Audio Track
# Extract audio
-i input.mp4 -vn -acodec copy output.aac
# Replace audio in video
-i video.mp4 -i audio.aac -c:v copy -map 0:v:0 -map 1:a:0 output.mp4
Watermark Overlay
-i input.mp4 -i watermark.png -filter_complex "overlay=W-w-10:H-h-10" output.mp4
W-w-10:H-h-10 — bottom right corner with 10 px offset.
Progress Monitoring
FFmpeg writes progress to stderr. ffmpeg-kit provides StatisticsCallback:
FFmpegKit.executeAsync(command,
withCompleteCallback: { session in /* done */ },
withLogCallback: nil,
withStatisticsCallback: { stats in
guard let duration = totalDurationMs else { return }
let progress = Double(stats?.getTime() ?? 0) / duration
DispatchQueue.main.async { self.progressBar.progress = Float(progress) }
}
)
stats.getTime() — current processing position in milliseconds. Get totalDurationMs via FFprobeKit.getMediaInformationAsync.
Performance and Limitations
On iPhone 14 Pro, transcoding 1-minute 1080p video via libx264 ultrafast takes ~45–60 seconds. That's a lot for UX. Alternatives:
-
System encoder via
VideoToolbox:-c:v h264_videotoolbox— uses Apple hardware acceleration, 5–10x faster, less bitrate control -
On Android:
-c:v h264_mediacodec— hardware H.264 via MediaCodec
Hardware encoder isn't available on all devices and versions. Add fallback to libx264:
val command = if (isHardwareEncoderAvailable()) {
"-i input.mp4 -c:v h264_mediacodec output.mp4"
} else {
"-i input.mp4 -c:v libx264 -preset ultrafast output.mp4"
}
Binary size: ffmpeg-kit-full adds ~30–50 MB to app size. ffmpeg-kit-min — ~8–12 MB. Choose minimal build for specific tasks.
Timeline
Basic integration with one or two operations (trim + convert) — 2 days. Full media processor with progress, task queue and system API fallback — 4–5 days.