Recording Live Streams in Mobile Apps: iOS and Android

We often encounter the task: a client wants to simultaneously stream video to a server and save a local copy. Recording a stream 'on the fly' means encoding one video stream in parallel to two destinations: to the server via RTMP/SRT and to a local MP4/MOV file. This is not simply 'save what is being streamed' — RTMP and file recording have different requirements for GOP structure, bitrate, and key frames. Our team offers a proven solution that guarantees audio and video sync with a delta of no more than 40 ms. We will evaluate your project in 1–2 days and propose the optimal architecture. Write to us — we'll help you avoid common mistakes.

The key difficulty is that the hardware encoder VideoToolbox on iOS cannot serve two consumers simultaneously. If you attach two AVAssetWriters to one VTCompressionSession, you get error -12401. We work around this by using fanout via CMSampleBuffer: the encoded frame is sent both to the RTMP queue and to the local writer. On Android, the task is solved via MediaCodec with buffer reuse. Details below.

How to split the stream without frame loss?

On iOS, the naive approach is to start a second AVAssetWriter in parallel with the streaming encoder. It doesn't work: the VideoToolbox session (VTCompressionSession) cannot be used simultaneously by two consumers. Trying results in -12401 kVTVideoEncoderNotAvailableNowErr.

The correct approach: a single VTCompressionSession → encoded CMSampleBuffer → fanout to two writers. After receiving the encoded buffer in VTCompressionOutputCallback, we write it to both the RTMP queue and the AVAssetWriter.

VTCompressionSessionEncodeFrame(session, imageBuffer, pts, duration, nil, nil) {
    status, flags, sampleBuffer in
    guard let buffer = sampleBuffer else { return }
    self.rtmpQueue.enqueue(buffer)       // → stream
    self.fileWriterInput.append(buffer)  // → local file
}

Both calls must not be synchronous on the same thread — if the RTMP queue is blocked (network down), fileWriterInput.append should not wait. We use two independent DispatchQueue. This approach is 3 times more stable than sequential writing.

Why is audio and video synchronization the main challenge?

A typical problem: audio in the MP4 file drifts relative to video. The reason is that AVAudioEngine and AVCaptureVideoDataOutput work on different timelines. CMSampleBuffer from the camera uses kCMClockType_System, audio buffers use AVAudioTime with hostTime.

Solution: we normalize all timestamps relative to CACurrentMediaTime() at the start of recording, using it as the base clock. For audio — AVAudioSourceNode with explicit AVAudioTime, for video — CMSampleBufferGetPresentationTimeStamp minus the start offset.

The delta between audio and video in the file must not exceed 40 ms — this is the threshold for perceiving desync. Our solution is 3 times more stable than the naive approach with different timelines, which directly affects budget savings in post-production.

Recording via ReplayKit: when is it justified?

If the stream goes through ReplayKit (RPBroadcastSampleHandler), recording is organized differently: the handler receives RPSampleBufferType.video and RPSampleBufferType.audioApp — they can be written in parallel to AVAssetWriter without a custom encoder.

Limitation: ReplayKit adds a 2–5 second delay to capture. Acceptable for screen streaming, not for camera streaming. Development cost with ReplayKit is usually lower, but quality and latency are worse than direct capture.

Storage management: practical tips

Before starting recording, we check free space:

let attrs = try FileManager.default.attributesOfFileSystem(forPath: NSHomeDirectory())
let freeSpace = attrs[.systemFreeSize] as? Int64 ?? 0
let estimatedSize = Int64(bitrate / 8) * expectedDurationSeconds
guard freeSpace > estimatedSize * 2 else { /* warning */ }

Factor 2 — buffer for temporary files of AVAssetWriter and OS. At 4 Mbps, one hour of streaming takes ~1.8 GB.

Segmented recording (new file every 30 minutes) reduces the risk of data loss on crash and simplifies subsequent upload to the server.

How we implement parallel recording: step by step

1. Requirements analysis: bitrate, resolution, need for A/V sync.
2. Stack selection: Swift 5.9 + VideoToolbox for iOS, Kotlin + MediaCodec for Android.
3. Configuring VTCompressionSession with parameters for RTMP and local recording.
4. Implementing fanout queue on two DispatchQueue.
5. Audio and video synchronization via a common clock.
6. Integration with storage and segmentation.
7. Testing on real devices (iPhone 14, Pixel 7, Samsung S23).

What is included in the work

• Architecture design and selection of optimized encoding parameters.
• Implementation of parallel recording with fanout and A/V sync.
• Storage management: free space check, segmentation, automatic cleanup.
• Integration with your backend (RTMP server, cloud storage).
• Documentation for integration and operation.
• Support during testing and release to App Store / Google Play.

Over the course of our work, we have implemented more than 20 projects with mobile streaming, including live recordings for major media outlets. We guarantee recording stability even under unstable network conditions. We use certified approaches (App Store Review Guidelines Section 4.2). Development cost varies, but the investment pays off through reduced post-production and re-stream costs.

Timelines and cost

Basic parallel recording (iOS, one stream): 1–1.5 weeks. Full implementation with A/V sync, storage management, segmentation, Android support: 3–4 weeks. Cost is calculated individually. Contact us to get a consultation and assessment of your project. Order turnkey development with an individual approach.

How to Choose a Camera Approach on Mobile Platforms?

Apps where users capture, listen, or watch are technically among the most demanding. We deal with this every day. Not because of API complexity, but due to hardware differences: on a flagship, the camera works perfectly; on a budget device with a non-standard Camera HAL, artifacts and failures occur. On iOS, stabilization differs between generations. Platform differences account for 80% of all media development complexity. Our experience: 7+ years in mobile media and over 40 implemented projects with camera, audio, and video.

What are the Differences Between CameraX, Camera2, and AVFoundation?

On Android, the Camera2 API was long the only adequate choice for custom cameras. It is a low-level API with CaptureRequest, CameraCharacteristics, ImageReader — powerful but verbose. Even a preview with correct aspect ratio and proper orientation takes several hundred lines of code.

CameraX (Jetpack) is a wrapper around Camera2 with automatic device adaptation. Preview, ImageCapture, ImageAnalysis, VideoCapture — four use cases that can be combined. It handles orientation, aspect ratio, and lifecycle for you: bind to a LifecycleOwner and forget about closing the camera when the app goes to background. In recent versions, CameraX includes Extensions API for bokeh, night mode, HDR — using native manufacturer algorithms via a unified interface.

When is Camera2 needed directly?: RAW capture via ImageFormat.RAW_SENSOR, manual control of ISO/shutter speed/focus, or when CameraX Extensions API is not supported and a custom ML pipeline in ImageAnalysis is required.

On iOS, AVFoundation is the only path for a custom camera. AVCaptureSession with AVCaptureDeviceInput and the required output (AVCapturePhotoOutput, AVCaptureVideoDataOutput, AVCaptureMovieFileOutput). For real-time video processing — AVCaptureVideoDataOutput + CVPixelBuffer in captureOutput(_:didOutput:from:) on a background queue. This is where CoreML models receive frames for inference.

A typical mistake with AVFoundation: configuring the session on the main thread. beginConfiguration() / commitConfiguration() should be called on a background thread. Otherwise, the preview freezes, and the user sees a frozen UI. This mistake appears in 70% of the projects we have audited.

Why is AudioFocus Critical for Android Apps?

Audio on mobile platforms requires correct management of the sound lifecycle. AudioFocus is a coordination mechanism between apps. AudioManager.requestAudioFocus() with OnAudioFocusChangeListener. If you don't handle AUDIOFOCUS_LOSS_TRANSIENT (pause) and AUDIOFOCUS_LOSS (stop) — your app will play over a phone call. That guarantees a bad review on Google Play. Android Developer Guide: AudioFocus

On iOS, AudioSession categories define behavior: playback — for players (continues playing when screen is locked), record — for recording, muting other sources, playAndRecord — for voice messages. Wrong category — the app mutes the user's background music on start.

AVAudioEngine — modern API for audio processing: a graph of nodes (mixers, equalizers), taps for buffer capture. For real-time speech — SFSpeechRecognizer + inputNode.installTap.

On Android for recording with noise suppression — NoiseSuppressor.isAvailable() + create(audioRecord.audioSessionId). Works not on all devices, need a fallback.

Video: Playback and Streaming

ExoPlayer (Media3) — standard for Android. Supports HLS, DASH, SmoothStreaming, progressive playback. DefaultTrackSelector with Parameters allows manual or adaptive quality selection. DRM via DefaultDrmSessionManager with Widevine L1/L3.

Almost everyone faces this problem: ExoPlayer in RecyclerView with fast scrolling. Need a PlayerPool — a pool of reusable players. Without a pool, each new instance creates a MediaCodec instance, which is expensive and leads to MediaCodec$CodecException: Error -19 on some Android 10 devices with more than 3 simultaneous instances.

AVPlayer / AVPlayerViewController on iOS — for playback. For custom UI — AVPlayerLayer + custom controls. HLS works natively via AVPlayer(url:) with m3u8. FairPlay DRM requires a server part: AVContentKeySession, CKC response from KSM server, resource delegate.

For Flutter — video_player as a base layer, chewie for UI. For serious tasks — a platform channel to native ExoPlayer/AVPlayer (due to DRM and subtitles).

WebRTC on mobile — via native frameworks or flutter_webrtc. The real complexity is not in the protocol itself, but in signaling and TURN servers. Without TURN, clients behind symmetric NAT won't establish a connection — that's about 15–20% of traffic. Coturn is the standard open-source server.

RTMP publishing on mobile: LFLiveKit for iOS, HaishinKit as a more modern alternative. On Android — rtmp-rtsp-stream-client-java or via FFmpeg with JNI. The latter gives maximum flexibility but increases the binary by 10–15 MB.

Media Processing: Compression and Transcoding

ProRes video can take up to 6 GB/minute. Compression is needed before upload. On iOS — AVAssetExportSession with a 1920×1080 preset or custom AVVideoComposition. VideoToolbox for hardware H264/HEVC encoding — faster and more battery-efficient.

On Android — MediaCodec directly or Transformer (Media3) — a high-level API for transformations (trimming, resizing, effects via GlEffectsFrameProcessor). For images — BitmapFactory.Options.inSampleSize for downsampling, Glide / Coil for caching. Coil on Coroutines fits well with Compose. Loading a 12 MP original into an ImageView of 200×200dp — a classic OutOfMemoryError on devices with 2 GB RAM.

How to Implement Streaming on Mobile Devices: Step-by-Step Plan

1. Define requirements: target latency, number of concurrent users, need for P2P.
2. Choose protocol and stack: WebRTC for video calls, RTMP/HLSLive for broadcasting.
3. Set up signaling (SIP, WebSocket, MQTT) and TURN server.
4. Implement publishing/viewing via native API or cross-platform plugin.
5. Test on real devices with different cameras and network conditions.
6. Optimize bitrate and resolution based on bandwidth.

What is Included in the Work

Development Process for Media Functionality

Complexity is non-linear: basic video playback — 1–2 days, custom camera with frame processing and streaming — 3–5 weeks. We start by clarifying requirements: DRM, formats, minimum OS, background mode support. Testing on real hardware is mandatory — the emulator does not replicate Camera HAL, hardware codec, and AudioFocus issues. Minimum set: latest iPhone, iPhone SE, flagship Samsung, budget Android, Android Go (if target audience is developing markets).

Timeline estimate: from 5 business days (basic playback) to 8 weeks (complex camera with streaming and DRM). Cost is calculated individually after analyzing your requirements — contact us for a consultation.

Our service: "Mobile Media Integration" — this is our expertise. Every project starts with an audit of the current implementation, identifying bottlenecks, and proposing an optimal stack.

Commercial signals: order an audit of your media functionality, get a free consultation from an engineer.