Audiobooks are challenging content: 15-hour files, unstable connections, users demanding precise position resumption. A typical bug is the player losing progress when switching between Wi-Fi and LTE, and the sleep timer cutting off a chapter without fade-out. We solved this at the architecture level: aggressive caching via URLSession background tasks, position synchronization with cloud storage every 5 seconds, and smooth volume fade before stopping. Result: the app doesn't crash, positions are saved, users don't complain.
File Formats and Streaming Playback
Audiobooks come mostly in MP3 (widest), M4B (AAC with chapter markers, Apple standard), M4A, and OGG/Opus. M4B is preferred for new catalogs—it natively contains chapters, cover art, and metadata without a sidecar.
Streaming playback (no full download before play): on iOS, AVPlayer opens an HTTP URL and starts playing without full load. On Android, ExoPlayer (Media3) does the same via DefaultDataSource.Factory. For long files (15 hours ≈ 800 MB), this is critical.
Chapters from M4B: AVAsset.loadChapterMetadataGroups(bestMatchingPreferredLanguages:) returns AVTimedMetadataGroup[] with timestamps and titles. On Android, use MediaMetadataRetriever.extractMetadata plus parsing MP4 box structure for chapter atoms, or use ExoPlayer with a custom MetadataRetriever.
Why M4B Is Best for New Catalogs
M4B uses the MP4 container with built-in chapter markers, simplifying navigation and metadata storage. Unlike MP3, no external chapter file is needed. During streaming, the client immediately gets the book structure.
Precise Position Saving
A user listens in the car, closes the app—next opening must resume exactly at that spot, to the second.
We save currentTime (Double, seconds) plus bookId and timestamp to UserDefaults/SharedPreferences on every background transition (applicationDidEnterBackground) and via a Timer every 5 seconds during playback. The timer runs every 5 seconds, not every second—that's overkill.
On restore: player.seek(to: CMTime(seconds: savedPosition, preferredTimescale: 1000)) with .seconds precision. Not MSEC—audio doesn't need it.
Multi-device sync: CloudKit CKRecord (iOS) or Firebase Firestore with userId/bookId key. When opening the book on a new device, we offer "Continue from 1:23:45"—the user decides.
How to Ensure Seamless Sync?
We use cloud storage with a "last write wins" conflict resolution. On each position save, we send a request to Firestore/CloudKit with a timestamp. When loading the book on another device, we compare timestamps and pick the latest. Additionally, we keep a local cache for offline scenarios.
Sleep Timer
A popular feature: playback stops after N minutes. Trivial: DispatchQueue.main.asyncAfter (iOS) or Handler.postDelayed (Android). Advanced variant: "stop at end of current chapter"—we get the endTime of the current chapter from metadata and schedule the stop at that time.
Smooth fade-out before stop: 30 seconds before the end, we linearly decrease player.volume to 0 via CADisplayLink / ValueAnimator. No abrupt stop—that annoys users.
Technical details of smooth fade-out
We use a timer at 60 FPS (CADisplayLink on iOS, Choreographer on Android). Each update reduces volume by (1/60)*(1/30) ≈ 0.00056. On reaching the stop time, we pause the player and restore volume to the original for subsequent playback.
Playback Speed and Pitch Correction
player.rate = 1.5 (AVPlayer) speeds up playback but raises pitch—the voice becomes squeaky. iOS automatically applies pitch correction for AVPlayer when rate ≠ 1.0 since iOS 16. On older versions, use AVAudioUnitTimePitch in an AVAudioEngine pipeline.
Android ExoPlayer: playbackParameters = PlaybackParameters(speed = 1.5f) — pitch correction built-in. Flutter just_audio: player.setSpeed(1.5) with default pitchCorrectionMethod.
Useful speed range: 0.75x (for complex technical content), 1.0x, 1.25x, 1.5x, 2.0x. Above 2x, intelligibility drops critically.
| Speed |
Use Case |
Pitch Correction |
| 0.75x |
Technical content |
Auto |
| 1.0x |
Standard |
None |
| 1.25x |
Accelerated listening |
Yes |
| 1.5x |
Fast reading |
Yes |
| 2.0x |
Intelligibility limit |
Yes |
"For audio playback, AVPlayer applies pitch correction when the rate is not 1.0." — Apple AVPlayer documentation.
Offline Download and DRM
Offline download uses URLSessionDownloadTask + URLSessionConfiguration.background (iOS): works even when the app is not running, progress survives crashes. Android: WorkManager + DownloadManager or OkHttp with custom progress.
For paid content, DRM is required. FairPlay Streaming (iOS) integrates with AVContentKeySession. Widevine L3 (Android) uses ExoPlayer with DefaultDrmSessionManager. The encryption key is requested from the license server on each playback. Without DRM, users can copy the file from /Library/Application Support and listen without a subscription.
Alternative without full DRM: AES-256 file encryption on disk with a key tied to the account and DeviceID. Cheaper to implement, but less protection.
| DRM |
iOS |
Android |
| FairPlay Streaming |
AVContentKeySession |
— |
| Widevine L3 |
— |
ExoPlayer + DrmSessionManager |
| AES-256 self-managed |
CommonCrypto |
Android Keystore |
Catalog and Search
User library: purchased and downloaded books. Store catalog: search, genres, new releases, recommendations. Pagination via cursor (Alchemy-style next token or offset/limit). Covers via SDWebImage/Coil/cached_network_image with aggressive disk cache (covers change rarely).
What's Included in the Work
- Source code of the app (iOS/Android) with comments.
- API documentation and database schema.
- Access to App Store Connect / Google Play Console.
- Team training for the client (2-3 sessions).
- 3 months of technical support after release.
- Integration with existing backend or setup of a new one.
| Feature |
MVP |
Full Version |
| MP3/M4B playback |
yes |
yes |
| Chapters and navigation |
no |
yes |
| Offline download |
no |
yes |
| Position sync |
no |
yes |
| DRM |
no |
yes |
| Sleep timer |
yes |
yes |
Timeline: MVP player with catalog and basic playback — 3-4 weeks. Full product with DRM, offline, sync, and subscription — 8-12 weeks. Savings on DRM licenses when using self-managed AES-256 can reach 30%.
Want to discuss your project? Contact us for an architecture assessment and optimal solution. Order turnkey audiobook app development.
How to Choose a Camera Approach on Mobile Platforms?
Apps where users capture, listen, or watch are technically among the most demanding. We deal with this every day. Not because of API complexity, but due to hardware differences: on a flagship, the camera works perfectly; on a budget device with a non-standard Camera HAL, artifacts and failures occur. On iOS, stabilization differs between generations. Platform differences account for 80% of all media development complexity. Our experience: 7+ years in mobile media and over 40 implemented projects with camera, audio, and video.
What are the Differences Between CameraX, Camera2, and AVFoundation?
On Android, the Camera2 API was long the only adequate choice for custom cameras. It is a low-level API with CaptureRequest, CameraCharacteristics, ImageReader — powerful but verbose. Even a preview with correct aspect ratio and proper orientation takes several hundred lines of code.
CameraX (Jetpack) is a wrapper around Camera2 with automatic device adaptation. Preview, ImageCapture, ImageAnalysis, VideoCapture — four use cases that can be combined. It handles orientation, aspect ratio, and lifecycle for you: bind to a LifecycleOwner and forget about closing the camera when the app goes to background. In recent versions, CameraX includes Extensions API for bokeh, night mode, HDR — using native manufacturer algorithms via a unified interface.
When is Camera2 needed directly?: RAW capture via ImageFormat.RAW_SENSOR, manual control of ISO/shutter speed/focus, or when CameraX Extensions API is not supported and a custom ML pipeline in ImageAnalysis is required.
On iOS, AVFoundation is the only path for a custom camera. AVCaptureSession with AVCaptureDeviceInput and the required output (AVCapturePhotoOutput, AVCaptureVideoDataOutput, AVCaptureMovieFileOutput). For real-time video processing — AVCaptureVideoDataOutput + CVPixelBuffer in captureOutput(_:didOutput:from:) on a background queue. This is where CoreML models receive frames for inference.
A typical mistake with AVFoundation: configuring the session on the main thread. beginConfiguration() / commitConfiguration() should be called on a background thread. Otherwise, the preview freezes, and the user sees a frozen UI. This mistake appears in 70% of the projects we have audited.
Why is AudioFocus Critical for Android Apps?
Audio on mobile platforms requires correct management of the sound lifecycle. AudioFocus is a coordination mechanism between apps. AudioManager.requestAudioFocus() with OnAudioFocusChangeListener. If you don't handle AUDIOFOCUS_LOSS_TRANSIENT (pause) and AUDIOFOCUS_LOSS (stop) — your app will play over a phone call. That guarantees a bad review on Google Play. Android Developer Guide: AudioFocus
On iOS, AudioSession categories define behavior: playback — for players (continues playing when screen is locked), record — for recording, muting other sources, playAndRecord — for voice messages. Wrong category — the app mutes the user's background music on start.
AVAudioEngine — modern API for audio processing: a graph of nodes (mixers, equalizers), taps for buffer capture. For real-time speech — SFSpeechRecognizer + inputNode.installTap.
On Android for recording with noise suppression — NoiseSuppressor.isAvailable() + create(audioRecord.audioSessionId). Works not on all devices, need a fallback.
Video: Playback and Streaming
ExoPlayer (Media3) — standard for Android. Supports HLS, DASH, SmoothStreaming, progressive playback. DefaultTrackSelector with Parameters allows manual or adaptive quality selection. DRM via DefaultDrmSessionManager with Widevine L1/L3.
Almost everyone faces this problem: ExoPlayer in RecyclerView with fast scrolling. Need a PlayerPool — a pool of reusable players. Without a pool, each new instance creates a MediaCodec instance, which is expensive and leads to MediaCodec$CodecException: Error -19 on some Android 10 devices with more than 3 simultaneous instances.
AVPlayer / AVPlayerViewController on iOS — for playback. For custom UI — AVPlayerLayer + custom controls. HLS works natively via AVPlayer(url:) with m3u8. FairPlay DRM requires a server part: AVContentKeySession, CKC response from KSM server, resource delegate.
For Flutter — video_player as a base layer, chewie for UI. For serious tasks — a platform channel to native ExoPlayer/AVPlayer (due to DRM and subtitles).
| Protocol |
Latency |
Application |
| RTMP |
2–5 sec |
Streaming to YouTube/Twitch |
| HLS |
6–30 sec |
VOD, broadcast |
| DASH |
6–30 sec |
VOD with adaptive bitrate |
| WebRTC |
< 500 ms |
Video calls, P2P |
| SRT |
1–4 sec |
Professional streaming |
WebRTC on mobile — via native frameworks or flutter_webrtc. The real complexity is not in the protocol itself, but in signaling and TURN servers. Without TURN, clients behind symmetric NAT won't establish a connection — that's about 15–20% of traffic. Coturn is the standard open-source server.
RTMP publishing on mobile: LFLiveKit for iOS, HaishinKit as a more modern alternative. On Android — rtmp-rtsp-stream-client-java or via FFmpeg with JNI. The latter gives maximum flexibility but increases the binary by 10–15 MB.
Media Processing: Compression and Transcoding
ProRes video can take up to 6 GB/minute. Compression is needed before upload. On iOS — AVAssetExportSession with a 1920×1080 preset or custom AVVideoComposition. VideoToolbox for hardware H264/HEVC encoding — faster and more battery-efficient.
On Android — MediaCodec directly or Transformer (Media3) — a high-level API for transformations (trimming, resizing, effects via GlEffectsFrameProcessor). For images — BitmapFactory.Options.inSampleSize for downsampling, Glide / Coil for caching. Coil on Coroutines fits well with Compose. Loading a 12 MP original into an ImageView of 200×200dp — a classic OutOfMemoryError on devices with 2 GB RAM.
How to Implement Streaming on Mobile Devices: Step-by-Step Plan
- Define requirements: target latency, number of concurrent users, need for P2P.
- Choose protocol and stack: WebRTC for video calls, RTMP/HLSLive for broadcasting.
- Set up signaling (SIP, WebSocket, MQTT) and TURN server.
- Implement publishing/viewing via native API or cross-platform plugin.
- Test on real devices with different cameras and network conditions.
- Optimize bitrate and resolution based on bandwidth.
Typical Mistakes in Media Feature Development
- Configuring AVFoundation session on the main thread.
- Missing AudioFocus Loss handling on Android.
- Ignoring
MediaCodec limitations on cheap devices.
- Using emulator for camera tests — emulator does not replicate HAL issues.
- Memory leaks when recreating media players without a pool.
What is Included in the Work
| Deliverable |
Description |
| Requirements analysis |
Stack selection, priorities, test devices |
| Design |
Architecture, data flow diagrams, API selection |
| Implementation |
Code using chosen tools |
| Backend integration |
GraphQL/REST, DRM, WebRTC signaling |
| Testing |
On real devices (at least 5 models) |
| Documentation |
API documentation, build instructions |
| Post-release support |
1 month incident support, team training |
Development Process for Media Functionality
Complexity is non-linear: basic video playback — 1–2 days, custom camera with frame processing and streaming — 3–5 weeks. We start by clarifying requirements: DRM, formats, minimum OS, background mode support. Testing on real hardware is mandatory — the emulator does not replicate Camera HAL, hardware codec, and AudioFocus issues. Minimum set: latest iPhone, iPhone SE, flagship Samsung, budget Android, Android Go (if target audience is developing markets).
Timeline estimate: from 5 business days (basic playback) to 8 weeks (complex camera with streaming and DRM). Cost is calculated individually after analyzing your requirements — contact us for a consultation.
Our service: "Mobile Media Integration" — this is our expertise. Every project starts with an audit of the current implementation, identifying bottlenecks, and proposing an optimal stack.
Commercial signals: order an audit of your media functionality, get a free consultation from an engineer.
