We integrate screen sharing into iOS and Android apps using ReplayKit and MediaProjection APIs. With over 50 projects delivered and certified Apple and Google engineers, we ensure stable production-ready solutions. This article covers architecture, frame transmission, and common pitfalls.
How Does ReplayKit Work on iOS?
On iOS, screen capture is only possible via ReplayKit. You cannot get the screen content directly from the main app process—this is a sandbox restriction. For real-time broadcasting, we use RPSystemBroadcastPickerView, which shows a system picker to launch the broadcast. The broadcast runs via a Broadcast Upload Extension—a separate target in the project that runs in its own process (com.apple.broadcast-services-upload). The extension and the main app have no shared memory—communication is only through App Group (shared UserDefaults, shared FileManager, or Darwin notify for signals).
Structure of the extension:
class SampleHandler: RPBroadcastSampleHandler { override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) { switch sampleBufferType { case .video: // Pass CMSampleBuffer to WebRTC/Agora/Livekit videoSource?.capturer(capturer, didCapture: toRTCVideoFrame(sampleBuffer)) case .audioApp: // System audio case .audioMic: // Microphone (only available with explicit permission in Info.plist) } } } The extension has a memory limit of 50 MB—in 90% of cases, crashes are due to exceeding this limit. Agora and Livekit provide lightweight versions of their SDK specifically for Broadcast Extension. Communication from extension to main app is via CFNotificationCenter (Darwin notifications). Resolution and FPS are limited by ReplayKit: max 1080p, up to 60 FPS. In practice, 720p at 15 FPS is sufficient for screen sharing—mobile device screens are small, and bitrate drops by 40% without loss of text readability.
Apple Developer Documentation: ReplayKit
What is the MediaProjection API on Android?
On Android, screen capture uses the MediaProjection API. The user must explicitly allow recording:
val mediaProjectionManager = getSystemService(MEDIA_PROJECTION_SERVICE) as MediaProjectionManager startActivityForResult( mediaProjectionManager.createScreenCaptureIntent(), REQUEST_MEDIA_PROJECTION ) In onActivityResult, you get an Intent with permission—from it you create a MediaProjection. VirtualDisplay with MediaProjection.createVirtualDisplay() provides a Surface onto which the system draws the screen content.
For transmission via WebRTC, we use ScreenCapturerAndroid (from org.webrtc): it wraps MediaProjection and delivers frames to VideoSource. Since Android 10, when starting screen capture via ForegroundService, you need android:foregroundServiceType="mediaProjection" in the manifest. On Android 14, additionally you must explicitly specify ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION when starting the service via startForeground(). Without this—SecurityException. When the connection is broken, MediaProjection.Callback.onStop() is called—must be handled and the user notified.
Android Developers: MediaProjection
How Are Frames Transmitted in Real Time?
On both iOS and Android, captured screen frames are CMSampleBuffer (iOS) or Bitmap/Image via ImageReader (Android). For network transmission over WebRTC, we convert to RTCVideoFrame with YUV420PlanarBuffer (iOS) or JavaI420Buffer (Android). YUV conversion from BGRA is an expensive operation—we perform it in native code (C++) or via a hardware converter. For Agora, AgoraRtcKit.startScreenCapture() accepts configuration with contentHint: .text to optimize the codec for textual content (less motion, high sharpness). WebRTC is more flexible than Agora in bitrate configuration, but Agora is easier to integrate with Broadcast Extension. For high-load projects (1000+ simultaneous streams), WebRTC offers 30% lower infrastructure costs compared to Agora.
Comparison of iOS and Android
| Parameter | iOS (ReplayKit) | Android (MediaProjection) |
|---|---|---|
| Minimum version | iOS 12+ | Android 5.0 (API 21) |
| Screen capture | Broadcast Extension (separate process) | Direct access via Intent |
| Memory limit | 50 MB on extension | No hard limit |
| Max resolution | 1080p | Depends on device, often 1080p+ |
| FPS | Up to 60 | Up to 30 (typically) |
| Audio capture | System + microphone | Microphone via AudioRecord |
| Foreground service | Not required | Mandatory (Android 10+) |
Typical Screen Sharing Issues
| Issue | Solution |
|---|---|
| Broadcast Extension crash on iOS | Use lightweight SDK (Agora/Livekit) and monitor the 50 MB limit |
| SecurityException on Android 14 | Explicitly set foregroundServiceType="mediaProjection" and call startForeground() with the correct type |
| User stops capture via Android notification shade | Handle onStop() and correctly switch UI without errors |
| Delay during YUV conversion | Use hardware converter or native C++ code |
Technical Details of Broadcast Extension
Broadcast Extension lives in a separate process with a 50 MB limit. If the SDK weighs more—the extension crashes without a clear message. Agora and Livekit provide "lite" versions for extensions. For data exchange between the extension and the main app, we use Darwin notifications via CFNotificationCenter. The extension cannot establish network connections directly—it only delivers frames, while the main app manages the WebRTC peer.
What Is Included in Screen Sharing Implementation?
Our deliverables include:
- Architecture and design: stack selection (WebRTC/Agora/Livekit), frame exchange scheme, lifecycle handling.
- Development of Broadcast Extension (iOS) and MediaProjection Service (Android): full capture and broadcasting cycle.
- Integration with signaling and WebRTC: peer connection configuration, ICE, relay setup.
- Permission management: screen recording requests, ATT, foreground service.
- Error handling and edge cases: extension crash, user cancels capture, network loss.
- Documentation and training: integration description, commented source code transfer, one training session.
- Testing: on real devices with 10+ different OS versions.
- Support: 3 months of post-deployment support and maintenance.
How to Choose the Stack for Screen Sharing?
Choosing between WebRTC, Agora, and Livekit depends on customization needs and time to market. WebRTC gives full control over bitrate and codec but requires more integration time—on average 1.5 weeks for both platforms. Agora and Livekit reduce this to 5 days thanks to ready modules, but limit configuration flexibility. For high-load projects (1000+ simultaneous broadcasts), WebRTC is 30% cheaper in infrastructure costs.
Estimated Timelines and Costs
iOS (ReplayKit + Broadcast Extension + WebRTC) — from 3 to 5 days if signaling is ready. Android (MediaProjection + WebRTC) — from 2 to 3 days. Both platforms with correct lifecycle, background mode, and UX notifications — from 1 to 1.5 weeks. Typical project cost ranges from $5,000 to $15,000 depending on complexity and stack. Contact us for a detailed quote—our certified engineers guarantee seamless integration and 100% pass through app store reviews.