Complete Guide to Video Overlay Integration for Mobile Streaming
We regularly encounter a situation where a client wants to overlay a logo on an iPhone stream but uses a regular UIView on top of AVCaptureVideoPreviewLayer. The result: the logo is not captured in the recording and is invisible to viewers. The issue is that UIKit draws in the display pipeline, but we need to modify the CVPixelBuffer before it reaches the video encoder. We solve this with Metal, embedding the overlay directly into the graphics pipeline. Below is the technical implementation for iOS and Android, with performance analysis and typical pitfalls.
An overlay on a mobile stream is not just a visual element but part of the user experience. If it's missing from the recording, viewers won't see the brand or critical information. Technically correct integration requires deep understanding of the platform's video pipeline.
How to Choose Between Metal and CoreImage for Mobile Streaming Video Overlay
For mobile streaming video overlay, the capture pipeline needs to modify the pixel buffer before encoding. The two approaches are CoreImage (simpler but slower) and Metal (faster but more complex). Below is a performance comparison.
| Parameter | Metal | CoreImage |
|---|---|---|
| Performance | <1ms per frame (GPU) | 4–6ms per frame (CPU) on iPhone 12+ |
| Animation support | Full (Metal shaders) | Limited (CIFilter) |
| Implementation complexity | High | Medium |
| Recommended for | 1080p60 without drops | 720p30 or prototypes |
Metal is 4–6x faster than CoreImage for overlay rendering. For production streams at 1080p and 30+ FPS, we use only Metal. This approach reduces per-frame overhead by 5x, translating to cost savings of $2,000–$4,000 compared to CPU-based implementations. A simple static logo overlay costs $3,000 on iOS.
How Does the Metal Pipeline Work for Logo Overlay?
We create an MTLTexture from the CVPixelBuffer via CVMetalTextureCacheCreateTextureFromImage, render the overlay on top through a Metal render pass, and the result stays in the same pixel buffer. Code:
Example overlay implementation in Swift
```swift class OverlayRenderer { private let device: MTLDevice private let commandQueue: MTLCommandQueue private var textureCache: CVMetalTextureCache? private var overlayTexture: MTLTexture?func apply(to pixelBuffer: CVPixelBuffer) -> CVPixelBuffer { var cvTexture: CVMetalTexture? CVMetalTextureCacheCreateTextureFromImage( nil, textureCache!, pixelBuffer, nil, .bgra8Unorm, CVPixelBufferGetWidth(pixelBuffer), CVPixelBufferGetHeight(pixelBuffer), 0, &cvTexture ) guard let texture = CVMetalTextureGetTexture(cvTexture!) else { return pixelBuffer } let commandBuffer = commandQueue.makeCommandBuffer()! // render pass: main texture + overlayTexture on top // ... commandBuffer.commit() commandBuffer.waitUntilCompleted() return pixelBuffer } }
</details>
The logo (overlayTexture) is loaded once at session start from a PNG with alpha channel. Do not load UIImage every frame — that adds 2–3ms per call.
## How to Implement Text Overlay Without FPS Loss?
Static text (channel name) is simply a Metal texture prepared via CoreText. Dynamic text (viewer count, donation messages) cannot be rendered directly by Metal because Metal doesn't draw text. The solution: create an offscreen CALayer with CATextLayer, render it into a UIGraphicsImageRenderer, get a UIImage, convert to MTLTexture. Do this on a background thread at most once every 500ms for counters and event-driven for donation messages.
Fade-in/fade-out for donation text is implemented by changing the alpha channel of the MTLTexture between frames — smooth appearance over 15–20 frames (0.5–0.7 seconds).
## Step-by-Step Implementation Guide
1. **Set up Metal device and command queue**: Initialize MTLDevice and MTLCommandQueue.
2. **Create texture cache**: Use CVMetalTextureCacheCreate to link CVPixelBuffer with Metal textures.
3. **Load overlay image**: Convert PNG to MTLTexture with alpha.
4. **Render pass**: In AVCaptureVideoDataOutputSampleBufferDelegate, get the pixel buffer, apply overlay via Metal render pass.
5. **Commit and encode**: The modified buffer goes to video encoder.
## Android Overlay: OpenGL ES vs Vulkan
The Android equivalent is SurfaceTexture + OpenGL ES 2.0. The camera renders into a SurfaceTexture, we overlay using GLES20.glBlendFunc, and the result goes to MediaCodec via a Surface. Vulkan is more powerful but requires Android 7+ and more boilerplate. We recommend OpenGL ES for compatibility, Vulkan for complex effects. Comparison:
| Parameter | OpenGL ES 2.0 | Vulkan |
|----------|---------------|--------|
| Minimum Android version | 2.2 (API 8) | 7.0 (API 24) |
| Performance | Sufficient for 1080p30 | Higher but overkill |
| Implementation complexity | Medium | High |
| Recommended for | Most projects | Special effects |
OpenGL ES is 2–3x simpler to implement than Vulkan while providing sufficient performance for most mobile streaming video overlay use cases. For specific video text overlay on Android, OpenGL ES provides adequate performance.
Overlay position is stored in relative coordinates (0.0–1.0 of frame size), which works correctly when resolution or orientation changes. For landscape rotation, the overlayRect of the render pass is recalculated automatically.
In Metal development, we always prioritize GPU-based solutions to avoid CPU bottlenecks. Similarly, in iOS streaming and Android streaming, overlay integration must be done at the buffer level.
## What's Included in Our Work: Deliverables
We provide a turnkey solution including:
- Code and architecture documentation
- Repository access with CI/CD pipeline
- Your team training on working with the code
- 3 months of technical support
- A separate maintenance contract upon request
## Timelines and Cost
Static logo + Metal pipeline on iOS: 1–1.5 weeks ($3,000–$5,000). Dynamic text, overlay animations, iOS + Android: 3–4 weeks ($8,000–$12,000). Cost is calculated individually after project evaluation. Development cost reduction — up to 40% compared to non-optimized solutions. We have 10+ years of experience in mobile development and 50+ successful streaming projects. We guarantee quality and deadline adherence.
For more details about the technology, see the official [Metal](https://developer.apple.com/documentation/metal) and [OpenGL ES](https://www.khronos.org/opengles/) documentation. Contact us for a project discussion. Request a consultation on optimizing your streaming pipeline — we will help integrate overlays without performance loss.







