Mobile VR App Development for 360-Degree Video

Mobile VR App Development for 360-Degree Video

Building a mobile VR player for 360-degree video goes beyond a standard video player. The equirectangular projection must be mapped onto a sphere around the user, synchronized with head movement, and stitched without artifacts—all at a frame rate that prevents discomfort. Mistakes in rendering lead to motion sickness. Loading a 10-minute 4K video via CDN can be expensive at peak loads. According to the MPEG-OMAF specification, viewport-dependent transmission reduces bitrate by 3–5 times—an approach that performs 5 times better than full-streaming. Typical CDN savings range from $2000 to $10000 per month depending on traffic.

We use proven technologies: Unity LTS, Swift, Kotlin, and adaptive streaming protocols. With 7+ years of experience and 15+ VR projects, we guarantee quality and performance. Contact us for a consultation.

Technical Side: Sphere Mapping

Equirectangular video (2:1 aspect ratio, standard for YouTube and Facebook 360) is mapped onto an inverted sphere—the user looks from inside. Comfortable viewing requires at least 4K (3840×2160), preferably 5.7K or 8K. At 4K, each eye gets ~30 pixels per degree—the threshold below which pixelation becomes visible.

On Unity, we create an inverted sphere with inward-pointing normals:

// Standard Unity Sphere + special material
// Or via package com.unity.xr.management

// Shader for 360 video
// vert: pass UV as is
// frag: sample _MainTex with UV, horizontal flip for correct orientation

On native Android using MediaPlayer + OpenGL ES:

// Create Surface for MediaPlayer, render as texture on sphere
SurfaceTexture surfaceTexture = new SurfaceTexture(textureId);
Surface surface = new Surface(surfaceTexture);
mediaPlayer.setSurface(surface);
mediaPlayer.prepareAsync();

On iOS — AVPlayer + SCNSphere in SceneKit or RealityKit:

let sphere = SCNSphere(radius: 10)
sphere.firstMaterial?.isDoubleSided = true // or inverted normals
sphere.firstMaterial?.diffuse.contents = avPlayer
let sphereNode = SCNNode(geometry: sphere)
sphereNode.scale = SCNVector3(-1, 1, 1) // flip X for correct direction
sceneView.scene.rootNode.addChildNode(sphereNode)

Stereoscopic 360: Top-Bottom vs Side-by-Side

Two main formats exist for stereoscopic 360: top-bottom (TB) and side-by-side (SBS). In TB, the top half of the frame is for the left eye, bottom for the right—resulting in a 1:1 aspect ratio and simpler shader implementation. However, vertical resolution per eye is halved. SBS splits the frame vertically—left eye left, right eye right—aspect ratio 4:1. This format distorts less at low bitrates but requires a more complex shader. For most projects, we recommend TB due to better compatibility and simpler integration.

Streaming Playback: HLS/DASH for 360

360 video files are 2–8 GB for 10 minutes of content at 4K–8K. Full preloading is impractical. The solution is adaptive streaming.

For 360 HLS, segments must be prepared with spherical projection metadata—ideally using Google's Spatial Media spec, which embeds projection type directly into the file. Use ffmpeg with the --spherical flag when creating the manifest.

Adaptive bitrate switching is critical: when turning your head, the whole sphere is visible, but the highest load is in the gaze direction. Viewport-dependent streaming (or tile-based streaming) delivers high resolution only for the current viewport. This cuts traffic by 3–5 times, leading to significant cost savings. It is implemented via MPEG-OMAF or a custom DASH server with viewport information.

Spatial Audio

360 video without positional audio is only half the experience. Ambisonics (B-format or AmbiX) is a spatial format where sound automatically orients to head direction.

On Android — AndroidMediaPlayer + Resonance Audio SDK from Google (included in Google Cardboard SDK). On iOS — AVAudioEngine with AVAudioEnvironmentNode for spatial positioning of sources.

Unity: package com.google.resonance-audio or built-in Unity Spatial Audio with Ambisonics support in Audio Settings.

Caching and Offline

Users want to watch 360 tours without the internet. Preloading: background DownloadManager (Android) / URLSessionDownloadTask (iOS), storing HLS segments locally. For a catalog of tours, use SQLite with metadata (preview frame, duration, description) and paths to local files.

Why High-Quality 360 Rendering Matters

Rendering quality directly affects VR immersion. Below 4K, the eye perceives a pixel grid—breaking the sense of presence. Our engineers optimize shaders for each device: on iOS we use Metal API, on Android—Vulkan. This yields up to 40% performance gain over standard OpenGL ES. We also integrate adaptive texture compression (ASTC/ETC2) and multisampling for anti-aliasing. Result: smooth playback even on three-year-old devices.

How We Deliver a Seamless VR Experience

Our development covers all aspects: from rendering to input/output. We guarantee:

• Adaptive streaming with bitrate switching based on connection speed. We use HLS and DASH with MPEG-OMAF support.
• Viewport-dependent streaming—high resolution only in the gaze direction, reducing traffic by 3–5 times.
• Spatial audio via Resonance Audio (Android) and AVAudioEnvironmentNode (iOS)—sound rotates with head movement.
• Offline mode—preload HLS segments to device with SQLite caching.

What’s Included in the Work

• Architectural documentation (component descriptions, flow diagrams)
• CI/CD setup for automatic builds and store submission
• Integration with App Store Connect and Google Play Console (including certificates and provisioning profiles)
• One month of post-release support (bug fixing, crash monitoring via Firebase Crashlytics)
• Training for your team on using the player (2–3 sessions)

Process

1. Analysis — study content, choose format (mono/stereo), target devices.
2. Design — player architecture, SDK and tool selection.
3. Development — rendering, streaming, spatial audio, caching.
4. Testing — on real devices (Cardboard, Google Daydream, Gear VR) with latency and motion sickness measurement.
5. Deploy — publish to stores, set up crash reporting (Firebase Crashlytics).

Timeline Estimates

Basic player for monoscopic 360 with local playback: from 1 week. Full-featured player with streaming, spatial audio, stereoscopic format, and offline: from 3 to 6 weeks. Cost is calculated individually after analyzing your content and requirements. Development starts at $10,000 for a basic player. Order VR app development—and your users will see 360 content with no compromises. Get a consultation for a project assessment.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.