AR Multiplayer: Synchronizing Space Across iOS and Android

AR Multiplayer: Synchronizing Space Across iOS and Android

Two people in the same room see the same AR object from different angles. Sounds simple. In practice, it's synchronizing the AR spaces of two devices, each living in its own coordinate system, plus network latency, plus positioning drift. Without a proper architecture, the game figure for the first player will be a meter away from where the second player sees it. Our experience shows: most problems are solved by choosing the right synchronization stack — from simple peer-to-peer to server-authoritative architecture.

How to Synchronize AR Spaces Across Platforms?

ARKit — Collaborative Session. Since iOS 13, ARKit supports ARCollaborationData — binary packets with feature points and anchor information that devices exchange directly. Via MultipeerConnectivity (Bonjour, peer-to-peer Wi‑Fi/Bluetooth) or via a custom relay server. Each device builds a shared spatial map, objects on ARAnchor synchronize automatically.

// Отправляем collaboration data другим участникам
func session(_ session: ARSession, didOutputCollaborationData data: ARSession.CollaborationData) {
    let encoded = try? NSKeyedArchiver.archivedData(withRootObject: data,
                                                     requiringSecureCoding: true)
    sendToPeers(encoded)
}

Limitation: works only iOS‑iOS, no Android. For cross‑platform you need a different solution. Source: ARKit documentation

ARCore — Cloud Anchors. Google provides server infrastructure: ARCore Cloud Anchors — hostCloudAnchor() uploads the feature map of an anchor to the cloud, returns a cloudAnchorId. The second user calls resolveCloudAnchor(cloudAnchorId) — gets the same anchor in their space. Through it we synchronize positions of all shared objects. Works on iOS (ARCore SDK for iOS) ↔ Android. Cloud Anchors are up to 3x more reliable for cross-platform synchronization than custom peer-to-peer solutions. Using Cloud Anchors reduces server infrastructure costs by up to 40%, potentially saving approximately $2,000 per month on server bills for mid-sized projects.

Unity + Photon + ARDK (Niantic Lightship). For game development projects, it's often better to take Niantic Lightship ARDK — native support for shared AR spaces, multiplayer via Photon, cross‑platform. But it's Unity, not native development.

With MultipeerConnectivity latency is 10–50 ms, sufficient for most AR games in the same room. A relay server adds 50–200 ms. ARKit Collaborative Session offers 2x lower latency than Cloud Anchors for local iOS-only scenarios.

Why We Choose Server‑Authoritative Physics?

If AR objects have physics (a ball rolling, cubes falling) — you need authoritative physics simulation. For 2–4 players, the server‑authoritative approach increases server load by 20–30% but eliminates desync. Server-authoritative simulation is 2x more consistent than client-side prediction in tests. Options:

• Server‑authoritative: server (headless) simulates physics, clients receive states and interpolate. Eliminates cheating, requires powerful server.
• Client prediction + rollback: client predicts result locally, server confirms or rolls back. Harder to implement, but better UX.

For casual AR games, client‑side physics with periodic synchronization via state snapshots is sufficient.

How to Avoid Typical AR Multiplayer Mistakes?

Applying interpolation reduces visible latency by 50–100 ms.

What's Included in the Work

• Architectural documentation (choice of synchronization stack, network scheme)
• Code repository (Swift/Kotlin/Flutter/Unity)
• Integration of ARKit/ARCore and network module
• CI/CD pipeline setup for testing on real devices
• Deployment guide for App Store / Google Play
• Team training (1–2 days)
• Technical support for 1 month after release

Process

Implementation goes through 5 stages:

1. Analysis — identify use cases, number of players, platforms, latency requirements.
2. Design — choose stack (ARKit vs ARCore vs Unity), synchronization architecture (state sync vs delta sync), protocol (WebSocket/WebRTC).
3. Implementation — write code, integrate AR layer and network layer, set up interpolation and dead reckoning.
4. Testing — on real devices in real space. CI/CD with Xcode Cloud / Firebase Test Lab, but final QA physically.
5. Deployment — publish to app stores, monitor crash reports and metrics.

Timelines and Cost

• Basic shared AR with Cloud Anchors for two players — 4–7 weeks, starting at $5,000.
• Multiplayer AR game with server synchronization, multiple rooms, and physics — 3–6 months, $30,000–$100,000.
• Cost includes full development and testing. Get a consultation — we'll evaluate your project in one working day.

Order AR multiplayer development right now — our engineers will evaluate your project in one day. Contact us to discuss your project. Our engineers have 10+ years of experience in mobile development and have completed 20+ AR projects. Guarantee stable synchronization and on‑time delivery.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.