Building Multi-User AR on iOS with Collaborative Session

A client asked us to develop an AR app for collaborative interior viewing: two designers with iPads simultaneously placing furniture in a shared virtual space. Without synchronization, each saw a different set of objects, and the real room ended up in different coordinates. The solution — Collaborative Session from ARKit. But implementation turned out to be nontrivial: MultipeerConnectivity periodically lost connection, maps didn't merge, and objects jumped. Below — how we build a reliable multi-device AR session, what pitfalls we avoid, and what to consider before starting development. Development timelines and pricing are determined individually.

Collaborative Session allows multiple iOS devices to exchange spatial maps in real time. Unlike Cloud Anchors, the internet is not required — data is transmitted over the local network with latency under 20 ms. We implemented a project for a design studio where 3 participants synchronized with latency below 15 ms; server cost savings amounted to around $1500 per month. However, this technology only works on Apple devices and is sensitive to network quality.

How Collaborative Session Works

Activating Collaborative Session requires just one line in the configuration:

let config = ARWorldTrackingConfiguration()
config.isCollaborationEnabled = true
arView.session.run(config)

After that, ARKit generates ARSession.CollaborationData packets — information about the device's environment. These need to be sent to all participants via a suitable transport: MultipeerConnectivity, WebSocket, or GameKit.

func session(_ session: ARSession, didOutputCollaborationData data: ARSession.CollaborationData) {
    let encoded = try? NSKeyedArchiver.archivedData(withRootObject: data, requiringSecureCoding: true)
    // send encoded to all peers
}

// On the receiving side:
let data = try? NSKeyedUnarchiver.unarchivedObject(ofClass: ARSession.CollaborationData.self, from: received)
session.update(with: data)

After 10–30 seconds, devices merge their maps — all participants get a unified coordinate system.

Why Collaborative Session Is Faster than Cloud Anchors

Collaborative Session is a direct alternative to Cloud Anchors. Here's a comparison:

For local scenarios (office, exhibition), Collaborative Session is 30–50% faster and requires no external servers. Infrastructure savings reach 50–60%.

How to Ensure Reliable Synchronization

• MultipeerConnectivity is unreliable. MCSession drops peers without clear errors. Solution: a MultipeerConnectivity heartbeat every 2 seconds, automatic reconnection after three missed pings.
• CollaborationData packet size. With active camera movement, traffic reaches 50–100 KB/s per device. For 4 participants — up to 400 KB/s. For 5 devices, it can approach 500 KB/s. Send CollaborationData over .unreliable (ARKit recovers from lost packets). Send critical data (object coordinates) over a separate reliable channel.
• Heartbeat mechanism. Our certified ARKit developers with 7+ years of experience guarantee that MultipeerConnectivity heartbeat ensures stable connections.

What Happens When Maps Merge?

Before merging, each device has its own local coordinate system. Anchors added earlier may jump. Track ARParticipantAnchor:

func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
    let participants = anchors.compactMap { $0 as? ARParticipantAnchor }
    if !participants.isEmpty {
        // Maps merged — can now place objects
        enableObjectPlacement()
    }
}

Place objects only after ARParticipantAnchor appears — otherwise desynchronization is inevitable.

What Data Is Synchronized in Collaborative Session?

CollaborationData contains feature points, depth map, and anchors. Custom objects (e.g., virtual furniture) are not synchronized automatically — their coordinates must be sent separately. We recommend a reliable channel for object positions and .unreliable for CollaborationData.

Our Process: From Analysis to Deployment

1. Analyze use case and network constraints.
2. Design architecture: transport selection, sync scheme.
3. Implement ARWorldTrackingConfiguration and transport layer.
4. Handle session lifecycle: participant discovery, map merging, peer departure.
5. Test on real devices under various network conditions.

What's Included in Our Work

• Collaborative Session architecture with transport selection (MultipeerConnectivity, WebSocket, GameKit).
• Implementation of heartbeat, reconnection, and buffering.
• Synchronization of custom objects via a separate reliable channel.
• Integration with your UI: placement, deletion, movement of objects.
• Documentation and source code access.
• Team training (2 hours online) and post-release support (7 days guarantee).

Timelines

A basic Collaborative Session with MultipeerConnectivity — 2–3 weeks. With full edge‑case handling, state synchronization, and UI — 4–7 weeks. Development cost starts at $5,000 for basic integration. Consultation available upon request.

Common Implementation Mistakes

• Packet loss under unreliable delivery is not critical for CollaborationData — ARKit rebuilds the map.
• Reliable delivery of CollaborationData causes queuing and lag — use .unreliable.
• Placing objects before map merge leads to anchor jitter — wait for ARParticipantAnchor.
• A departing participant without buffer clearing accumulates packets — clear the buffer on disconnect.

We are ready to help with your AR project.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.