VPS integration for precise AR on iOS and Android

GPS provides 3–5 meters accuracy. For AR experiences in an urban environment, this is catastrophically insufficient: overlaying navigation arrows on a specific building entrance or showing an AR annotation above a particular sculpture in a museum courtyard with 5-meter error is impossible. We implement Visual Positioning System (VPS) for mobile AR applications turnkey — from scanning the space to SDK integration. With VPS, the user points the camera at the surroundings, the algorithm matches the frame against a pre-recorded visual map, and returns a position with 10–30 centimeter accuracy. We will evaluate your project and select the optimal solution.

How VPS determines position

Two stages: mapping (offline) and localization (online, real-time).

Mapping. A room or outdoor area is scanned: an operator with a phone or specialized rig walks the entire area, recording video. Keyframes are extracted from the video, a Structure from Motion (SfM) algorithm runs — building a sparse point cloud and a set of 6DOF camera poses. Feature point descriptors (ORB, SuperPoint + SuperGlue for better accuracy) are indexed in a database for fast retrieval.

Localization. The phone captures a frame, sends it to the server (or processes locally on powerful devices). An image retrieval algorithm finds the nearest keyframes from the database → PnP (Perspective-n-Point) computes the camera pose → returns a 6DOF transform. All within 200–500 ms with server-side processing. For offline localization on the device, frames are recorded into a local database, increasing privacy.

How to build a visual map for VPS

1. Shoot the space with at least 60% frame overlap, uniform lighting, and full coverage of user paths.
2. Upload the video to the Immersal or other provider's server.
3. SfM processing: 30–120 minutes, after which you receive a mapId.
4. Accuracy verification: test localization at control points.

If the interior changes — rescan the changed areas and merge with the existing map via the provider's console. We guarantee that the final map will provide stable localization even with partial camera occlusion.

Which SDK to choose for VPS

For custom indoor spaces (museum, office, warehouse), Immersal is best — it provides 10x better accuracy indoors than Apple ARKit + GPS. For urban AR experiences on iOS and Android — ARCore Geospatial API or Niantic VPS. Azure Spatial Anchors is good when integrating with the client's Azure infrastructure.

How to integrate Immersal into a native application

Immersal provides a REST API for cloud localization and a Unity SDK — the latter we are not interested in; we work natively.

iOS: HTTP request to https://api.immersal.com/localize with JPEG frame and camera intrinsics → JSON response with pose in the map coordinate system → convert to ARKit world space via matrix transformation.

struct LocalizeRequest: Encodable {
    let token: String
    let fx, fy, ox, oy: Double  // camera intrinsics from ARCamera.intrinsics
    let image: String            // base64 JPEG
}
// Get mapToWorld matrix, apply to ARSession.currentFrame

Android: similarly via retrofit2 + moshi, camera intrinsics from CameraCharacteristics.

Do not run localization on every frame (200 ms latency) — trigger on position change of 1+ meter or when ARCore tracking is lost.

What's included in the work

• Technical specification with provider selection and architecture.
• Scanning and building visual map for indoor spaces.
• SDK integration into native iOS and/or Android application.
• API documentation and map update instructions.
• Accuracy and stability testing on target devices.
• One month post-launch support.

Timelines and cost

Integration of ARCore Geospatial for urban AR experience: from 2 to 4 weeks. Custom VPS with Immersal for indoor space including scanning: from 4 to 8 weeks. Custom VPS server based on HLoc (hloc + SuperPoint + SuperGlue + Colmap) without external dependencies: 3–5 months. Cost is calculated individually. Get a consultation — we will provide a preliminary estimate. Order VPS integration into your application.

We develop AR solutions for retail, museums, and industry. Contact us to discuss your task and select the optimal VPS solution.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.