AR Plane Detection in Mobile Apps: Implementation & Optimization

Implementing AR Plane Detection in a Mobile App

We often encounter this scenario: a client wants to place a virtual sofa in a room, but the app shows a "floating" object without clear alignment to the floor. The user is disappointed, and the developer hits the limitations of platform SDKs. Stable plane detection is key to convincing AR. Without it, any AR object feels foreign. We have learned to bring plane detection to a "looks like it belongs" state within 5-8 days.

Platform APIs and Their Real Limitations

ARKit (iOS). ARWorldTrackingConfiguration with planeDetection: [.horizontal, .vertical]. ARKit returns ARPlaneAnchor with ARPlaneGeometry—a plane mesh that updates as scanning progresses. The problem: in the first seconds, ARKit returns a small rectangle that aggressively changes size and orientation. If you place an object immediately, it will "jump" on the next update.

The solution is a minimum confidence threshold and debounce on updates. ARPlaneAnchor does not have an explicit confidence field, but the plane size (extent) serves as an indirect indicator of maturity: do not show the placement UI until extent.x < 0.3 and extent.z < 0.3 meters.

ARCore (Android). Plane with TrackingState.TRACKING and PlaneType.HORIZONTAL_UPWARD_FACING / VERTICAL. ARCore additionally provides Plane.getSubsumedBy()—when two planes merge into one. This breaks the logic if anchors were attached to the original planes; you need to transfer the Anchor to the subsuming plane.

Vertical planes. ARKit reliably detects vertical planes on textured surfaces (patterned wallpaper works well, monotonous white walls do not). ARCore with vertical detection is even less confident. For products where wall mounting is critical (pictures, shelves), it is better to combine plane detection with LiDAR (iPhone 12 Pro+) to fill in missing geometry.

How LiDAR Changes the Game?

On devices with LiDAR (iPhone 12 Pro, 13 Pro, 14 Pro, 15 Pro, iPad Pro), ARKit builds a dense mesh of the environment through ARMeshAnchor. Plane detection with LiDAR works fundamentally differently: planes are derived from the mesh, not from visual SLAM. This yields:

• Plane detection in 1-2 seconds instead of 5-10
• Stable boundaries even on uniform surfaces
• Correct detection of steps, ramps, and sloped planes

For applications where LiDAR devices are the primary target audience (professional photography, renovation, construction), switching between ARWorldTrackingConfiguration and a configuration with sceneReconstruction: .mesh provides a significant quality leap.

Why Does the Plane "Jump" and How to Fix It?

The main reason is placing the object before the plane stabilizes. Confidence threshold and debounce solve 80% of the issue. Our proven approach:

1. Do not show the placement button until the plane size exceeds 0.3 meters on each axis.
2. When the plane updates, smoothly move the anchor with animation (e.g., SCNAction in SceneKit or UIView.animate in RealityKit).
3. Use ARWorldTrackingConfiguration.isAutoFocusEnabled to improve frame quality.

Visualizing Scanning Progress

The user does not know they need to "sweep" the camera—clear guidance is needed. Typical patterns:

• Animated scanning ray from the bottom of the screen
• Plane outline that "grows" as detection progresses
• Text instructions that auto-hide after the first successful detection

For drawing plane boundaries in RealityKit—ModelEntity with wireframe material attached to PlaneAnchor. In SceneKit—SCNNode with SCNGeometry from ARPlaneGeometry.boundaryVertices. In ARCore with SceneKit/Filament—a custom mesh from Plane.getPolygon().

Typical Production Problems

Plane "breaks" with excessive camera motion—tracking state transitions to LIMITED(.excessiveMotion). Block object placement and show a warning, do not crash.

On dark surfaces (dark laminate, black carpet), ARKit and ARCore lose features for visual SLAM. Warning via ARCamera.TrackingState.Reason.insufficientFeatures—must be handled and communicated to the user.

What's Included in the Work

• Testing on target devices with stability evaluation
• LiDAR optimization (if supported)
• Multi-plane selection (ability to select a plane by tapping)
• ARWorldMap persistence for scene reuse
• Integration and configuration documentation
• Guarantee of stable operation: 30 days of free support after delivery

Visualization Methods Comparison

Apple ARKit documentation: ARKit Plane Detection Google ARCore documentation: Plane Detection

Estimated Timeline

Basic plane detection with visual hints and object placement—5 to 8 days. Adding LiDAR support, multi-plane selection, and ARWorldMap persistence—another 5-7 days. Cost is calculated individually, depending on complexity. Get a consultation—contact us to evaluate your project.

Our team has 5+ years of AR development experience and over 20 successful projects. Order a turnkey implementation and get a free consultation.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.