Building a Virtual Fitting Room: AR Try-On Implementation

Building a Virtual Fitting Room: AR Try-On Implementation

We develop AR try-on for mobile applications — from simple accessories to full clothing with mesh deformation. The core problem we solve: realism. Without accurate pose tracking, scaling to user parameters, and physically correct lighting, the try-on becomes a "cardboard" collage and fails to boost conversion.

Our team integrates native frameworks — ARKit (iOS) and ARCore (Android) — as well as alternative SDKs to accelerate time-to-market. Each product category requires a specific technology: glasses need face tracking, clothing needs body tracking with mesh deformation, shoes require foot tracking or plane detection, and accessories need hand tracking.

The term "try-on" is an umbrella that breaks down into fundamentally different technical tasks. For each product category, a dedicated approach is needed: glasses and jewelry — face tracking (ARKit/ARCore), clothing — body tracking with mesh deformation, shoes — foot tracking or plane detection, hand accessories — hand tracking. The table below summarizes categories, technologies, and complexity estimates.

The most complex category is clothing. Without correct mesh deformation based on body pose, the clothing looks like a cardboard silhouette over the person.

How to Achieve Realistic Mesh Deformation for Clothing?

iOS: ARBodyTrackingConfiguration (A12+, iOS 13+) provides a 91-point skeleton in world coordinates. ARSkeleton3D with jointModelTransforms gives matrices for each joint. A skinned mesh of clothing is overlaid on the skeleton: each vertex is bound to 1-4 joints with skinning weights. When a joint moves, vertices follow according to the weights.

The skinned mesh format is USDZ with a SkinningComponent in RealityKit. Mesh preparation in Blender with rigging to the standard ARKit skeleton is a critical step. If the artist is not familiar with ARKit joint hierarchy, the rib cage and spine will deform incorrectly. More details can be found in Apple documentation.

Android: ARCore does not provide body tracking out of the box. Options include MediaPipe Pose (Google), BlazePose (33 keypoints), MoveNet (Google). These are not ARKit-level accuracy, but sufficient for a marketing try-on. Mesh deformation is handled via custom OpenGL/Vulkan shaders or TensorFlow Lite with pose estimation.

Why Lighting Is Critical for AR Try-On?

PBR material with correct roughness/metallic values plus ARKit environment probe (AREnvironmentProbeAnchor) or automaticEnvironmentTexturing is the minimum for realistic appearance. Without an environment map, the shiny leather of a bag looks identical to the matte fabric of a jacket.

Shadows from virtual clothing onto the real body require an occluder mesh: an invisible body model casts shadows onto the real floor/walls. In RealityKit, use the .occlusion material on an occluder entity. This greatly enhances realism but adds complexity (an accurate 3D body model of the user is needed).

Comparison of Third-Party SDKs

For a quick launch, ready-made SDKs can be used, but for a premium experience, a custom implementation with full control over deformation and lighting is better. Custom solutions typically outperform SDKs by 30-40% in user engagement and conversion.

Scaling to the User

Glasses are tried on without entering parameters — face tracking keeps them precisely on the nose. Clothing is different. The basic approach uses standard sizes S/M/L with mesh scaling based on the skeleton's shoulder bounding box. Accurate fitting with chest/waist circumference requires user input or body scanning (a separate task with even higher complexity).

What Our Work Includes

We have been carrying out AR projects for over 5 years, with 20+ integrations for fashion retail. Each project undergoes code review and load testing before release. Our 5+ years of certified AR expertise guarantee a seamless implementation.

Timelines and Costs

Accessory try-on (glasses, jewelry) using face tracking — 2-3 weeks, from $5,000. Clothing with body tracking and basic deformation — 6-10 weeks, from $20,000. Third-party SDK integration (Perfect Corp, Snap Camera Kit) — 1-2 weeks plus license. The cost is calculated individually after analyzing your product catalog.

If you need a realistic AR try-on, contact us — we will assess your catalog and propose the best turnkey solution. Get a consultation today.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.