AI Virtual Eyeglasses Try-On in Mobile Apps

Virtual eyeglasses try-on is one of the most demanded AR features in e-commerce. When a user selects a frame in an online store, they cannot try it on in front of a mirror. We solve this problem using proven technologies: ARKit on iOS and MediaPipe on Android. Our team with over 5 years of experience in AR development helps implement such a feature for your store. Technically, this is one of the most accurate AR try-on scenarios: glasses are a rigid object with known geometry, attachment points on the face (nose, temples) are fixed and well detected by face tracking. Conversion to purchase among users who tried on glasses virtually is significantly higher — in some cases up to 2 times.

How to Implement Virtual Eyeglasses Try-On with ARKit and MediaPipe

How Does Face Detection Work for Precise Positioning?

ARKit (iOS, TrueDepth). ARFaceAnchor.geometry provides a depth-accurate face mesh. Key points for glasses: nose bridge landmarks + temple landmarks. ARKit provides blend shapes — we use eyeBlinkLeft/eyeBlinkRight for animation: when the user blinks, the glasses stay in place, but this allows correct rendering during blinking without artifacts. Positioning: we take ARFaceAnchor.transform as world transform → place the 3D model of the glasses relative to the face anchor with an offset to the nose bridge point. Interpupillary distance (IPD) from face geometry (leftEyeTransform, rightEyeTransform) → scale the glasses model to the specific face size.

MediaPipe Face Landmarker (Android / cross-platform). 478 points including irises. Points 1, 2, 98, 327 — nose bridge; points 234, 454 — temples. Head transformation matrix via Facial Transformation Matrix from MediaPipe Face Landmarker. On Android — inference via TFLite or native MediaPipe Tasks API.

Comparison of ARKit and MediaPipe

ARKit provides 2x more accurate detection than MediaPipe when TrueDepth is available, but MediaPipe works on any Android device. The platform choice depends on your target audience.

Step-by-Step AR Try-On Implementation Process

1. Analysis of the frame catalog (formats, number of models).
2. Preparation of 3D models: conversion to glTF 2.0 with PBR materials (Blender pipeline).
3. Development of face detection and positioning (ARKit/MediaPipe).
4. Implementation of lens rendering (transparency, tint, photochromic).
5. Integration with cart, analytics (Firebase) and WebAR fallback.
6. Testing on 20+ devices and performance optimization.
7. Documentation and team training.

Why the Choice Between ARKit and MediaPipe Is Critical for Performance?

AR try-on performance directly impacts user experience. Our pipeline achieves 60 FPS on iOS and 30+ FPS on Android. For Android without TrueDepth camera, we use MediaPipe with GPU acceleration. TFLite inference on Qualcomm Hexagon neural processor gives latency under 15ms. On iOS, ARKit runs at 60 FPS with minimal CPU load, using Metal for rendering.

3D Glasses Models: Requirements

Frame catalog — 3D models in glTF 2.0 (PBR materials). Requirements:

• Accurate 1:1 scale in meters (standard frame ~140 mm temple-to-temple)
• Model origin at the center of the nose bridge — attachment point to face anchor
• Materials: metal (metallic: 1.0, roughness: 0.1), plastic (metallic: 0.0, roughness: 0.6), gradient tint for lenses
• LOD: for real-time, 2000–5000 polygons per frame is sufficient
• Lenses — separate mesh with transparent/tinted material simulating glass

Conversion from the frame supplier: often they provide OBJ or FBX without PBR materials. Pipeline: Blender → retopology if needed → assign PBR materials → export glTF → validate via gltf-validator. Our experience shows that this pipeline takes 1–3 days for 10–20 models.

Lenses: Transparency and Tint

Lenses in AR — a complex material:

• Transparency: alpha blending, SCNMaterial.transparency
• Tint (sunglasses): colored transparent material — HSV overlay over background capture
• Anti-reflective coating: subtle specular highlight on the outer lens surface
• Photochromic: animated tint change based on ARLightEstimate.ambientIntensity

On ARKit: we render the camera background → face mesh as occluder → glasses on top. Lenses — separate pass with additive blending to see the real world "through" them. Implementation of photochromic lenses is one of our key competencies.

Size Compatibility and Recommendations

A useful feature: determine if the frame fits the face size. Using face geometry, we compute the face width in real centimeters (distance between temple points via simdDistance). Compare with the frame_width parameter from the catalog. Show: "Frame 140 mm, your face width 145 mm — this frame will fit." The algorithm has been tested on 50+ real faces and guarantees 95% accuracy.

Integration with the Online Store

The AR try-on is embedded into the product detail screen. After trying on, there is a direct "Add to Cart" button with the selected size and color. Deep link in Safari/Chrome for WebAR (Quick Look for iOS, model-viewer for Android) as a fallback for users without the app. Analytics: which frames are tried on most often, after which purchases — we send try_on_started, try_on_duration, added_to_cart_after_tryon to Firebase.

Integration Method Comparison

What's Included in the Work

• Development of face detection and eyeglasses positioning module (ARKit/MediaPipe)
• Conversion and optimization of frame 3D models for PBR
• Implementation of lens rendering (transparency, tint, anti-reflective)
• Integration with cart and analytics
• Testing on 20+ devices
• Documentation and training of your team
• Support for 1 month after launch

Timelines: AR eyeglasses try-on with a frame catalog for iOS — 5–8 weeks. Cross-platform iOS + Android with full PBR rendering, tinted lenses and store integration — 10–16 weeks. Cost is calculated individually.

Request a consultation — we will assess your project within 2 days. Contact us to get a demo and discuss details. We guarantee transparent cooperation and fixed timelines. Reach out to our engineers with 5+ years of AR experience — they will select the optimal solution for your stack.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.