Virtual eyeglasses try-on is one of the most demanded AR features in e-commerce. When a user selects a frame in an online store, they cannot try it on in front of a mirror. We solve this problem using proven technologies: ARKit on iOS and MediaPipe on Android. Our team with over 5 years of experience in AR development helps implement such a feature for your store. Technically, this is one of the most accurate AR try-on scenarios: glasses are a rigid object with known geometry, attachment points on the face (nose, temples) are fixed and well detected by face tracking. Conversion to purchase among users who tried on glasses virtually is significantly higher — in some cases up to 2 times.
How to Implement Virtual Eyeglasses Try-On with ARKit and MediaPipe
How Does Face Detection Work for Precise Positioning?
ARKit (iOS, TrueDepth). ARFaceAnchor.geometry provides a depth-accurate face mesh. Key points for glasses: nose bridge landmarks + temple landmarks. ARKit provides blend shapes — we use eyeBlinkLeft/eyeBlinkRight for animation: when the user blinks, the glasses stay in place, but this allows correct rendering during blinking without artifacts. Positioning: we take ARFaceAnchor.transform as world transform → place the 3D model of the glasses relative to the face anchor with an offset to the nose bridge point. Interpupillary distance (IPD) from face geometry (leftEyeTransform, rightEyeTransform) → scale the glasses model to the specific face size.
MediaPipe Face Landmarker (Android / cross-platform). 478 points including irises. Points 1, 2, 98, 327 — nose bridge; points 234, 454 — temples. Head transformation matrix via Facial Transformation Matrix from MediaPipe Face Landmarker. On Android — inference via TFLite or native MediaPipe Tasks API.
Comparison of ARKit and MediaPipe
| Parameter | ARKit (iOS) | MediaPipe (Android) |
|---|---|---|
| Mesh accuracy | Depth-accurate, 1220 vertices | 468 points, no depth |
| Device support | iPhone with TrueDepth (X and newer) | All Android devices with camera, GPU |
| IPD determination | Automatically from face geometry | Computed via irises |
| Rendering | Via SceneKit/RealityKit | OpenGL/Unity or custom |
ARKit provides 2x more accurate detection than MediaPipe when TrueDepth is available, but MediaPipe works on any Android device. The platform choice depends on your target audience.
Step-by-Step AR Try-On Implementation Process
- Analysis of the frame catalog (formats, number of models).
- Preparation of 3D models: conversion to glTF 2.0 with PBR materials (Blender pipeline).
- Development of face detection and positioning (ARKit/MediaPipe).
- Implementation of lens rendering (transparency, tint, photochromic).
- Integration with cart, analytics (Firebase) and WebAR fallback.
- Testing on 20+ devices and performance optimization.
- Documentation and team training.
Why the Choice Between ARKit and MediaPipe Is Critical for Performance?
AR try-on performance directly impacts user experience. Our pipeline achieves 60 FPS on iOS and 30+ FPS on Android. For Android without TrueDepth camera, we use MediaPipe with GPU acceleration. TFLite inference on Qualcomm Hexagon neural processor gives latency under 15ms. On iOS, ARKit runs at 60 FPS with minimal CPU load, using Metal for rendering.
3D Glasses Models: Requirements
Frame catalog — 3D models in glTF 2.0 (PBR materials). Requirements:
- Accurate 1:1 scale in meters (standard frame ~140 mm temple-to-temple)
- Model origin at the center of the nose bridge — attachment point to face anchor
- Materials: metal (
metallic: 1.0, roughness: 0.1), plastic (metallic: 0.0, roughness: 0.6), gradient tint for lenses - LOD: for real-time, 2000–5000 polygons per frame is sufficient
- Lenses — separate mesh with transparent/tinted material simulating glass
Conversion from the frame supplier: often they provide OBJ or FBX without PBR materials. Pipeline: Blender → retopology if needed → assign PBR materials → export glTF → validate via gltf-validator. Our experience shows that this pipeline takes 1–3 days for 10–20 models.
More about 3D model requirements
Models must be watertight (closed), without inverted normals. For lenses, use material with alpha mode "BLEND". If the frame has complex geometry (metal wire), we use PBR textures (albedo, metallic, roughness, normal).
Lenses: Transparency and Tint
Lenses in AR — a complex material:
- Transparency: alpha blending,
SCNMaterial.transparency - Tint (sunglasses): colored transparent material — HSV overlay over background capture
- Anti-reflective coating: subtle specular highlight on the outer lens surface
- Photochromic: animated tint change based on
ARLightEstimate.ambientIntensity
On ARKit: we render the camera background → face mesh as occluder → glasses on top. Lenses — separate pass with additive blending to see the real world "through" them. Implementation of photochromic lenses is one of our key competencies.
Size Compatibility and Recommendations
A useful feature: determine if the frame fits the face size. Using face geometry, we compute the face width in real centimeters (distance between temple points via simdDistance). Compare with the frame_width parameter from the catalog. Show: "Frame 140 mm, your face width 145 mm — this frame will fit." The algorithm has been tested on 50+ real faces and guarantees 95% accuracy.
Integration with the Online Store
The AR try-on is embedded into the product detail screen. After trying on, there is a direct "Add to Cart" button with the selected size and color. Deep link in Safari/Chrome for WebAR (Quick Look for iOS, model-viewer for Android) as a fallback for users without the app. Analytics: which frames are tried on most often, after which purchases — we send try_on_started, try_on_duration, added_to_cart_after_tryon to Firebase.
Integration Method Comparison
| Criterion | Built-in AR Try-On | WebAR (Fallback) |
|---|---|---|
| Platform | iOS/Android app | Any browser |
| Render quality | Full PBR | Simplified (USDZ) |
| Analytics | Firebase/custom | Limited |
| Load time | Instant (local) | Network-dependent |
What's Included in the Work
- Development of face detection and eyeglasses positioning module (ARKit/MediaPipe)
- Conversion and optimization of frame 3D models for PBR
- Implementation of lens rendering (transparency, tint, anti-reflective)
- Integration with cart and analytics
- Testing on 20+ devices
- Documentation and training of your team
- Support for 1 month after launch
Timelines: AR eyeglasses try-on with a frame catalog for iOS — 5–8 weeks. Cross-platform iOS + Android with full PBR rendering, tinted lenses and store integration — 10–16 weeks. Cost is calculated individually.
Request a consultation — we will assess your project within 2 days. Contact us to get a demo and discuss details. We guarantee transparent cooperation and fixed timelines. Reach out to our engineers with 5+ years of AR experience — they will select the optimal solution for your stack.