AI Virtual Makeup Try-On for Mobile Apps

The Real Challenge: Perfect Lipstick Application on Camera

Your users point their front camera, expecting to see how that new lipstick looks on them — in real time, with millimeter precision. Built-in camera filters won't cut it. Competitors like Sephora Virtual Artist, Perfect Corp YouCam Makeup, and MAC Virtual Try-On already use this tech, but we push further. With over 10 years of experience and 50+ AR projects, we match professional beauty brands. Our virtual makeup try-on solution uses AR makeup and AI makeup technologies for mobile makeup app development, delivering a realistic virtual cosmetics trial and contactless makeup try-on.

Technically, the solution combines two approaches: geometric (face mesh) and pixel (face segmentation). The first provides rough contours; the second gives a mask for each element — allowing makeup to stay within natural boundaries.

Why Segmentation Beats Face Mesh for Makeup

Face mesh gives coarse outlines, but precise makeup needs a pixel mask. We use CoreML and MediaPipe Selfie Segmentation for lips, eyelids, and skin. On A15 Bionic, inference takes 8-15 ms; on older devices, we run every 3 frames with interpolation. This ensures accuracy even during fast movements. Guarantee: lipstick never bleeds outside lip contours. Our MediaPipe makeup technology provides 478 points including iris — ideal for eye makeup.

Color Accuracy Across Devices

We define product colors in Lab color space — perceptually uniform. When rendering, we convert to sRGB considering the device's display profile. To calibrate, we lock auto white balance via AVCaptureDevice.whiteBalanceGains. This ensures the shade matches the real product. Our team is Apple and Google certified for AR technologies.

Face Mesh Fundamentals

Face mesh technology builds a wireframe of the face. On iOS, ARFaceTrackingConfiguration uses the TrueDepth camera to generate a mesh of 1220 vertices (via ARFaceAnchor with geometry and blend shapes). UV-mapping textures onto the mesh is standard. On Android, ML Kit Face Mesh provides 468 points via MediaPipe FaceMesh (works on RGB camera). We use it via C++ native.

Rendering Makeup Over the Face

The key challenge is realism. We use PBR materials: for lipstick metallic=0.3, roughness=0.2; for eyeshadow metallic=0.0, roughness=0.8. Lighting adapts to ARLightEstimate. Physically correct rendering increases user engagement by 2–3 times according to our data.

Our Process

• Analysis: Define product list (lipstick, eyeshadow, foundation), accuracy and performance requirements.
• Design: Create UV makeup textures, tune PBR parameters per product.
• Development: Integrate face mesh, renderer, segmentation, color correction, video recording.
• Testing: On 20+ devices including iPhone 8 and 2018 Android — under load.
• Deployment: Publish to App Store and Google Play (StoreKit 2, Billing 6). Train your team.

Approach Comparison

ARKit provides 2–3 times more accurate makeup overlay due to depth camera, but MediaPipe is suitable for budget devices. For iOS ARKit makeup and Android ML Kit makeup, we optimize separately.

Execution Time Comparison

Timing on flagship devices (iPhone 15, Galaxy S23). On older devices, latency can rise to 100 ms, but system compensates with frame interpolation.

What's Included

• API integration and architecture documentation
• Source code in Swift (iOS) and Kotlin (Android)
• Brand book adaptation (colors, textures, logo)
• Post-release support (2 months incident support)
• Performance optimization for target devices
• Integration with your product catalog

Timelines and Cost

MVP with lipstick and eyeshadow on iOS via ARKit — 6–9 weeks. Starting from $15,000. Cross-platform version with full catalog — 4–7 months, starting from $40,000. Cost is determined individually. Contact us to evaluate your project. Request an MVP development quote and we'll prepare a detailed commercial proposal.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.