AI Virtual Makeup Try-On for Mobile Apps

The Real Challenge: Perfect Lipstick Application on Camera

Your users point their front camera, expecting to see how that new lipstick looks on them — in real time, with millimeter precision. Built-in camera filters won't cut it. Competitors like Sephora Virtual Artist, Perfect Corp YouCam Makeup, and MAC Virtual Try-On already use this tech, but we push further. With over 10 years of experience and 50+ AR projects, we match professional beauty brands. Our virtual makeup try-on solution uses AR makeup and AI makeup technologies for mobile makeup app development, delivering a realistic virtual cosmetics trial and contactless makeup try-on.

Technically, the solution combines two approaches: geometric (face mesh) and pixel (face segmentation). The first provides rough contours; the second gives a mask for each element — allowing makeup to stay within natural boundaries.

Why Segmentation Beats Face Mesh for Makeup

Face mesh gives coarse outlines, but precise makeup needs a pixel mask. We use CoreML and MediaPipe Selfie Segmentation for lips, eyelids, and skin. On A15 Bionic, inference takes 8-15 ms; on older devices, we run every 3 frames with interpolation. This ensures accuracy even during fast movements. Guarantee: lipstick never bleeds outside lip contours. Our MediaPipe makeup technology provides 478 points including iris — ideal for eye makeup.

Color Accuracy Across Devices

We define product colors in Lab color space — perceptually uniform. When rendering, we convert to sRGB considering the device's display profile. To calibrate, we lock auto white balance via AVCaptureDevice.whiteBalanceGains. This ensures the shade matches the real product. Our team is Apple and Google certified for AR technologies.

Face Mesh Fundamentals

Face mesh technology builds a wireframe of the face. On iOS, ARFaceTrackingConfiguration uses the TrueDepth camera to generate a mesh of 1220 vertices (via ARFaceAnchor with geometry and blend shapes). UV-mapping textures onto the mesh is standard. On Android, ML Kit Face Mesh provides 468 points via MediaPipe FaceMesh (works on RGB camera). We use it via C++ native.

Rendering Makeup Over the Face

The key challenge is realism. We use PBR materials: for lipstick metallic=0.3, roughness=0.2; for eyeshadow metallic=0.0, roughness=0.8. Lighting adapts to ARLightEstimate. Physically correct rendering increases user engagement by 2–3 times according to our data.

Our Process

• Analysis: Define product list (lipstick, eyeshadow, foundation), accuracy and performance requirements.
• Design: Create UV makeup textures, tune PBR parameters per product.
• Development: Integrate face mesh, renderer, segmentation, color correction, video recording.
• Testing: On 20+ devices including iPhone 8 and 2018 Android — under load.
• Deployment: Publish to App Store and Google Play (StoreKit 2, Billing 6). Train your team.

Approach Comparison

ARKit provides 2–3 times more accurate makeup overlay due to depth camera, but MediaPipe is suitable for budget devices. For iOS ARKit makeup and Android ML Kit makeup, we optimize separately.

Execution Time Comparison

Timing on flagship devices (iPhone 15, Galaxy S23). On older devices, latency can rise to 100 ms, but system compensates with frame interpolation.

What's Included

• API integration and architecture documentation
• Source code in Swift (iOS) and Kotlin (Android)
• Brand book adaptation (colors, textures, logo)
• Post-release support (2 months incident support)
• Performance optimization for target devices
• Integration with your product catalog

Timelines and Cost

MVP with lipstick and eyeshadow on iOS via ARKit — 6–9 weeks. Starting from $15,000. Cross-platform version with full catalog — 4–7 months, starting from $40,000. Cost is determined individually. Contact us to evaluate your project. Request an MVP development quote and we'll prepare a detailed commercial proposal.