AR Cosmetics Try-On Development: Virtual Makeup Turnkey

AR Cosmetics Try-On Development: Virtual Makeup Turnkey

Virtual makeup is technically the most demanding face AR scenario. Lipstick must precisely follow lip contours, eyeshadow must lay on eyelid creases, and foundation must blend with the actual skin tone. The user instantly notices if the boundary 'drifts' when the head turns 20°. That's why most cosmetics manufacturers don't build this stack themselves — they turn to us for specialized SDK integration. We've delivered 15+ AR try-on projects for cosmetic brands.

Why ARKit/ARCore Aren't Enough for Cosmetics

ARKit ARFaceAnchor provides a 1220-vertex face mesh (geometry.vertices) and blendshapes (blendShapes) for expressions. That's sufficient for masks and AR filters. For lipstick, it's not: the lip contour in the ARKit face mesh is about 80 vertices, resulting in jagged geometry during rendering, especially noticeable on thin lips.

Specialized SDKs work with 400–900 vertices just for lips, track upper and lower eyelids separately, the nose bridge, and cheek areas. This yields a realism that standard tools cannot achieve.

How to Choose an SDK: Popular Solutions Compared

Perfect Corp YouCam SDK is the industry standard. It uses its own ML-based face mesh, 120+ effects out of the box, native iOS (Swift/ObjC) and Android (Kotlin/Java) SDKs. We are a certified Perfect Corp integrator, granting access to premium effects.

Banuba Face AR SDK is a flexible alternative with customizable effects via the Banuba Effect Player. It allows you to create effects in the editor. Supports iOS, Android, and Web. An Enterprise license provides full access.

Custom implementation via MediaPipe Face Mesh: 468 points, works on iOS and Android through MediaPipe Tasks. Free and open source. It requires building a custom cosmetics renderer — 3–5 months of work — and significantly lags behind specialized SDKs in realism. However, it's suitable for budget projects.

How We Achieve Realistic Rendering

Lipstick. Lip mesh → UV unwrapping → texture with lipstick color, blended using multiply/screen blend modes with the original camera image. Opacity controls the 'saturation' of the shade. Edges are feathered via Gaussian blur on the mask. With 400+ lip vertices, the contour remains smooth even when the head is turned 60°.

Eyeshadow. Gradient texture applied over the eyelid, accounting for surface curvature. The main challenge is that when the user looks down, the eyelid closes and the shadow must deform accordingly. Without proper deformation, the shadow 'peels off' the skin. Our algorithm uses 30 blendshapes for just the eye area.

Foundation / Concealer. Not just an overlay — we use skin tone matching. The algorithm analyzes skin pixels outside the application area, determines the undertone (warm/cool/neutral), and adjusts the product color to match the real skin tone. Without this, foundation looks like a mask. Our optimization achieves matching in 2–3 ms on modern devices.

Live Camera vs Photo Mode

For live preview via camera, we require 30 fps minimum; otherwise, latency makes the try-on uncomfortable. Perfect Corp SDK maintains 30 fps on iPhone X+, 25 fps on Android with Snapdragon 845+. On weaker Android devices, performance degrades — plan a fallback to photo mode.

Photo mode: the user selects a photo from the gallery and gets a static makeup render. Technically simpler (no real-time requirement), the result is more accurate due to more processing time.

What's Included in Our Work

We provide a full service cycle:

• Requirement analysis and optimal SDK selection for your budget and timeline
• SDK integration into a native app (iOS/Android) or cross-platform project (Flutter/React Native)
• UI for product selection from the catalog, effect configuration, saving and sharing results
• Testing on 10+ devices (including low-end Android)
• Operational and support documentation
• Assistance with App Store and Google Play moderation (considering AR app requirements, App Store Review Guidelines)
• Training for your team (1 to 5 people)

Estimated Timelines

Custom implementation on MediaPipe starts from 12 weeks. Pricing is determined individually.

Common Mistakes We Often See

• Ignoring shadow deformation when facial expression changes — effects peel off
• Lack of skin tone matching for foundation — mask instead of makeup
• Choosing an overly complex SDK for a simple project: e.g., MediaPipe for a single lipstick color is overkill
• Skimping on testing: the effect works great on flagships but fails on mid-range devices
• Publishing without considering Section 5.1.1 App Tracking Transparency — risk of rejection

Contact us for a consultation: we'll help you choose an SDK, plan development, and avoid typical issues. We'll evaluate your project within 2–3 days.

Our experience: 7+ years in mobile development, 50+ AR projects, certified by Perfect Corp and Banuba. Order your AR try-on turnkey — get a ready solution in 5–8 weeks.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.