AI System for Beauty and Cosmetics Industry
Standard segmentation into 4–6 skin types doesn't work: skin condition changes with age, season, and hormonal cycle. Customers get disappointed with "personalized" recommendations when a cream isn't suitable. We build AI systems that solve this problem — from diagnostics to virtual try-on, with measurable improvements in conversion and loyalty.
Our experience: 5+ years in Computer Vision and ML, with over 20 completed projects for retailers and cosmetics manufacturers. Each solution is embedded into existing infrastructure: cloud, on-premise, or hybrid. Below is a breakdown of key components.
Personalized skincare is built on multimodal ML models: skin photos, questionnaires, purchase history. Instead of segmentation — a continuous skin state space. Virtual try-on — via neural rendering with GAN and WebGL. Skin analysis, formula stability prediction, quality control — complete the ecosystem. We guarantee quality at every stage: intermediate metrics, A/B tests, reports. MLOps practices based on Weights & Biases and MLflow ensure experiment reproducibility. Contact us for a consultation — we'll discuss your project and provide an estimate within 2 days.
How AI Personalizes Skincare?
The problem with classic systems is division into 4–6 types. Skin type is not static: age, climate, stress change it. ML approach combines multiple data channels:
- Questionnaire (sensations, problems, preferences)
- Skin photos (oiliness, pores, texture — via CV)
- Purchase history and reactions
- Context (climate, season, age)
Architecture: multi-modal encoder — ViT for photos, dense encoding for questionnaire, collaborative filtering for history. Concatenation → MLP predicts affinity score to each product. In a pilot with a beauty retailer (120k SKUs, 80k users), precision@10 = 0.41 vs 0.27 for the skin-type engine — 1.5 times higher. Lift in basket size: +14%, which for an average retailer means additional revenue of up to 1.5 million rubles per month.
Why Virtual Try-On Requires Neural Rendering?
Virtual Try-On is technically the most complex component. Simple texture overlay with alpha blending does not account for lighting and face geometry.
Foundation: MediaPipe FaceMesh provides 478 3D landmarks in real-time on CPU. Zone segmentation (lips, eyelids, eyebrows) — via BiSeNet-V2 (≈5ms on GPU).
Realistic results require:
- Lighting estimation (SphericalHarmonics) — correction for scene lighting
- Geometry-aware blending — color deformation based on lip curvature
- Specular highlights — for glossy products (Phong model)
Advanced systems use BeautyGAN: conditional generation with shade as an embedding solves class imbalance. GAN rendering is 5x more realistic than texture overlay, while WebGL + MediaPipe is 10x faster in latency (20ms vs 200ms), critical for mobile devices.
| Approach | Latency | Quality | Applicability |
|---|---|---|---|
| Texture overlay (alpha blend) | <5ms | Low | Prototypes |
| WebGL + MediaPipe | <20ms | Medium | Browser, mobile |
| GAN rendering | ~200ms | High | E-commerce, showcases |
| Neural Rendering | >500ms | Very High | Presets, photo |
Skin Analysis from Photos: Details
Pipeline:
- Face detection (MTCNN or RetinaFace)
- Face alignment via landmarks
- Region-specific classification:
- Oiliness/dryness: LBP + CNN
- Pores: high-res crop → anomaly detection
- Wrinkles: edge detection + density scoring
- Hyperpigmentation: CIE Lab + blob detection
- Acne: object detection (YOLO, Faster R-CNN)
Public datasets are scarce — we use FFHQ + internal dermatologist labeling. On 5000 photos macro-F1 = 0.79. MLOps pipeline based on Weights & Biases tracks experiments and model versions. Important: the system includes a disclaimer — this is not medical diagnostics.
Formula Development and Stability Prediction
Surrogate models: ML predicts viscosity, SPF, emulsion stability from INCI composition. INCI encoded via one-hot + physicochemical descriptors. Ingredient compatibility prediction — GNN on ingredient–ingredient graph (AUC-ROC 0.84). Accelerated stability testing: predict 12-month stability from 3-month data (viscosity change RMSE ≈3.1%).
Example model setup for formulation
We use BoTorch (Bayesian Optimization) for recipe tuning: objective function — stability + sensory. Iteratively suggest 5–10 formulations for testing; the model refines. Savings: up to 60% on lab testing costs, hundreds of thousands of rubles per year.Quality Control and Packaging Defects
Computer vision on the conveyor: check bottle fill levels, label defects, cap integrity. YOLOv8 on typical defects; few-shot learning (Siamese network) on rare ones. Throughput 800+ units/min on NVIDIA Jetson AGX Orin.
What's Included
- Data and process audit (2 days)
- ML architecture design
- Model development and training (CV, recommendations, formulation)
- Infrastructure integration (API, backend)
- Deployment (cloud, on-premise)
- MLOps support: CI/CD for models, drift monitoring
- Documentation, team training, 3-month support
Get in touch — receive a consultation on your project. We'll evaluate in 2 days, provide a roadmap and budget.
Timelines and Investment
Timelines — from 6 weeks to 14 months depending on scope. Investment — from 500,000 to 2,000,000 rubles for an average project. Precise estimate after audit.
Tech Stack
Task Tools Face landmarks / AR MediaPipe FaceMesh, WebGL Skin analysis EfficientNet, BiSeNet-V2, OpenCV Virtual try-on BeautyGAN, PSGAN, Three.js Recommendations Two-Tower model, LightGBM Formulation BoTorch, GP, chemprop Quality control YOLOv8, EfficientDet, NVIDIA Jetson MLOps Weights & Biases, MLflow, Kubeflow We guarantee quality: at every stage — intermediate metrics, A/B tests, reports. Request a consultation — discuss your scenario.