AI Hairstyle Try-On: Segmentation, Coloring, Generation

A client walks into a salon, shows a Pinterest haircut photo, and within 10 seconds sees themselves with that hairstyle on screen. No 'shall we try?', no test dyes. Virtual hairstyle try-on is one of the hardest computer vision problems. Hair has semi-transparent edges, thin strands down to 1 pixel, and any mask inaccuracy gives a 'plastic' look. Over 15+ projects for beauty-tech, we have refined a pipeline combining alpha matte segmentation, diffusion generation, and identity preservation. The result: realistic try-on of any hairstyle, be it color change or drastic shape change. Contact us to discuss your project — we'll help you find the optimal solution.

How hair segmentation works

Accurate hair mask is the foundation of everything. The problem: the hair/background boundary is blurry, semi-transparent, with individual strands 1–3 pixels wide. Standard semantic segmentation gives coarse edges that kill realism when overlaying.

Specialized models:

• Hair-SAM / Matting Anything — segmentation with alpha matte: not a binary mask but float32 values 0..1 per pixel. On edges — fractional values representing transparency.
• MODNet — lightweight matting model, 35fps on mobile. Optimized specifically for portraits. 3x faster than classical methods (e.g., DIM).
• ViTMatte — transformer-based matting, higher quality on complex strands. According to the ViTMatte paper, it outperforms predecessors on Intersection over Union.

from matting import load_model, estimate_foreground_ml

model = load_model('vitmate_vit_b_pretrained.pth')

def segment_hair(image, trimap):
    # trimap: 0=definitely background, 128=border zone, 255=definitely hair
    alpha, foreground = model(image, trimap)
    return alpha  # float32 mask, 0..1

Why alpha matte is critical

Binary masks do not convey semi-transparency — resulting in sharp steps on edges. Alpha matte gives per-pixel transparency, which is crucial for overlaying hairstyles: the result looks natural, not like a sticker.

Overlaying the new hairstyle

Two approaches:

Texture transfer — take a 3D model or 2D reference of the target hairstyle, deform it to the user's head shape (face shape analysis via 3DMM), overlay with blending. Works for similar lengths and shapes. Problem: if the user has long straight hair and the target is a short bob — the original hair must be removed under the new hairstyle, requiring scalp inpainting.

Generative approach (diffusion models) — more powerful:

1. Segment hair and scalp
2. Describe the desired hairstyle in text: "short bob, brown, with bangs"
3. Run inpainting through Stable Diffusion with ControlNet (conditioning on face shape and head position)
4. Preserve the face (face preservation via IP-Adapter or InstantID)

The main challenge: preserve the person's identity when changing hairstyle. Without face preservation, the model may generate a different person with the desired hairstyle.

Why generative approach is better

The generative approach enables drastic shape changes, unlike texture transfer which is limited by original length. We use ViTMatte for matting and Stable Diffusion for inpainting — this allows creating hairstyles the person never had, with identity preservation.

Hair coloring

Simpler than shape change: change the color within the segmented hair mask. Tools:

• Selective color transfer in LAB color space: preserve L-channel (texture), change AB (color)
• Neural color transfer via AdaIN for complex effects (ombre, balayage, highlights)
• Balayage/ombre: gradient mask + separate color transfer by gradient from roots to tips

import cv2
import numpy as np

def recolor_hair(image, hair_mask, target_color_lab):
    lab = cv2.cvtColor(image, cv2.COLOR_BGR2LAB).astype(float)
    # Calculate average hair color
    hair_pixels = lab[hair_mask > 128]
    mean_ab = hair_pixels[:, 1:].mean(axis=0)
    # Shift AB channels toward target color
    shift = target_color_lab[1:] - mean_ab
    lab[:, :, 1:] += hair_mask[..., None] / 255.0 * shift
    lab = np.clip(lab, 0, 255).astype(np.uint8)
    return cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)

Real-time vs photo mode

Real-time (video): MODNet for segmentation + texture-based recoloring. Limitation: complex shape changes are not realistic in real time.

Photo mode: full pipeline with ViTMatte + diffusion generation. Time: 3–15 seconds depending on method.

What's included in the work

Estimated turnaround for turnkey: 5–18 weeks depending on complexity. Cost is calculated individually. Time savings compared to developing from scratch — 2–3 months, allowing faster product launch. Get a consultation: contact us to evaluate your project.

We guarantee matting accuracy, identity preservation, and fast integration. Experience: 15+ projects in beauty-tech. For real-time hair try-on we adapt the model to your requirements. Alpha matte hair — our expertise. Identity preservation — mandatory stage. Request a timeline estimate for your project — it takes no more than a day.