AI Face Match for Mobile App Photo Verification
The Problem of KYC Without Face Match
A KYC flow without face-to-document comparison is vulnerable to identity fraud – anyone can use a stolen passport. Face Match closes this gap by comparing a selfie with the document photo and returning a confidence score (0.0–1.0). Technically, it involves extracting a face embedding and computing cosine similarity. But the devil is in the details: document photos often have glare and moiré, selfie lighting is uncontrolled, and aging can differ by years. With years of experience in mobile AI solutions, we guarantee a robust and accurate verification system. Our engineers have tested over 20 models on real data and can achieve FRR under 2% at FAR under 0.1%.
How Face Embedding Comparison Works
A classic pipeline includes face detection, alignment, CNN-based embedding extraction, and cosine similarity. For example:
-
Detection –
Vision.VNDetectFaceRectanglesRequeston iOS, ML KitFaceDetectoron Android. - Alignment – Normalize eye and nose coordinates to a canonical position. Without alignment, accuracy drops by 15–20%.
- Embedding – A CNN model (ArcFace, MobileFaceNet) converts a 112×112 px face into a 512-dimensional vector.
- Cosine similarity – Between two vectors: a value ≥0.65 typically indicates a match, but the threshold depends on the model and target demographics.
Importantly, the threshold is not universal. Different demographic groups have different baselines. A good model is trained on a balanced dataset (MS-Celeb-1M, VGGFace2) and validated on LFW and AgeDB with demographic breakdown. A model without such validation poses a risk of discrimination and false rejections for older users.
On-Device vs. Server Verification
The choice between on-device and server depends on accuracy, speed, and privacy requirements. Below is a comparison of key parameters:
| Parameter | On-Device (MobileFaceNet) | Server (ArcFace R100) |
|---|---|---|
| Model size | 1.1 MB | 250–300 MB |
| Accuracy on LFW | 99.2% | 99.6% |
| Inference time | 25–180 ms | 150–300 ms |
| Privacy | Data stays on device | Embedding sent to server |
| Audit trail | Limited | Full logging |
| Best for | Non-critical services, high speed | Financial sector, regulatory requirements |
MobileFaceNet (1.1 MB) is 250× smaller than ArcFace R100 (250 MB) with only a 0.4% accuracy difference, making it ideal for on-device scenarios. On iOS, inference on Apple Neural Engine (A14+) takes ~25 ms; on iPhone SE 2nd gen ~180 ms. For budget devices, server-side inference is more efficient.
Why Document Photo Preprocessing Matters
Passport photos are often compressed, printed, and re-captured. Typical issues: overexposure (glare), moiré patterns, low resolution. Without quality preprocessing, accuracy drops by 10–15%. We use an automated pipeline: gamma correction, denoising (Core Image CINoiseReduction), glare removal via CIHighlightShadowAdjust, and CLAHE for local contrast. After that, detection and alignment proceed as usual.
Additionally, we account for the aging factor: if the document was issued more than 5 years ago, we lower the similarity threshold by 0.03–0.05. This is a compromise between FRR and FAR, validated on a sample of 10,000 real cases. For more background, see Wikipedia: Face recognition.
Protecting Against Attacks with Liveness Detection
Face Match without liveness is vulnerable to a simple photo. Without anti-spoofing, it can be attacked with a mask or deepfake. In production, Liveness Detection must be integrated as the first step: first verify that a live person is in front of the camera, then run face comparison. We combine texture-based (LBP), motion-based (optical flow), and depth-based (TrueDepth on iOS) methods. This combined approach reduces the probability of bypass to <0.01%.
Hybrid Architecture: On-Device + Server
The optimal balance is achieved with a hybrid architecture: on-device MobileFaceNet for initial screening (low-risk filter) and server ArcFace R100 for high-risk verification. This reduces server costs by 40% without sacrificing accuracy. For example, if on-device confidence >0.9, the transaction is accepted immediately; if 0.7–0.9, it is sent to the server; below 0.7, additional checks are triggered. Investment in such architecture typically pays back within 3–6 months by reducing manual verification.
Face Match Implementation Process
- Analysis – Study your data, accuracy, and speed requirements.
- Design – Choose architecture (on-device, server, or hybrid).
- Implementation – Integrate into the app and server side (iOS/Android).
- Testing – Edge cases: glasses, beard, poor lighting, old photos.
- Deployment – Publish to App Store and Google Play.
- Support – Monitor accuracy, fine-tune model if needed.
Estimated timelines:
| Stage | Duration |
|---|---|
| Integration of ready model (CoreML/TFLite) | 3–5 weeks |
| Server verification + audit trail + fine-tuning | 8–14 weeks |
What's Included in the Work
- Requirement analysis and model selection (on-device, server, or hybrid)
- Integration of detection, alignment, and embedding on iOS/Android
- Threshold tuning and testing on a representative sample
- Implementation of a preprocessing pipeline for document photos
- API and integration documentation
- Deployment and operation instructions
- Team training and technical support during launch
Timelines and Pricing
Cost is calculated individually based on complexity, model choice, and required modifications. Our engineers have years of experience in mobile AI and over 20 successful Face Match projects. We guarantee data confidentiality and GDPR compliance.
Ready to upgrade your KYC flow with Face Match? Contact us for a consultation – we'll show you how to reduce manual verification costs by up to 80% while improving security.