Document Verification: Passport, License, ID Card Turnkey

Document Verification: Passport, License, ID Card Turnkey

Financial and car-sharing services lose up to 30% of clients at the KYC stage due to poor verification. Typical problem: the system rejects real documents or passes fakes. Central Bank data indicates that up to 15% of forgeries go undetected by single-layer OCR. That means direct financial losses and reputational risks. Over the past years, we have developed an approach that minimizes such scenarios — verification accuracy exceeds 98% with a false rejection rate below 1%.

Document verification is the task of checking the authenticity of a physical document based on its digital image. It is used in KYC processes (banks, fintech, car-sharing, mobile operators). The task is more complex than simple OCR: you need not only to read the data but also to confirm its authenticity. Errors at this stage lead to direct financial losses and reputational risks.

How to Ensure Verification Accuracy Above 98%?

Single-layer OCR is not enough. Modern forgeries contain machine-readable zones that are visually indistinguishable from real ones. Therefore, we use a cascade of checks, each filtering a certain class of fraud. According to Central Bank data, up to 15% of forgeries are not detected by single-layer OCR — our pipeline reduces this figure to 1%.

class DocumentVerificationSystem:
    def __init__(self):
        self.ocr = PaddleOCR(use_angle_cls=True, lang='ru')
        self.mrz_reader = MRZReader()
        self.authenticity_checker = AuthenticityChecker()

    def verify(self, image_path: str,
               doc_type: str = 'passport_ru') -> VerificationResult:
        result = VerificationResult()

        # 1. Image quality
        quality = self.assess_image_quality(image_path)
        if quality.score < 0.6:
            result.rejected = True
            result.reason = 'low_image_quality'
            return result

        # 2. OCR all visible fields
        result.ocr_fields = self.extract_fields(image_path, doc_type)

        # 3. MRZ (Machine Readable Zone) for passports
        if doc_type in ['passport_ru', 'foreign_passport']:
            mrz_data = self.mrz_reader.read(image_path)
            result.mrz_data = mrz_data
            # Cross-check MRZ vs OCR fields
            result.mrz_consistency = self.cross_check_mrz(
                result.ocr_fields, mrz_data
            )

        # 4. Security features check
        result.security_features = self.authenticity_checker.check(image_path)

        # 5. Final verdict
        result.verified = self._make_verdict(result)
        return result

Why Is Multi-Level Document Check Critical for KYC?

Each level solves its own task: image quality detection filters blurry photos, MRZ checks data integrity, and analysis of security elements (guilloches, microtext, UV glow) detects forgeries. In practice, this gives accuracy >98% with a false rejection rate below 1%. For comparison, standard OCR solutions without MRZ validation yield about 85% accuracy on real data.

How Do We Read MRZ and Check Checksums?

MRZ is a standardized zone at the bottom of the passport with encoded data. Checksums allow verifying data correctness. The passporteye library gives a baseline accuracy of about 95%, but after our refinement with post-processing and cross-validation, it reaches 99%+. Our processing pipeline is 3 times faster than standard solutions thanks to GPU optimization (batch inference on Triton Server).

from passporteye import read_mrz

class MRZReader:
    def read(self, image_path: str) -> dict | None:
        mrz = read_mrz(image_path)
        if mrz is None:
            return None

        data = mrz.to_dict()
        return {
            'surname': data.get('surname', ''),
            'names': data.get('names', ''),
            'country': data.get('country', ''),
            'number': data.get('number', ''),
            'nationality': data.get('nationality', ''),
            'date_of_birth': data.get('date_of_birth', ''),
            'sex': data.get('sex', ''),
            'expiry_date': data.get('expiry_date', ''),
            'personal_number': data.get('personal_number', ''),
            'valid_composite': data.get('valid_composite', False),
            'valid_number': data.get('valid_number', False),
            'valid_dob': data.get('valid_dob', False),
            'valid_expiry_date': data.get('valid_expiry_date', False),
        }

What About Expiry Date and Format Check?

def validate_passport_fields(fields: dict) -> dict:
    """Check Russian passport field formats"""
    errors = []

    # Series and number: 4 digits series, 6 digits number
    if fields.get('series'):
        if not re.match(r'^\d{4}$', fields['series']):
            errors.append({'field': 'series', 'error': 'invalid_format'})

    if fields.get('number'):
        if not re.match(r'^\d{6}$', fields['number']):
            errors.append({'field': 'number', 'error': 'invalid_format'})

    # Issue date: not later than today, not earlier than 2000 (acceptable for existing documents)
    if fields.get('issue_date'):
        issue_date = parse_date(fields['issue_date'])
        if issue_date:
            from datetime import date
            if issue_date > date.today():
                errors.append({'field': 'issue_date', 'error': 'future_date'})
            if issue_date.year < 2000:
                errors.append({'field': 'issue_date', 'error': 'too_old'})

    # Date of birth
    if fields.get('birth_date'):
        birth_date = parse_date(fields['birth_date'])
        if birth_date:
            age = (date.today() - birth_date).days / 365
            if age < 14 or age > 120:
                errors.append({'field': 'birth_date', 'error': 'invalid_age'})

    return {'valid': len(errors) == 0, 'errors': errors}

How to Set Up a Verification Pipeline in 5 Steps?

1. Requirements gathering — analyze document types, acceptable image formats, speed requirements.
2. Component selection — OCR (PaddleOCR, Tesseract), quality detector, MRZ reader (passporteye), authenticity check module.
3. Pipeline development — implement DocumentVerificationSystem class with cascade checks, integrate GPU inference.
4. Integration — REST API or gRPC endpoint, support for batch requests for high throughput.
5. Testing and monitoring — run on your data (at least 1000 samples), record metrics (latency p99, accuracy, false positives).

Case Study: Bank with 1M Clients Reduced KYC Time from 10 Minutes to 30 Seconds

After implementing our pipeline, the client achieved a one-time verification in 20 seconds, and repeat verification in 2 seconds (using result caching). The false rejection rate dropped from 12% to 1.5%, reducing operator workload by 8x. Contact us to achieve similar results — reduce operational costs by up to 45% through automation. We will estimate your project in 2 days.

Supported Document Types and Implementation Timelines

What's Included in Turnkey Work?

We provide the full cycle: requirements analysis, architecture selection (RAG, classifier, or hybrid), pipeline implementation, REST API integration, testing on your document pool (at least 1000 samples), documentation, operator training, and 1 month post-launch support. We guarantee verification accuracy of at least 98% with a false rejection rate below 1%. Typical ROI is 6–9 months.

Our experience includes 15+ projects in fintech, banking, and government sectors. Order a free pilot — we will verify 100 of your documents and show the result. Get demo access to an already running system.

How Distribution Shift Kills CV Model Metrics in Industry

On a production line, a camera is installed to control product quality. The model is trained on 10,000 labeled images—test accuracy mAP 0.84. Deployed to production, and in the first week it misses 30% of defects. Lighting on the line changes between shifts; distribution shift nullifies the metrics. This is a classic story with computer vision in industry, where pattern recognition fails without proper drift handling.

Our engineers, with experience from 60+ computer vision projects, know how to eliminate such scenarios. We guarantee stable model performance under real conditions.

Object Detection: YOLO, RT-DETR, and Everything in Between

YOLO is the standard for real-time detection. YOLOv8 and YOLOv11 from Ultralytics are the most used versions in production: simple API, active community, built-in validation, and export to ONNX/TensorRT. For tasks with high accuracy requirements and less critical latency, RT-DETR, a transformer-based architecture without NMS, gives better mAP on COCO at comparable speed to YOLOv8l.

A typical mistake when training a detector: dataset of 8000 images, 3 classes, fine-tune YOLOv8m—F1 0.73 on validation. Look at confusion matrix—one class is almost never detected. Cause: imbalance 1:23. Solution: oversampling rare class, focal loss for objectness, augmentations (Mosaic, MixUp disabled for rare class as they "blur" it). Transfer learning is mandatory: pretrained on COCO weights reduces data requirement by 10 times. Fine-tuning on 500–2000 domain images yields a working model in 1–2 days on a single GPU.

For edge deployment: export to ONNX → TensorRT engine. YOLOv8n in TensorRT FP16 on Jetson AGX Orin gives 150+ FPS at P99 latency < 8 ms—3 times faster than ONNX Runtime without TensorRT. On server A10G: 700+ FPS for YOLOv8n in TensorRT INT8.

How Does Fine-Tuning YOLO Help in Pattern Recognition?

Suppose you need to find micro-defects on a metal surface—a task with high resolution and class imbalance. We use YOLOv8m pretrained on COCO and fine-tune on 2000 proprietary images. Apply augmentations Mosaic, MixUp, random perspective. After 200 epochs, mAP 0.5 reaches 0.93. Key techniques:

• Focal loss for the objectness head—reduces contribution of easily classified examples.
• Class-balanced sampling—equalizes representation of rare classes.
• Test Time Augmentation (TTA)—increases recall by 5–7% through averaging over flips and scales.

Get a consultation on architecture selection for your task—contact us.

Segmentation: SAM, Mask R-CNN, and Instance Segmentation

SAM (Segment Anything Model) from Meta changed the approach to segmentation. SAM 2 works with video, supports object tracking across frames—for interactive object selection by point or bbox, it's the best out-of-the-box choice. For production instance segmentation without interactive prompting, Mask R-CNN or YOLOv8-seg are used. YOLOv8-seg trains like a regular detector with additional masks, convenient in the same pipelines. Semantic segmentation (each pixel is a class) uses SegFormer, DeepLabV3+. SegFormer-B5 provides a good balance of accuracy and speed for satellite imagery or medical segmentation.

Case study: cell segmentation on microscopic images. Dataset of 400 images with manual annotation. Training Mask R-CNN on ResNet-50 backbone gave IoU 0.61—poor. Problem: objects (cells) overlap; standard NMS kills overlapping predictions. Solution: switch to cellpose (specialized architecture for biomedical tasks) + soft-NMS. IoU increased to 0.79.

OCR: When Tesseract Fails

Tesseract is a starting point for simple tasks: printed text, good lighting, straight layout. As soon as there are handwritten elements, non-standard fonts, perspective distortions, or multi-column layouts, Tesseract degrades quickly.

PaddleOCR is a production-grade solution: text block detection + recognition + structural analysis. Works out of the box for 80+ languages, including Russian. Supports tables and complex document structures. TrOCR (Microsoft) is a transformer OCR with strong results on handwritten text. For Russian handwritten text, fine-tuning is needed: the base model is trained mostly on Latin script.

What to Do When Tesseract Cannot Handle Pattern Recognition on Documents?

For tasks like "extract data from invoices/contracts/passports," we use LayoutLMv3 or Donut—these models understand document layout, not just text. Integration via Hugging Face Transformers, fine-tuning on 200–500 annotated documents. Typical pipeline:

1. Preprocessing: deskew, denoising, binarization via OpenCV.
2. Text block detection: PaddleOCR detection or CRAFT.
3. Recognition: PaddleOCR recognition or TrOCR.
4. Post-processing: normalization, validation via regex or LLM for structured fields.

For documents with fixed structure, template matching + OCR by coordinates is often more reliable than an end-to-end solution.

Face Recognition: Identification and Verification

Face recognition = detection + alignment + embedding + matching. Each stage matters.

Detection: RetinaFace or InsightFace for accurate face localization and keypoints. MTCNN is older but reliable. Embedding: ArcFace (InsightFace) is state-of-the-art for face recognition embeddings. Models iresnet50/iresnet100 pretrained on MS1MV3 (5M identities). Embedding vector 512 float32, comparison by cosine similarity. Threshold tuning: decision threshold is a critical parameter. At threshold 0.6, typical FPR on LFW benchmark is 0.001, TPR is 0.985. In production, threshold must be calibrated to the real distribution: people in masks, with changed appearance, different lighting conditions. Liveness detection is mandatory: MiniFASNet—lightweight model on CPU; FaceX-Zoo contains several pretrained liveness detectors.

Video Analytics

Video is a sequence of frames plus a temporal dimension. A naive approach—detecting on every frame—is expensive.

Tracking: ByteTrack and BoT-SORT are the standard for multi-object tracking. They work on top of any detector, adding persistent IDs to objects across frames—enabling object counting, motion tracking, velocity.

Optimization: not every frame needs processing. For static scenes, detect every 5–10 frames, with tracking in between. For event detection (person entering a zone), background subtraction (OpenCV MOG2) serves as a lightweight pre-filter before neural detection. Action recognition: SlowFast, VideoMAE for action classification. Heavy models—for production use ONNX export + TensorRT or offline processing.

How to Measure Pattern Recognition Model Quality in Production?

Quality monitoring is key to MLOps. We track:

• Prediction confidence distribution.
• Share of low-confidence predictions (indicator of OOD data).
• Drift of input images via feature distribution (embeddings from backbone).

A drop in average confidence from 0.87 to 0.71 over a week is an early signal of distribution shift. NVIDIA Triton Inference Server recommends tracking these metrics via Prometheus. Our certified engineers set up monitoring and guarantee SLA for inference quality.

Deployment of CV Models

For online inference, we use Triton Inference Server (NVIDIA)—production standard for serving CV models. Supports TensorRT, ONNX, PyTorch, dynamic batching, multiple instances. REST and gRPC API. We guarantee stable operation under load.

Edge deployment: ONNX Runtime on ARM/x86 CPU. TensorFlow Lite for mobile devices. OpenVINO for Intel CPU/GPU/VPU—gives 2–3× speedup on Intel hardware compared to ONNX Runtime. After deployment, we hand over the model with documentation and train personnel.

What Is Included in the Work

Deadlines and Cost

A prototype detector on existing data takes 1–2 weeks. Production system with optimization for target hardware takes 4–8 weeks. Full cycle including data labeling (1000–5000 images) takes 2–4 months. Cost is calculated individually for each task. Typical savings from implementing a quality control system can be significant per production line.

We have been in the market for over 5 years and completed 60+ computer vision projects. We will evaluate your project end-to-end—request a consultation to get a quote and technical proposal.