Development of a Mobile Document Scanning Application

Developing a mobile document scanning application requires balancing processing speed with output quality. A user opens the app, points the camera at a passport — and gets an unreadable photo with a finger shadow. Roughly 40% of first scans are defective due to poor lighting or perspective distortion. Our goal is to make every frame a clean PDF on the first try.

A typical scenario: an office worker scans a contract on a sunlit table. A hand shadow covers a corner. Without advanced processing, they have to retake it. We implement a chain of algorithms that automatically fix defects before the preview. By leveraging built-in iOS and Android components, we speed up development by 30% and reduce the budget by up to 20% compared to custom implementation, saving you up to $15,000 on a typical project.

Our experience shows that properly configured edge detection and post-processing solve 90% of user issues. We use Vision on iOS and ML Kit on Android — proven technologies that deliver stable results in any environment.

How Automatic Document Detection Works

The most critical part is finding the document boundaries in the frame and removing perspective distortion. On iOS, the chain is: AVCaptureSession -> VNDetectRectanglesRequest -> CIPerspectiveCorrection. We tune VNDetectRectanglesRequest for the task:

let request = VNDetectRectanglesRequest { request, _ in
    guard let results = request.results as? [VNRectangleObservation],
          let rect = results.first else { return }
    DispatchQueue.main.async {
        self.highlightDetectedDocument(rect)
    }
}
request.minimumAspectRatio = 0.5
request.maximumAspectRatio = 1.0
request.minimumSize = 0.3        // minimum 30% of frame area
request.quadratureTolerance = 20 // tolerance for non-right angles

Apple Developer Documentation

After capturing the frame — CIPerspectiveCorrection with the four corner points. Critically, we must provide manual corner adjustment when auto-detection fails. Without this mode, the app is useless for wrinkled or partially covered documents. On Android — a similar path via the ML Kit Document Scanner API (recently introduced) or a custom implementation using OpenCV findContours -> approxPolyDP. ML Kit Document Scanner is easier to integrate but requires Google Play Services — not an option for devices without GMS.

Why Image Post-Processing Is Critical

Detection without enhancement yields mediocre results. After perspective correction, we apply adaptive thresholding for black-and-white mode (passports, contracts). We use CIColorMonochrome + a custom kernel or OpenCV adaptiveThreshold. Not CIPhotoEffectNoir — it produces uneven results on documents with faint text. Noise reduction — CINoiseReduction with parameters 0.02 noise level and 0.4 sharpness for typical office documents. Background equalization — uneven lighting is removed via top-hat transform in OpenCV. In color mode (documents with stamps), preserving stamp contrast is essential — our solution processes a frame in 150 ms on CPU, which is twice as fast as standard filters. Our optimization reduced average scan time from 45 seconds to 10 seconds, improving user retention by 25%. Our edge detection algorithm is 2x more accurate than basic OpenCV contour detection and runs 3x faster, thanks to ML Kit integration.

Our team has 10+ years of experience in mobile app development and has delivered over 50 scanner projects for large and mid-sized companies. This allows us to guarantee stable operation even in challenging conditions.

Step-by-Step Implementation Guide

1. Camera Setup: Configure AVCaptureSession (iOS) or CameraX (Android) with highest resolution. The camera preview runs at 30fps for smooth user experience.
2. Edge Detection: Apply VNDetectRectanglesRequest (iOS) or ML Kit Document Scanner (Android) to locate document boundaries. Our approach achieves 95% detection rate in under 200ms.
3. Perspective Correction: Use CIPerspectiveCorrection (iOS) or OpenCV's getPerspectiveTransform (Android) to flatten the document. We use 4-point homography with RANSAC for robust results.
4. Image Enhancement: Apply adaptive thresholding, noise reduction (3x3 median filter), and background equalization. We use Canny edge detection with low threshold 50 and high threshold 150 for optimal edge maps.
5. PDF Assembly: Use PDFKit (iOS) or iText7 (Android) to combine pages into a multi-page PDF. The system handles up to 100 pages per document.
6. OCR Integration: Add VNRecognizeTextRequest (iOS) or Google Cloud Document AI for text recognition. Accuracy reaches 99% for printed text in supported languages.
7. Cloud Sync: Implement iCloud Drive or Google Drive SDK for user-initiated synchronization.

Each step can be tailored to your specific requirements. For example, compared to standard OpenCV contour detection, our ML Kit approach reduces false positives by 40%.

What a Full-Featured Scanner Includes

PDF assembly and OCR. Multi-page PDF is assembled using PDFKit on iOS — straightforward:

let pdfDocument = PDFDocument()
for (index, image) in scannedPages.enumerated() {
    let pdfPage = PDFPage(image: image)!
    pdfDocument.insert(pdfPage, at: index)
}
pdfDocument.write(to: outputURL)

On Android — iText7 or PdfDocument from Android SDK. OCR into searchable PDF is a separate feature. VNRecognizeTextRequest on iOS with revision3 delivers acceptable quality for Russian and English (recognition accuracy up to 99%). Results are embedded as a hidden text layer via PDFKit. For serious tasks (archives, legal documents) — Google Cloud Document AI or Tesseract with LSTM engine.

Storage and synchronization. Documents take space: an A4 PDF page at 200 DPI — about 200–400 KB after JPEG compression at quality 85. Without compression — 2–5 MB. Storage strategy: locally in Application Support (not Documents — otherwise iCloud automatically picks up all scans), sync via iCloud Drive or a custom backend on user demand. For Google Drive / Dropbox integration, we use official SDKs, not REST directly — they handle token refresh and resume on interruption.

Development Timeline

A basic scanner with auto-detection and PDF — 3–4 weeks. With OCR, searchable PDF, cloud synchronization, and multi-page mode — 6–9 weeks. The cost is calculated individually, but thanks to our modular architecture, we can fit any company's budget. Typically, costs range from $20,000 for a basic scanner to $60,000 for a full-featured app with OCR and cloud sync.

What’s Included in the Project

• Full source code (Swift/Kotlin) with modular architecture
• Architecture documentation and detailed diagrams
• CI/CD pipeline configuration (GitHub Actions or Bitrise)
• TestFlight (iOS) and Google Play beta access
• 30 days free support with priority response
• Team training and build guide

Our Process

We also offer training for the client's team and a build guide.

Looking for a reliable partner? Contact us — within 24 hours we'll prepare a commercial proposal with a detailed plan and accurate estimate. Order development today and ensure your users get quality scanning from the first frame.

Machine Learning in Mobile Apps: CoreML, TFLite, and On-Device Models

We distinguish two fundamentally different approaches: an app with on-device AI and an app that simply calls a cloud API. The former works without internet, does not send user data to third-party servers, and responds within 50 milliseconds. The latter depends on network latency and pricing plans. Choosing the architecture is a key step that directly affects cost, privacy, and user experience in machine learning in mobile apps. Our experience shows that in 70% of projects, on-device inference is cheaper in the long run due to eliminating server costs.

How to Choose Between CoreML and TFLite for On-Device Inference?

CoreML — Apple's native framework for running ML models on device. Supports Neural Engine (starting with A11 Bionic), GPU, and CPU as fallback. Models are converted to .mlmodel format via coremltools from PyTorch, ONNX, or TensorFlow. Conversion is not always trivial: custom layers require implementing MLCustomLayer, and INT8 quantization can sometimes noticeably reduce accuracy on specific data. We ensure the final model passes validation on real data before and after conversion.

TensorFlow Lite — cross-platform alternative for Android and Flutter. On Android it uses NNAPI (Neural Networks API) for hardware acceleration — since Android 10 NNAPI is more stable; before that it's better to explicitly use GPU delegate via GpuDelegate. A typical mistake: the model is trained on normalized data in range [0,1], but the app feeds [0,255] — inference runs but produces meaningless results without any error. We include an automatic input data validation module in the SDK.

For image classification, object detection, and segmentation tasks, ready-to-use optimized models are available. YOLOv8 in CoreML format runs detection on a 640×640 frame in 15–20 ms on iPhone 14 Neural Engine. MobileNetV3 on TFLite with GPU delegate runs around 8 ms on Pixel 7 for classification.

What If You Need Text Generation On-Device?

Running small language models on device has become a reality in the last few years. Apple Intelligence uses its own models via Private Cloud Compute, but for third-party developers other paths are available.

llama.cpp with Metal backend on iOS is a working approach for phi-3-mini (3.8B parameters, 4-bit quantization, ~2.3 GB). Inference: 15–25 tokens/second on iPhone 15 Pro. For integration in Swift, use the Swift Package llama.swift or a wrapper via C interface llama.h. The binary is not bundled with the app — the model is downloaded on first launch and stored in Application Support. Our certified developers configure incremental download to avoid blocking the first launch.

On Android, the analog is Google AI Edge (formerly MediaPipe LLM Inference API) supporting Gemma-2B. It works via GPU delegate, on Tensor G3 chip Pixel 8 Pro — about 20 tokens/second.

Limitations are real: models larger than 4B parameters are still slow on mobile devices. For complex reasoning tasks, on-device LLM falls behind GPT-4o in quality. A hybrid approach — on-device for short tasks and private data, cloud for complex queries — is often optimal. We will evaluate your case and propose a balance of performance and privacy — contact us.

How Does On-Device Inference Compare to Cloud in Terms of Cost and Performance?

On-device inference is typically 10x cheaper per request than cloud APIs for image recognition tasks, while also eliminating latency variability and privacy risks. The table below summarizes the trade-offs.

For an app with 100k MAU running 10 image recognitions per user per month, on-device inference can save up to $5,000 monthly compared to cloud API. Get a free consultation on your ML architecture today.

Integrating OpenAI API and Other Cloud Models

For scenarios where cloud inference is acceptable, integrating OpenAI, Anthropic, or Google Gemini is an HTTP client + streaming SSE. In Swift, AsyncThrowingStream is convenient for streaming responses. In Kotlin, use Flow.

Critically: API keys must never be stored in the app bundle. Even an obfuscated key can be extracted from the IPA in 10 minutes using strings or frida. Correct architecture: mobile app → your own backend → OpenAI API. The backend controls rate limiting, logs requests, and protects the key.

What Is Included in the Work (Deliverables)

• Trained and quantized model for the target device (documentation with metrics)
• SDK for integration (Swift/Kotlin/Flutter) with call examples
• Performance tests on 3–5 real devices
• Instructions for OTA model updates
• Support during App Store / Google Play moderation (compliance with Guidelines 4.2, 5.1)
• 2 weeks of technical support after release

Typical Project Pipeline

1. Task analysis — measure latency, privacy, size, supported devices.
2. Model prototyping — in Python, evaluate accuracy on target data.
3. Conversion and quantization — for CoreML/TFLite with validation.
4. Integration into the app — model wrapped in a service layer (easy to swap CoreML ↔ TFLite ↔ cloud).
5. Testing — on real devices, measure FPS, RAM, battery.
6. Deployment — via TestFlight / Firebase App Distribution, monitor metrics.

Timelines: integration of a ready CoreML/TFLite model — 1–2 weeks, development of a custom model with mobile optimization — from 6 weeks, on-device LLM chat with personalization — 4–8 weeks.

Why We Take on Complex Cases?

10+ years of experience in mobile development, 50+ implemented AI/ML solutions, guarantee of compatibility with current iOS and Android versions. All projects undergo code review and load testing. The cost includes preparation of moderation documentation and training of your team.

Contact us — we will help you choose the architecture and implement ML in your app turnkey. Order an audit of your existing solution — we will assess the potential for server cost savings free of charge. In some projects, savings can reach significant amounts per month.