AI Car Recognition (Make & Model) for Mobile Apps
Recognizing a car's make and model from a photo is a well-studied problem. Models trained on Stanford Cars Dataset (196 classes) or CompCars achieve 90%+ accuracy on clean side shots. The main production challenges are angles, partial visibility (front only or rear only), nighttime conditions, and niche market vehicles.
Our team has 5+ years of experience in mobile AI solutions and has delivered over 50 Computer Vision projects for automotive. We've tackled these issues in real projects for insurance and dealer apps. In this article, we'll cover how to build a robust car recognition system that works under challenging conditions, and compare implementation options—from ready-made APIs to custom CoreML/TFLite models. We'll describe specific technical solutions including multi-angle capture and a hybrid VIN+Visual approach. If you need an estimate for a similar project, contact us for a free consultation.
Ready APIs and Their Limitations
| Service | Number of Models | Notes |
|---|---|---|
| CarAPI / CarQuery | 10,000+ | Good for classification, weaker on old/rare cars |
| AutoVIN API | Broad database | VIN decoding combined with photo |
| Imagga | Custom tags | Requires fine-tuning for automotive |
| Google Cloud AutoML Vision | Custom | Needs own labeling |
For most projects: a custom CoreML/TFLite model based on EfficientNetV2, fine-tuned on a combined dataset (Stanford Cars + VMMRdb). Model size — 25–40 MB, Top-1 accuracy on popular models — 88–93%. A custom model yields 10-15% higher accuracy than ready APIs, especially on rare cars. Inference time on iPhone 13 — under 50 ms.
How to Choose the Approach for Car Recognition?
The choice between ready API and custom model depends on your requirements. If you only need to recognize popular models (top 100-200) and accuracy isn't critical, CarAPI will suffice. For insurance or dealer apps where every detail matters, a custom model with multi-angle capture is the only reliable option. We recommend starting with an API prototype to evaluate accuracy on real data, then migrating to a custom solution.
iOS Implementation with CoreML
class CarRecognitionService {
private lazy var model: VNCoreMLModel = {
let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine
let mlModel = try! CarClassifierV3(configuration: config).model
return try! VNCoreMLModel(for: mlModel)
}()
func recognize(image: UIImage) async throws -> [CarPrediction] {
guard let cgImage = image.cgImage else { throw CarError.invalidImage }
return try await withCheckedThrowingContinuation { continuation in
let request = VNCoreMLRequest(model: model) { request, error in
if let error = error {
continuation.resume(throwing: error)
return
}
let results = (request.results as? [VNClassificationObservation]) ?? []
let predictions = results
.filter { $0.confidence > 0.05 }
.prefix(5)
.map { CarPrediction(
makeModel: $0.identifier, // "Toyota Camry"
confidence: $0.confidence
)}
continuation.resume(returning: Array(predictions))
}
// Normalizing image orientation is critical — otherwise accuracy drops
request.imageCropAndScaleOption = .centerCrop
let handler = VNImageRequestHandler(cgImage: cgImage,
orientation: image.cgImageOrientation)
try? handler.perform([request])
}
}
}
The parameter imageCropAndScaleOption = .centerCrop is a non-obvious detail. By default, CoreML scales images differently than the model expected during training, causing a 5–8% accuracy loss.
Why Multi-Angle Capture Improves Accuracy?
For high-accuracy tasks (insurance apps, car dealers), one shot is not enough. We request three angles:
enum CarPhotoAngle: CaseIterable {
case frontThreeQuarter // 3/4 front — optimal for make/model
case rear // for rear (additional verification)
case side // side — for body style and generation
var instruction: String {
switch self {
case .frontThreeQuarter: return "Photograph the car from front-side (45°)"
case .rear: return "Photograph from the rear"
case .side: return "Photograph strictly from the side"
}
}
}
// Aggregating results from three shots — weighted voting
func aggregatePredictions(_ predictions: [[CarPrediction]]) -> CarPrediction {
let weights: [Double] = [0.5, 0.3, 0.2] // frontThreeQuarter more important
// ... weighted voting by makeModel
}
Year and Generation Identification
Visually determining the year is harder than make/model: facelifts alter appearance minimally. Two approaches:
- Generation classifier (separate head in multi-task model)
- Hybrid: VIN via OCR (if visible) + visual generation classification
VIN approach is more accurate: if OCR reads the VIN from the license plate or windshield, all data (make, model, year, trim) is decoded without AI via NHTSA API or paid VIN decoders. VIN OCR recognition time is about 200 ms.
Work Stages (Turnkey)
- Requirements analysis and dataset collection (if rare models needed)
- Model training and validation (EfficientNetV2, CoreML/TFLite)
- Recognition module integration with app UI
- Testing on real photos under different conditions
- Post-release support (fine-tuning, API updates)
What's Included in the Deliverable
| Deliverable | Description |
|---|---|
| Trained model | CoreML/TFLite, 25–40 MB |
| Source code | Swift/Kotlin with comments |
| Documentation | API, architecture, retraining instructions |
| Support | 1 month after delivery, bug fixes |
| Accuracy guarantee | 90%+ on popular models |
Our experience reduces labeling costs by 30% through transfer learning and selective key image sampling.
Timeline Estimates
Integration of a ready CoreML model with result display UI — 3–5 days. Full system with multi-angle capture, hybrid VIN+Visual approach, car feature database, and iOS + Android — 1–2 weeks.
To get an estimate for your project, fill out the form or write to us — we'll reply within a day. Get a free consultation.