Core Problems of Plant Recognition by Photo
A user photographs an unfamiliar flower in the park — within a second expects the name, description, and toxicity warning. Technically, the task is not new: PlantNet and Google Cloud Vision have ready endpoints. But the difference between "works in demo" and "works in production" lies in handling bad photos and UX under low model confidence. Over five years we have accumulated experience in this area and guarantee stable results. The hybrid approach (on-device + cloud) combines offline speed with cloud accuracy, cutting cloud request costs by 2x.
Problems We Solve
- Low-quality photos. Blurry, dark, without leaves or flowers. Without filtering, such images lead to wasted API calls and poor user experience. We use a Laplacian filter to assess sharpness and a separate classifier based on EfficientNet for plant detection.
- Low model confidence. When the top-1 candidate has 30% probability, uncertainty must be presented correctly. We show top-3 with confidence percentages and prompt the user to retake a clearer photo.
- Offline mode. Users expect the app to work without internet, but on-device models lag behind cloud accuracy, especially for rare species. We recommend a hybrid: offline for quick response, cloud for refinement when connection is good.
How We Do It: Stack and Principles
Our core stack: Swift, Kotlin, CoreML/TFLite for on-device, and Plant.id API or PlantNet for cloud verification. We use EfficientNet-B4, fine-tuned on the PlantCLEF dataset, achieving up to 95% accuracy on the test set. For rare species, we employ model ensembles: combining predictions from multiple neural networks boosts overall accuracy by 5–10%. On-device recognition runs 10–30 times faster than cloud, critical for instant feedback.
In one project, we integrated the hybrid approach for a houseplant care app. Users photographed a plant, received instant on-device identification, and with internet access got additional care info (watering, light). This increased retention by 20% and cut cloud API costs by 60%.
Implementation Options: Cloud vs. On-Device
| Criterion | Cloud API | On-Device (CoreML/TFLite) |
|---|---|---|
| Accuracy | 90–95% (depends on API) | 80–85% (lower for rare species) |
| Response time | 1–3 seconds (internet) | 30–100 ms (offline) |
| Dependency | Internet required | Fully offline |
| Database update | Automatic | Requires OTA |
| Cost | Per-request model | Free after development |
For most projects, a hybrid is optimal: on-device provides a fast hypothesis (top-3), cloud refines when internet is available.
Comparison of Popular APIs
| API | Accuracy | Specialization |
|---|---|---|
| PlantNet | 85–90% | Wide species coverage |
| Plant.id | 90–95% | Indoor and garden plants |
| iNaturalist | 80–85% | Community and rare species |
Ensuring Accuracy in Poor Lighting
Before sending to the API, we evaluate image quality: sharpness via Laplacian filter, plant presence via a CoreML classifier. If the image is blurry or lacks a plant, we request a retake, avoiding wasted calls. This reduces cloud API costs by 30–40%. Additionally, we apply auto-exposure and contrast adjustment before submission.
Why Choose the Hybrid Approach?
It combines on-device speed with cloud accuracy. Users see results instantly, and refinement happens in the background. Also, without network, users are not left without an answer. The hybrid architecture reduces cloud computing costs by 2x compared to a pure cloud solution. Our certified specialists (Apple, Google) perform turnkey integration.
Technical Details of Image Processing
For sharpness assessment, we use a Laplacian filter with a threshold of 100. The EfficientNet-Lite0 classifier is trained on 5,000 images with and without plants. The on-device model compresses to 5 MB, inference time ~50 ms on iPhone 12.Process of Work
- Analytics — define target audience, usage scenarios, required APIs.
- Design — choose architecture (cloud/on-device/hybrid), design UX for low confidence.
- Implementation — integrate APIs, train on-device model, error handling.
- Testing — validate on a set of 100+ plants, including rare species.
- Deployment — publish to App Store/Google Play, set up monitoring.
Estimated Timelines
Basic integration of one API (Plant.id or PlantNet) with result handling — 1–2 days. Adding on-device model, quality filtering, offline mode, and cross-platform support — 1–1.5 weeks.
What's Included
- Source code with comments
- API documentation
- Deployment setup (CI/CD)
- Consultations for further customization
Contact us to discuss your project details. Request a consultation on AI recognition integration.