Animal Recognition from Photos in Mobile Applications
Similar architecture to plant recognition, but with a key difference: animals move. User photographs a bird on a branch—it flies off while they raise their phone. Beyond classification accuracy, consider capture speed and recognition of blurred shots.
Choosing a Model for the Task
Task depends heavily on target animal class:
| Category | Ready API / Model | Species Count |
|---|---|---|
| Birds | Merlin Bird ID (Cornell Lab API), iNaturalist | 10,000+ |
| Wildlife | iNaturalist API, INat Seek SDK | 100,000+ taxa |
| Fish | iNaturalist, FishVerify API | 30,000+ |
| Pet breeds | Google Cloud Vision, custom CoreML | 200–400 breeds |
| Insects | iNaturalist, iNat Seek | 500,000+ species |
For most projects with mixed audience, iNaturalist API is optimal: broad taxonomic database, confidence score at species/genus/family level (honest at low photo quality), mobile SDK available—Seek with on-device model for iOS and Android.
iNaturalist Seek SDK Integration on Android
class AnimalRecognitionManager(private val context: Context) {
// Seek uses TFLite model ~15MB
private val seekModel by lazy {
SeekClassifier(context, modelPath = "seek_v2.tflite")
}
fun recognizeFromBitmap(bitmap: Bitmap): List<TaxonResult> {
val resized = Bitmap.createScaledBitmap(bitmap, 299, 299, true)
val results = seekModel.classify(resized)
return results
.filter { it.score > 0.15f }
.sortedByDescending { it.score }
.map { result ->
TaxonResult(
taxonId = result.taxonId,
name = result.name,
commonName = result.commonName,
rank = result.rank, // SPECIES, GENUS, FAMILY
confidence = result.score,
photoUrl = result.defaultPhotoUrl
)
}
}
}
Rank matters for UI: if species confidence is 30%, show "Family Fringillidae (85%)" rather than specific species with low confidence.
Fast Capture for Moving Objects
// iOS: frame buffering to select sharpest
class AnimalCaptureViewController: UIViewController {
private var frameBuffer: [CMSampleBuffer] = []
private let bufferSize = 10 // last 10 frames
func captureOutput(_ output: AVCaptureOutput,
didOutput sampleBuffer: CMSampleBuffer,
from connection: AVCaptureConnection) {
frameBuffer.append(sampleBuffer)
if frameBuffer.count > bufferSize {
frameBuffer.removeFirst()
}
}
// On button tap—pick sharpest frame from buffer
func captureWithMotionCompensation() -> UIImage? {
return frameBuffer
.compactMap { UIImage(from: $0) }
.max(by: { sharpnessScore($0) < sharpnessScore($1) })
}
}
This approach reduces blurred shots by ~3x versus normal capturePhoto().
Timeline Estimates
Integrating one API or Seek SDK with basic UI—1 day. Adding frame buffering, best shot selection, animal card with description and links (Wikipedia, iNaturalist), recognition history, iOS + Android—up to 2 days on basic task, up to 1.5 weeks for custom requirements.