AI-Powered NSFW Content Detection for Mobile Apps
When publishing user images through mobile app, NSFW detection task appears at two levels: speed (show content with minimal delay) and accuracy (false blocks destroy trust). Both conflict, right architecture always trades off.
Common Mistakes
Fully Server Classification Without Pre-Filter
If each uploaded image goes to API service and waits for answer before showing — latency grows under peak load, UX degrades. Single AWS Rekognition DetectModerationLabels request takes 300–800 ms. For photo chat or marketplace with fast uploads — unacceptable.
On-Device Classification "By Force"
Running full NSFW model on every video frame or each gallery photo heats device and drains battery. iPhone 12 with Open NSFW (~50 MB in CoreML) on continuous processing thermal throttles in 8–10 minutes.
Two-Step Architecture
Optimal scheme: light on-device pre-filter + cloud verification of edge cases.
On-Device (CoreML / TFLite)
Client runs light binary model (~8–15 MB): MobileNetV3-Small or specialized NSFW model in coremltools conversion. Result — two classes: safe / unsafe, plus confidence score.
// iOS: CoreML inference before upload
func checkImage(_ image: UIImage, completion: @escaping (NSFWResult) -> Void) {
guard let pixelBuffer = image.resized(to: CGSize(width: 224, height: 224)).toCVPixelBuffer() else { return }
let request = VNCoreMLRequest(model: nsfwModel) { request, _ in
guard let results = request.results as? [VNClassificationObservation],
let top = results.first else { return }
let result = NSFWResult(
label: top.identifier,
confidence: top.confidence
)
DispatchQueue.main.async { completion(result) }
}
try? VNImageRequestHandler(cvPixelBuffer: pixelBuffer).perform([request])
}
Thresholds: confidence > 0.92 for unsafe → block on client, don't upload. confidence between 0.65 and 0.92 → upload in hidden state, send for server verification.
Android: ML Kit + TFLite
Use ImageClassifier from TFLite Task Library — manages model lifecycle and Bitmap processing without manual buffer management:
val classifier = ImageClassifier.createFromFileAndOptions(
context,
"nsfw_lite.tflite",
ImageClassifier.ImageClassifierOptions.builder()
.setMaxResults(2)
.setScoreThreshold(0.5f)
.build()
)
val tensorImage = TensorImage.fromBitmap(bitmap)
val results = classifier.classify(tensorImage)
val nsfwScore = results.flatMap { it.categories }
.firstOrNull { it.label == "nsfw" }?.score ?: 0f
Server Verification via Google Cloud Vision / AWS Rekognition
For edge cases and final check before publishing:
// send to server only edge cases
if (nsfwScore in 0.65f..0.92f) {
uploadForReview(imageUri, nsfwScore)
}
Google Cloud Vision SafeSearch returns 5 categories: adult, spoof, medical, violence, racy — each with VERY_UNLIKELY to VERY_LIKELY. Lets fine-tune policy: medical apps whitelist medical category, kids' apps set racy = POSSIBLE as block trigger.
Video: Frame-by-Frame Analysis with Sampling
For video UGC grab frames via AVAssetImageGenerator (iOS) at 1-second intervals, run on-device model in parallel via DispatchQueue.concurrentPerform. Android — MediaMetadataRetriever.getFrameAtTime() + coroutines with Dispatchers.Default. If any frame exceeds unsafe threshold — entire clip marked for review.
Process
Content policy analysis: which categories to block, which require human review, medical/art whitelist needed.
Select and test on-device model on representative dataset.
Integrate two-step logic into client + server verifier.
Tune thresholds per app audience (rating).
Timeline Guidance
On-device pre-filter with CoreML/TFLite — 2–3 days. Full two-step system with server verification and video processing — 1–1.5 weeks.