Development of a Face Detection System (Face Detection)

Face detection: where it breaks on real footage

In a test video, face detection looks almost solved: even lighting, frontal angles, people walking one at a time — accuracy on the video stream stays around 95%.

In production, the same code on a camera with dense traffic produces dozens of false positives per shift and a noticeable share of misses, and the security service stops trusting the boxes on the screen.

The first failure point is backlight: backlighting from the street or a window burns out half the face, local contrast drops, and the detector loses the contour where a person is still recognizable.

The second is motion: motion blur during a fast step and desync between frames shift the box, the tracker loses the target and reports a new person instead of the same one.

The third is masks, hoods, glasses, and hands near the face, which cover exactly the areas the model uses to make a decision.

Then complex shooting conditions come into play: an angle of 60–70 degrees to the lens, a small face deep in the hall, noise from a night sensor, and compression artifacts.

A single global confidence threshold does not cover such frames: lower it and false positives increase on posters, glare, and reflections; raise it and you lose people deep in the frame.

That is why tuning is not done with one number but by zones: one threshold at the entrance, another at the turnstile, a third at the far end of the corridor.

Practical conclusion: quality is determined not by the chosen model, but by frame preprocessing, thresholds for specific zones, testing on your own archive of recordings, and regression control after every change. This is the work that separates a demonstration from a system. Below we break down how this is addressed at each stage.

Measurable effect: processing speed, accuracy, fault tolerance

We measure throughput in frames per second per processing node, not in the “number of connected cameras” — that way you can see where the headroom ends.

The detector processes every frame of key streams, the tracking module links findings between frames, so one node handles dozens of streams without missing events. When there are more streams, a second node is added — and the logic does not need to be rewritten.

We regulate the false positive rate with the confidence threshold and post-filter: candidates below the threshold go into a queue of disputed cases rather than into events.

We verify quality metrics against the open testing protocol NIST FRVT, and we select the threshold on the customer’s recordings, not on an abstract dataset — that is why accuracy holds under real lighting, angles, and traffic density. The result is units of false positives per hundreds of events instead of dozens.

A separate class of failures is communication loss. Frames and detection results are buffered on the node, each event has its own identifier, so after the channel is restored, the accumulated data goes into the archive without duplicates and without “blind” windows. The operator sees a continuous picture even if the external channel was down.

Manual review of recordings as a process is eliminated: a person looks only at disputed cases, everything else is closed automatically.

Review time drops from hours per shift to a few minutes of exception handling — and this is the same effect as before, only without a constant observer at the monitor.

Face detection approaches: comparing modes

The choice of face detection mode is determined not by the model itself, but by the latency budget, channel stability, and privacy requirements.

The same detector delivers noticeably different quality depending on where the frame is processed and what goes further down the chain — raw video, coordinates of detected faces, or only metadata.

At the start of a project, the choice comes down to four schemes. The difference between them is not “better or worse,” but different trade-offs between latency, channel load, and accuracy in dense scenes.

In practice, people most often settle on a hybrid scheme: the device sends only events and cropped frames, while heavy verification and storage remain on the server.

The channel stops being clogged with streaming video, and accuracy on complex frames increases — provided that detector versions on devices and on the server are compatible and updated consistently.

Implementation stages: from camera audit to system handover

The order of work is structured so that detection quality is confirmed before commissioning, not after. Mistakes during installation and angle selection cannot be compensated for by model tuning, so we start with the site, not with code.

1. Camera audit. We review angles, installation height, obstructions, and actual lighting day and night. We record which cameras are suitable as is, which need to be repositioned or given additional lighting, and where detection will be unstable.
2. Placement layout. We define control zones and installation points so that the face enters the frame fully and front-facing, not in profile at the exit. We approve the layout before installation: moving a camera after launch is harder than designing it correctly from the start.
3. Tuning trigger thresholds. We select the confidence threshold, the number of confirming frames, and the minimum face size. A higher threshold means fewer false positives, but the share of misses grows; we select the balance for the control task, not “by eye.”
4. Testing on a real stream. We run the system on recordings over several days with varying load and lighting, counting misses and false positives separately for each camera. We adjust settings until the metrics reach the agreed level.
5. Regulations and resilience. We describe behavior during camera loss, storage overflow, and power failure, and configure data rotation and degradation alerts. The system should report a problem itself, not after an incident review.
6. System handover. We hand over the configuration, access, and documentation, and train your operations service. Acceptance is based on metrics from real-stream testing, not on the fact that “everything started.”

After handover, the configuration remains open for changes: when traffic changes or cameras are repositioned, thresholds are reviewed selectively.

Deliverables: layouts, configurations, regulations

The documentation set is part of the result, not an appendix to it. We write it so that the customer’s engineer can deploy the system at a second site, change detection zones, and roll back the model without contacting us. What we hand over:

• Video stream layout — sources, connection points, backup routes, and behavior when the channel drops: buffering, reconnection, duty officer notification.
• System configuration — parameters for each camera (area of interest, sensitivity thresholds, analysis interval) are placed in separate files rather than hardcoded.
• Data access policies — roles and permission matrix, separation of biometrics and event metadata, retention periods, export procedure on request.
• Operations regulation — what to check per shift, which metrics to track (false positive rate, dropped frames), and at what values to escalate.
• Operator instructions — typical scenarios: camera unavailable, false positive, changing the detection zone, connecting a new source.
• Model card — version, training data, accuracy evaluation methodology, and method for retraining on new shooting conditions.
• Update and rollback procedure — how a new version is released, who approves it, and which step we return to if metrics drop.

Case study: access control in a retail chain

A chain of several dozen stores already had cameras, but access control relied on people: service entrances and the order pickup area opened by cards, and disputed passages were reviewed manually.

The camera fleet remained heterogeneous — from Hikvision early series to newer models inherited from a previous contractor.

Event review looked like this: a request from the manager, export of recordings by time, frame-by-frame review of three streams.

One episode took 40–60 minutes; with a flow of 8–12 requests per week, this consumed an entire security shift, and there was still no exact entry mark — the passage moment had to be searched manually.

What we changed and what we achieved

We did not change the cameras: we connected to the existing streams and calibrated detection to their quality.

Weak angles are marked as “requires review” instead of producing a false positive, lists are separated by roles — staff, contractors, repeat violations — and each event is logged with time, camera, zone, and match score.

After tuning, review of one event dropped from 40–60 minutes to 2–3 minutes: the operator opens the card and sees the needed fragment, not the entire shift.

The share of episodes closed without manual video review grew to about 70%, repeated passages using others’ cards became visible the same day, and false positives at a busy entrance were cut severalfold through thresholds and angle selection.

What to do if other video analytics is already deployed?

Replacing a working system entirely is not always necessary. Most often, replacing video analytics follows two paths: a parallel system alongside the current system or a phased migration by zones. Which is cheaper in time and resources depends on the camera fleet and access to their settings.

We check compatibility with current cameras before work begins: whether the camera provides a separate stream for analytics, supports an open standard such as ONVIF, and whether it is not monopolized by the existing system.

If the firmware is closed and the stream is provided only by the native server, we install a gateway — it captures the video and passes it to face detection.

We build the migration by zones: first one camera group, and on it we compare metrics with the previous system — misses, false positives, delay until the log entry. Only when the new detection is not inferior do we migrate the next zone.

Rollback is a return to the previous stream, so we keep the old configuration until the end of the migration.

We discuss the limitations of each path in advance, not after launch. A parallel system gives an honest comparison but requires resource headroom; an overlay is faster in terms of timeline but depends on how open the cameras are.

Common technical concerns before starting

The detection model is tested not on a demo stand but on real streams: an entrance group with backlight, a warehouse with masks, a night parking lot with lighting. Below are the concerns that come up most often and how we address them at the design stage, not in operation.

What about night shooting and low light?

At night the camera switches to black-and-white mode with IR illumination, and this does not hinder detection: facial features here are contrast-based, not color-based.

We run the model on frames with noise, glare from headlights, and motion blur, and if necessary add preprocessing — alignment and noise suppression. If the scene does not provide the required contrast, we say so before the start and propose another installation point or lens.

What will the load be on existing equipment?

We calculate load by the number of streams and processing frequency: usually it is enough to analyze a subset of frames, not the entire stream. We place model inference next to the camera or on a shared server — based on measurements at your site, not nameplate figures.

Existing cameras are most often suitable: what is critical is not the model but frame stability and angle.

How is biometric data stored?

By default, we store not images but mathematical templates — feature vectors from which a face cannot be reconstructed. Retention periods are set by policy: events and timestamps exist for a limited period, templates — until consent is revoked.

Access is delimited by roles, actions are logged, and if a local system is required, the solution is deployed in your infrastructure without data leaving it.

Let’s discuss the task and come back with an estimate and plan

To come back with a project estimate and implementation plan, we need to understand the site and scenarios — face detection on paper and at a specific site differ greatly. We will collect the initial data, fix the quality criteria, and determine the scope of the pilot system. Below is what will be needed for this.

• Site layout: number of control zones, camera installation points, obstructions, staircases, and “blind” areas where a person passes out of view.
• Video sources: own cameras or an existing surveillance system, how streams are provided, who is responsible for access and rights to recordings.
• Scenarios: entry to the site, visitor counting, archive search, alert for a restricted zone — what we launch first and what we leave for the second phase.
• Shooting conditions: day and night lighting, traffic density, hats and glasses, angles where the face is partially visible.
• Infrastructure: where we place compute — on site or in a secure system, which communication channels are available, how long we store frames and labels.
• Processing rules: requirements for biometric data storage, retention periods, access roles to results, anonymization procedure.
• Pilot system: one zone and one scenario on which we check accuracy and false positives on your frames before scaling.

Describe the task in free form — we will come back with a timeline estimate, stages, and scope of work. We launch the pilot system first: it gives a measurable result on a real stream and removes risks before deployment across the entire site.