Why image segmentation breaks on real data
Image segmentation breaks not on the test set but on a real stream of frames.
A model that produces smooth contours on prepared data starts tearing masks in production: boundaries jitter from frame to frame, holes appear inside the object, and a rare object class disappears from the prediction altogether.
The first reason is unstable capture. Lighting changes across shifts, the camera shifts by a couple of degrees, the object arrives at a different angle, and noise in the frame grows: glare, dust on the optics, motion blur.
The model learns not the object but the shooting conditions, and under new lighting it confidently marks shadows instead of a defect.
The second is heterogeneous mask labeling. While annotators work by different rules, one outlines the outer contour, another the inner one, and disputed pixels go to the background.
The model averages the contradictions and returns a blurred boundary where pixel-level precision is needed.
The third layer is a rare object class. When there are only a few examples and the background dominates, the model learns the frequency distribution and predicts the majority.
The error surfaces where a decision is already made based on the mask: batch rejection, defect area calculation, geometry control. The operator stops trusting the system and returns to manual inspection — the gain from automation is eaten up.
Let's break down how this is closed: from the shooting protocol and labeling regulations to testing on deliberately difficult frames.
What changes after implementation: effects and metrics
The first measurable effect is the share of frames going to manual review. Before implementation, the operator reviews almost the entire stream; after — only disputed cases, usually 10–20% of frames.
Their time goes to borderline situations where the model is unsure, not to mechanically reviewing the entire archive.
Model accuracy is counted not as a single number but by defect classes: separately recall — the share of real defects found — and separately the false-positive rate.
A missed defect is more expensive than an extra alert, so thresholds for critical classes are shifted toward sensitivity, and object boundaries are refined by a second model like SAM: the mask does not "float" on low-contrast frames.
Frame processing speed is visible in two modes. Batch processes the archive in minutes instead of hours; streaming keeps up with the camera on the line — and the bottleneck becomes not the network but mask post-processing and rejection logic.
Separately, we test robustness to changes in shooting conditions: different lighting, new optics, shifted angle, changed raw material batch.
The metric here is the quality drop on a holdout set of new frames; if it is higher than agreed, we fine-tune the model on 5–10% fresh examples and run the check again.
The number of missed defects and the share of manual review live not in the handover report but in constant comparison: the reference frame set is run on every model update, and regression is visible before rollout to the line.
Comparison of approaches: pretrained model, fine-tuning, custom architecture
The choice between the three modes is determined not by the dataset size but by how much the target objects differ from what the model has already seen.
A pretrained model — Segment Anything and its analogs — produces a working mask with almost no labeling as long as object classes are universal: people, vehicles, road surface, buildings.
As soon as narrow classes appear — weld seam defects, specific fabric types, fruit ripeness stages — quality drops: boundaries "float," and the model confidently highlights the wrong thing.
Fine-tuning on your own data closes this gap but runs into contour labeling: pixel masks are labeled many times slower than boxes and require agreement on what counts as an object boundary.
A custom architecture is rarely justified — when there are strict requirements for on-device speed, non-standard input channels, or domain logic needs to be embedded directly into the model.
| Option | When it fits | Limitations |
|---|---|---|
| Pretrained model | Object classes are universal, no or little labeling | Does not distinguish narrow classes, weak control over boundaries |
| Fine-tuning on your own data | There are hundreds of contour-labeled frames, classes are specific | Expensive labeling, drift control needed when shooting changes |
| Custom architecture | Strict speed requirements, non-standard channels, domain logic | Takes longer to implement, harder to maintain and update |
In practice, we start with a pretrained model and fine-tuning — this gives the first working result on dozens of labeled frames.
We propose a custom scheme and architecture choice only when we can show what exactly they provide compared to fine-tuning: a speed gain or robustness to non-standard input. Describe the task — we'll come back with an estimate of the labeling volume and a plan.
Work stages: from data audit to model handover
Segmentation at the start runs into data, not architecture: the choice between U-Net, SegFormer, and a SAM-based approach is a question for the last step.
Labeling and metrics determine whether the model will be usable in production, so the project follows a fixed sequence where each stage is closed with a verifiable artifact.
-
Task and metrics decomposition. We fix what counts as an object in the image: classes, boundaries, minimum size of a significant area. We agree on the target metric — IoU and Dice for each class separately, not a single averaged number.
-
Data audit. We look at the actual corpus: how many frames, how they were captured, which classes are rare, where duplicates and corrupted files are. The result is a report with an honest assessment of whether there is enough data and what needs to be added.
-
Labeling scheme and dataset preparation. We write instructions for annotators with borderline cases, introduce cross-checking of part of the masks and automated checks for empty and degenerate areas.
We split data by shooting scenarios so that frames from one series do not end up in both training and validation.
-
Training and control. We select the architecture and resolution for the task, monitor overfitting, and save checkpoints. We review intermediate metrics not only on average but also by rare classes.
-
Holdout validation. The final evaluation is on a test set that was not involved in either training or parameter selection. We attach an analysis of the worst examples to the metrics — where the model confuses classes and why.
-
Model handover and team training. We hand over weights in an inference format, preprocessing, versioning rules, and a reproduction script. We show how to fine-tune the model on new data and how to notice quality degradation.
What you receive in hand: labeling, weights, documentation
A model that works well on demo examples and a model that can be operated are different things. The difference is what remains in your hands after acceptance: a reproducible dataset, weights tied to specific metrics, and rules by which the solution is updated without us.
That is why we hand over not a "trained network" but a package where every artifact can be verified and rebuilt.
- Labeled dataset — masks for all target classes in a unified format, with a fixed split into training, validation, and holdout control. Labeling is versioned: you can see which areas were corrected, by whom, and on what basis, and the metric of any model version is reproduced on the same data slice.
- Trained model weights — final and training checkpoints together with the configuration under which they were obtained: architecture (U-Net, SegFormer, SAM approach — depending on the task), augmentation scheme, training parameters, random seed fixation. The run is repeatable and gives a comparable result.
- Labeling versions — a change log with the reason for edits and a "data version → metric" link. This makes it possible to distinguish model improvement from labeling improvement and not retrain on annotator errors.
- Pipeline diagram — preprocessing, slicing large images into fragments with overlap and reassembly, mask post-processing (filtering small areas, smoothing boundaries), output format for your system.
- Quality control regulations — inter-annotator agreement metric (IoU by class) and the share of disputed pixels; a mask goes for re-checking if the discrepancy is above the threshold, rather than "by eye."
- Retraining regulations — triggers (metric drop on a new domain, shooting drift, new object class), procedure, who decides on rollout, and how the new version is validated before replacing the current one.
- Model documentation — a model card with training data, applicability boundaries, known error classes, launch and rollback instructions to previous weights.
- Training and evaluation scripts — a reproducible run on your infrastructure so that retraining does not turn into a one-off project.
Case study: defect control on a production line
In a section, defect control relied on operators: only a sample of frames from the stream was inspected, and the rest went to shipment.
By the end of the shift, attention drops, and small scratches, dents, and edge lack of fusion surfaced already at the customer's incoming inspection — this means batch rework, repeat logistics, and line downtime.
First, several thousand frames from the real line were labeled, then a segmentation model based on SAM was trained and fine-tuned for four defect classes, separating them from acceptable processing marks.
The model run was integrated into the camera stream: the decision is made before the part moves to the next operation, not at final acceptance.
Metrics before and after
| Metric | Before | After |
|---|---|---|
| Share of frames with manual review | sample, about a third of the stream | entire stream, operator looks only at disputed frames |
| Missed defect in shipment | isolated cases every shift | resolved at the section, before the next operation |
| Defect composition | mostly large ones were caught | small scratches and edge lack of fusion are also under control |
| False positives | not tracked | threshold selected so that the line does not stop |
The confidence threshold was tuned separately: the model marks a borderline frame as disputed and gives it to the operator; the line does not stop because of it.
This removed the main reason continuous inspection had not taken hold before — the stream of false positives, after which operators stop trusting the system.
Fine-tuning on new frames follows the same loop: labeling, holdout validation, weight replacement after metric confirmation.
How is image segmentation different from object detection?
Detection answers the question "where is the object": the model returns a bounding box with coordinates and a class.
Segmentation answers the question "which exact pixels belong to the object": the output is a mask the size of the original frame, where each pixel is assigned a class or background label.
A box always captures background and does not separate objects that overlap or touch each other. That is sufficient for counting: detect, count, send for review.
But when you need to measure defect area, separate foliage from ground, or cut out an object along its contour, the box gives an error that cannot be corrected afterward — the mask is built along the real boundary.
The difference in labor costs shows up in labeling. A box takes seconds per object, a mask takes minutes to draw, and the requirements for annotator consistency are higher: disputed pixels on the boundary blur training.
The customer pays for this complexity only when the final solution actually uses pixel-level precision.
Boxes are used for counting and tracking objects, catalog search, and alerting when an object appears in the frame. Masks are used for medical images, defectoscopy, aerial photography, background removal, and any calculations of area and shape.
In practice, this is not a mutually exclusive choice: a detector proposes regions, and a segmentation model (U-Net, SegFormer, and related architectures) refines boundaries within them.
A hybrid scheme reduces the volume of expensive labeling and maintains pixel-level accuracy where it is needed.
Technical doubts before starting a project
Customer engineers ask similar questions, and almost all of them are about the process, not the architecture: how much data to prepare, how to catch overfitting, what will remain inside the perimeter. Below are short answers with which the technical part of the discussion begins.
How many images need to be labeled?
We calculate the volume from the number of classes and scene diversity, not from the archive size. First a small set and a baseline model, then error analysis on the holdout set — and targeted labeling of failing scenes. The accuracy of object boundaries affects the result more than the number of frames.
How can you tell that the model has overfitted?
We keep a holdout set collected on different shooting days: a divergence between metrics on training and on it is a signal to stop, strengthen regularization, and add rare scenes. An overfitted model confidently errs on new frames, so we perform acceptance only on them.
What to do when frames become outdated?
A change in optics, lighting, or angle is a common cause of quality drop. We build in fine-tuning: fresh frames with manual correction return the model to previous metrics without a full rebuild. We store labeling and weights so that rollback to a previous version takes one run.
Can the solution run on our hardware?
Yes: we hand over the training configuration, weights, and labeling preparation scripts; your engineers deploy everything on their side. Frames do not leave the perimeter; if necessary, training runs on an isolated stand without external access.
Let's discuss the task and come back with an estimate and plan
To make the project estimate based on facts rather than assumptions, we need examples of real images, a list of classes, and accuracy requirements to start.
From them, we reproduce labeling on a small sample and check whether the target area is separable from the background — often even here it is clear that some classes are indistinguishable without changing shooting conditions.
In response, we send a task analysis, scope of work, quality validation scheme, and a work plan with approval points. The pilot run is on a limited dataset: metrics are calculated on your images, and the decision on full scope is made based on them, not on promises.
- What we need from you: 100–300 images per shooting scenario, an expert to review disputed cases, access to the deployment environment.
- Task analysis: segmentation type — binary, multiclass, instance — and result format: masks, polygons, object areas.
- Scope of work: dataset labeling, training and comparison of architectures, mask post-processing, embedding the model into your process.
- Quality validation scheme: holdout set, share of correct pixels and intersection of predicted masks with the reference, separate control on rare classes.
- Pilot run: training on part of the data and measuring metrics on real images; we fix acceptance criteria before work begins.
- Work plan: stages, timelines, approval points, and a responsible person on each side. Handover plan: model weights, launch configurations, instructions for fine-tuning on new data.
Describe the task — we'll come back with an estimate and a plan.
