AI in the food industry: where quality control breaks down
On most lines, quality control in food production still relies on spot sampling. An operator takes a few points from a batch — by volume, that is single-digit percentages of the flow.
A defect distributed unevenly simply does not make it into the sample, and the batch moves on with a defect already baked in.
The second source is the spread in assessment between shifts. Color, texture, and foreign inclusions are assessed on an internal scale that each operator keeps in their head.
One shift rejects what another lets through, and the human factor in control ceases to be an error — it becomes a systemic cause of unstable quality.
The third failure point is the moment of detection. Batch defects usually surface at packaging or in the warehouse, when the raw materials have already been consumed and the line has run a shift.
Raw material losses accumulate nonlinearly: the later a defect is found, the more volume goes into rework and waste, and along with it the share of product that cannot be shipped grows.
Documentation deserves separate mention. HACCP requires every check to be confirmed by a record, and records are kept manually at the end of a shift — from memory and rounded in one's own favor.
When investigating an incident, it is impossible to reconstruct the real picture from such logs: it is unclear neither when the defect arose on the line nor which batch of raw materials brought it.
All three points — manual sampling, subjective assessment, late detection — are one task: make control continuous and reproducible. We will break down how this is addressed.
What AI control delivers: defect classes and measurable effect
The effect of machine vision on the line breaks down into three measurable quantities: what share of the batch we see at all, which defect classes we cover, and how consistently the decision is made across shifts.
Manual spot control by an operator gives coverage of single-digit percentages of the batch — the rest goes to the consumer or is uncovered already at the retailer's acceptance.
Defect classes covered by continuous control on the conveyor:
- foreign inclusions — metal, film, packaging fragments, pit;
- geometry and caliber — deviation in shape, size, portion weight;
- color and texture — underbaking, burning, uneven roasting, pale cut;
- package integrity — seam rupture, weak seal, loss of vacuum, label misalignment;
- filling and arrangement — underweight, gaps, product sticking together.
Coverage grows to the entire batch without reducing line speed, and the share of missed defects ceases to depend on shift fatigue: the trigger threshold is configured once during commissioning and does not drift by the end of a shift.
Assessment stability is measured by the divergence of decisions between shifts and areas — it is a number in a report, not a technologist's feeling.
Every deviation leaves a frame, a class label, and a timestamp. The decision log and photo capture cover traceability under HACCP, and complaint investigations proceed from facts, not the operator's recollections.
Control modes: manual, spot, continuous, hybrid
The control mode is chosen based on process variability, defect class, and the cost of a false pass, not on shift habit.
Manual inspection remains for small batches and incoming raw material control, but at line speed the eye can only assess part of the product — the rest goes to packaging.
| Mode | When it fits | Limitations |
|---|---|---|
| Manual | small-batch production, unique recipes, incoming raw material control | depends on shift and operator fatigue, no decision history |
| Spot | stable process, defect manifests in a batch rather than a unit | between samplings, defects pass to packaging unimpeded |
| Continuous (in-line control) | high line speed, homogeneous product, tight tolerances | requires calibration for the product and protection of cameras from contamination |
| Hybrid | multiple lines or products in one area | harder to set up, requires unified logic for thresholds and escalation |
Sampling frequency is a calculated value, not a regulatory constant: it is derived from line speed and operator response time, so between samplings there is always a window in which a defect has already moved further down the flow.
Continuous control closes this window but pays for it with sensitivity to product changeover and contamination — thresholds must be recalculated for each recipe.
In practice, a mode is rarely adopted in pure form: usually they start with spot control, keep a decision log for each sample, and only then add the continuous branch where losses from a missed defect are higher than the cost of a false alarm.
This is how a hybrid control mode is assembled for a specific line — with different sampling frequencies by area and a common escalation threshold.
Implementing quality control: from line audit to regulations
The work sequence is the same for a bottling, packaging, and raw material sorting line: first we capture the physics of the process, then we build the control scheme, and only then hand it over to the shift.
Each stage relies on decisions from the previous one, so a skipped step does not save time but sets the project back two positions.
- Line audit. We look at where the observation equipment is placed, how light falls on the product, at what speed the flow moves, and what defects the operator currently finds by eye. The output is a defect map and an honest list of what is physically distinguishable on this line and what is not.
- Control scheme. We determine which deviations the model catches, which remain under manual inspection, and where the tolerance boundary lies. Here we also decide what happens to the product on a trigger: line stop, diversion to a separate tray, or a mark in the log.
- Sample annotation. We collect frames across the entire shift and different batches, necessarily including rare defects and borderline cases. Disputed frames are annotated by a technologist and go into a separate set — these are exactly what the model is later tested on.
- Pilot on one line. The solution runs in observation mode and does not affect the operator's actions: the system shows a verdict, but a human performs the rejection. This is how we compare machine assessment with manual and see the real share of misses and false triggers without risk to output.
- Model validation across shifts. We run it on different crews, raw material batches, and lighting modes, and calibrate the trigger threshold. We separately check behavior during raw material changeover and after equipment changeover.
- Control regulations. We describe the shift's action on a trigger, the confirmation and rejection procedure, responsible persons, and the event log. This also includes the procedure for updating the model when a recipe or supplier changes.
- Handover and monitoring. We train the shift, agree on periodic re-evaluation on fresh annotated batches and on the threshold after which the model needs retraining.
A stage is considered closed when the shift works according to the regulations without an engineer's support, and the divergence from manual control falls within the agreed norm.
Deliverables: model, line diagrams, control regulations
After commissioning at the production site, what remains is not only a trained model but everything that makes it possible to operate it without the authors.
Below is the list of what we hand over together with the system: files, diagrams, and regulations that both the shift and the support engineer work from.
- Settings package: model configuration with confidence thresholds and rejection rules, every version is saved — any edit can be rolled back to the previous one.
- Camera placement diagram: mounting points and field-of-view zones by line position, lighting and background requirements, so that inspection can be reproduced on a second line without repeated measurements.
- Model documentation: defect class map with examples of "good / conditionally good / reject", applicability conditions, and known limitations — which products and modes the solution has not been tested on.
- Control regulations: operator actions on a trigger, procedure for recording false triggers, frequency of checks on a reference sample.
- System access rules: roles and permissions, who changes thresholds and on what grounds, action log, login via enterprise accounts.
- Support procedure: how a model update is released, where the previous version is stored, and how quickly rollback is performed.
- Shift instructions: workplace cards — what to do when the line stops, camera communication is lost, or a frame is disputed.
Such a package makes the system transferable: an in-house engineer or another contractor can deploy it on a second line without reconstructing the logic from correspondence.
Without it, the settings live in one person's head, and changing thresholds turns into an experiment on a live line.
Case study: portion and packaging control on a confectionery line
The confectionery line produces cookies in three packaging formats, and before implementation, portion control relied on spot weighing: an operator removed several items from the conveyor once every half hour.
Weight variation within a batch was found already in the finished goods warehouse, and underfilling in one cell moved further down the flow along with the rest.
A camera and load cell were installed on the outgoing conveyor. A segmentation model separates the package contour from the background and, together with weight, checks the seam, seal, and label position — a deviation is rejected before being placed in the box.
The operator adjusts the rejection threshold for the batch, and retraining for a new format takes a couple of days.
Metrics before and after
| Metric | Before implementation | After |
|---|---|---|
| Portion weight outside tolerance | 4.1% of items | 0.5% |
| Packaging defect reaching the consumer | 1.7% of batches | 0.2% |
| Batches sent for re-inspection | 19% | 3% |
| Decisions confirmed manually by the operator | 100% | 8% |
Re-inspection of a batch ceased to be the norm: the line rejects disputed cells immediately, and the operator only reviews cases where the model did not reach the confidence threshold.
The measurement log for each item satisfies the requirements of HACCP — batch traceability is restored without manual excerpts.
How does in-line control differ from spot batch inspection?
Spot control answers a question about the batch, while in-line control answers about each item. A lab technician takes a sample according to regulations, and the decision is made statistically: as long as the sample is within tolerance, the batch is considered good.
A defect that did not make it into the sample goes to the warehouse together with the batch — the error scales to the entire volume.
In-line, the decision is made on every unit: the system compares the item with a reference and rejects the specific product, not the batch. Hence the different cost of a missed defect.
A spot scheme, when it misses, means a recall or rework of the entire batch; in-line control cuts out defects pointwise, and the withdrawal volume is measured in units.
The burden on personnel also differs. A spot scheme relies on a person: sampling, visual assessment, logging — that is time, fatigue, and a subjective threshold that inevitably drifts by the end of a shift.
In-line control holds a single criterion throughout the shift, and the operator gets involved only in disputed cases that the system flags separately.
Comparison of modes does not boil down to which is better. Microbiology, moisture, and composition are determined by the laboratory — here spot sampling remains primary.
In-line control covers what a human does not have time to inspect piece by piece: geometry, color, foreign inclusions, and packaging completeness.
In practice, the modes complement each other: in-line control cuts out defects at the moment of production, and sampling confirms process stability. If the cost of a defect is higher than the cost of stopping the line, spot batch inspection becomes insufficient.
Technical concerns: answering substantively
A model in a workshop does not operate in a laboratory: raw materials arrive in different batches, light changes between shifts, recipes are adjusted on the fly. Hence the main concerns of engineers — we address them substantively.
The raw material variety or supplier changed — what happens to the model?
Robustness rests on class features rather than a specific batch: color, texture, and the nature of reflection in the control spectral bands are comparable for the same product.
With a noticeable shift, we add the batch to the sample and retrain the model; the previous classes remain in the set so as not to lose recognition.
Workshop lighting varies — is that critical?
Imaging is set up so that the frame does not depend on ambient light: dedicated illumination in the control zone, fixed exposure, calibration against a reference at the start of a shift.
For training, we take frames from different shifts and line positions — then lighting fluctuations become noise rather than a control error.
How much data is needed to start?
The sample size is determined by the number of classes and the diversity of defects, not by the "the more, the better" principle.
We start with a trial run on one line: annotate real deviations, see where the model errs, and add exactly those cases — that is shorter than collecting a large set blindly.
Who supports the model after launch?
Support is part of the work, not an option: we monitor the share of missed defects and divergence from laboratory checks, retrain on new batches when data drifts, record the model version, and maintain rollback capability. We compare metrics with real line behavior regularly, not from a one-off report.
Let's discuss the quality control task on your line
To come back with a substantive assessment and implementation plan rather than average industry figures, we need to discuss the task using your material: look at the line and the data.
Below is the minimum set we capture at the start; without it, a pilot on one line turns into guesswork.
- What we control: defect, color or texture deviation, foreign inclusion, underweight — and which decisions the operator makes manually.
- Data: cameras in areas, scales, metal detector, laboratory samples, defect logs — for what period they exist and in what form.
- Annotation: how many "good/reject" examples are confirmed by a reference expert and how rare defect classes are not lost.
- Recipes and modes: which incoming raw material indicators affect output and where tolerances are tight and where they can vary.
- Pilot on one line: we select the area with the greatest losses and run the model in operator prompt mode.
- Acceptance criteria: share of missed defects, false triggers, response time to a deviation — we fix these before the start.
- Integration and access: where we return the result (control loop, accounting system, report), who owns the system.
Next, we prepare data according to this checklist, determine the pilot area, and compare the result with current checks. Describe the task — we will come back with an assessment and plan: stages, data preparation requirements, and roles on both sides.
