AI Planogram: why display diverges from the layout
Display diverges from the approved layout not because someone is slacking off, but because the layout and the shelf live on different time scales.
A planogram fixes a revision as of the approval date, while the shelf changes daily: a product runs out, a batch arrives in new packaging, an employee assembles the display from memory during peak hours.
AI Planogram addresses not the question of “who is to blame,” but display control with a regularity unattainable by manual walkthroughs.
Sources of divergence from the layout repeat from location to location. Look-alike products are visually similar — an adjacent flavor or a different weight takes the wrong place in the row.
A change of supplier or packaging redesign breaks the width calculation, and the layout retains a position that is no longer on the shelf. After a sale, “holes” remain that are filled with random products instead of a substitution rule.
Add seasonal rearrangements — and part of the sales floor operates under an outdated revision that no one remembers.
A manual walkthrough costs more than it seems. Walking one category takes tens of minutes; a full walkthrough of a location takes half a shift, so checks are done once a week or in response to a complaint.
Between checks, divergence goes unnoticed: shelf structure drifts, impulse zones lose sales, and the report captures the inspector’s impression rather than the fact. There is no way to compare the shelf state with what it was a month ago.
Image-based display detection removes this discreteness: the algorithm matches the frame with the layout by shelf geometry and positions, and shelf-check automation turns a one-off walkthrough into a regular report with evidence. Let’s break down how this works.
AI Planogram: measurable effect for a retail chain
The effect of display control is measured not by the fact of implementation, but by three numbers: the time to walk the sales floor, the share of detected divergences, and the spread of these indicators between locations.
A manual walkthrough of one store takes 40–60 minutes, and by the end of the shift the inspector looks at shelves noticeably less attentively — some errors remain unnoticed.
The AI system walks the same floor in 5–7 minutes: an employee photographs the shelves, the model calculates the rest, and the speed of floor checking stops depending on the number of categories in the assortment matrix.
Detection is built on an open model from the YOLO family, fine-tuned on images from a specific chain and its product groups: it recognizes the product, the number of positions in a row, the order on the shelf, and the presence of a price tag.
The model does not guess the brand by packaging color — it compares the display geometry with the reference and returns a list of divergences linked to the shelf.
Therefore, the reduction of display errors comes through completeness rather than strictness: the share of detected deviations grows by roughly a third compared with manual walkthroughs, and disputable cases go to human confirmation rather than an automatic penalty.
The main effect is repeatability. The same set of rules is applied across all locations, and chain comparison stops depending on who specifically conducted the merchandising audit.
When AI planogram control is built into the daily routine, divergence is recorded on the day it appears, not at a monthly check: correction costs a shelf adjustment, not a repeat supervisor visit.
Display control modes: photo capture, sales floor cameras, hybrid
The control mode determines not only the source of the frame but also all subsequent processing: display capture during a walkthrough, permanent cameras on the sales floor, or a combination of both sources.
This affects the frequency of reconciliation with the planogram, the robustness of recognition to angle and occlusions, and how suitable the image is as evidence for a supplier.
The choice runs into three constraints — sales floor area and number of product locations, required check frequency, and shooting conditions: light, angle, people in the frame.
| Mode | When it fits | Constraints |
|---|---|---|
| Display capture during walkthrough | Stores with infrequent reconciliation, frequent rearrangements, need linkage to employee and time | Human factor: missed zones, varying angle, interval between checks |
| Sales floor cameras | Large floors where reaction during the day matters, not after a walkthrough | Dead zones, occlusion by customers, costs of installation and shelf tracing |
| Hybrid scheme | Mixed assortment: top shelves under observation, the rest via walkthrough | Harder to merge two sources and label a reference for each angle |
Processing also differs. During a walkthrough, on-device computation gives a result immediately on site but is limited by frame quality; with stationary cameras, processing goes to a server where history can be kept and frames compared with each other.
The hybrid scheme takes from both modes: continuous background coverage of key zones plus periodic capture of the entire display — and this is what we most often design as the primary shelf control mode.
Implementing display control: stages of engineering work
Implementation of display control is divided into stages, and each is closed with a verifiable artifact: a process description, an annotated set of frames, a pilot report. This order removes the risk that the system works confidently in a demo and falls apart in a store.
Project stages: from process audit to pilot
- Process audit. We look at how the display is currently reconciled with the planogram: who photographs the shelves, how often, where divergences are recorded, which categories produce the most deviations.
- Reference data. We collect and label images: a correctly assembled shelf and typical violations — shift, emptiness, wrong product.
- Setup for your planograms. Recognition is trained on your categories and shelf format, not on an abstract set.
- Category pilot. Launch at a limited number of locations, comparison with manual checking, calculation of the share of correct detections.
- Result acceptance. We agree on the quality threshold and escalation rules: what becomes an employee task, what is considered acceptable.
- Handover to operation. Regulations, staff instructions, quality monitoring, and team training.
After that, the solution operates without our constant involvement: new planograms are loaded as standard, quality is tracked by metrics, and when a category changes, the model is fine-tuned on new frames instead of restarting the project.
Project artifacts: labeling, detection model, regulations
A demo works confidently on prepared frames, so the value of the project is determined by what physically remains in the customer’s environment after handover.
We deliver not a closed system “as is,” but a set of artifacts that your team can use to change rules, fine-tune the model on new data, and review disputable detections without our involvement.
- Data labeling. Reference frames by equipment type and angle, a dictionary of defect classes, labeling rules for disputable cases — where a deviation is considered a violation and where it is a tolerance.
- Detection model. Trained weights, launch script, and metrics report: precision and recall separately for each class, share of false positives on the validation set.
- Matching rules. Linking detected objects to planogram positions, confidence threshold for automatic correction and for manual review by an operator.
- Reaction regulations. Who receives a remark and in what form, within what time it is closed, under what conditions escalation is triggered, and how the result is recorded.
- Base for fine-tuning. Requirements for capturing new frames and the procedure for adding difficult examples to the training set — so that quality grows on your data.
- Documentation. Description of the operating scheme, instructions for launching and updating the model, and a map of responsibility zones between your team and the supporting contractor.
- Handover package. Accesses, configurations, detection log, and incident reviews for the trial operation period.
Acceptance is performed for each item separately: until an artifact is reproducible on your side without our involvement, the stage is not considered closed.
How display control changed in the chain: metrics before and after
How it was before implementation
The pilot was launched in a retail chain of one format — several dozen locations with the same layout, so that comparison was based on comparable data.
Control relied on manual walkthroughs: an employee walked the floor with the planogram and marked deviations by eye — about 40 minutes per location.
Two to three weeks passed between recording a violation and rechecking, so some deviations disappeared on their own, and some appeared again.
What happened after automation
Automatic checking was built into the same process: an employee photographs the shelf according to the regulations, the system matches the image with the planogram, and returns a list of deviations linked to product positions. Inside the chain, the workflow was called Смарт-выкладка.
- Divergences from the planogram: manual walkthrough found 18% of violations, automatic checking — 34% on the same shelves. The difference is rearranged blocks, wrong package size, understocking by depth.
- Location check time: 40 minutes → 7–10 minutes, including capture after correction.
- Recheck cycle: two to three weeks → two to three days.
- Share of divergences that remained unclosed during recheck: 45% → 12%.
The growth in findings is not a deterioration of the display but a change in control sensitivity. We compared on the same locations and one capture regulation: first a month of manual walkthroughs, then a comparable period of automatic checking.
How does photo-based display control differ from video analytics?
Both approaches start with a frame of the shelf, but then diverge. Photo analytics receives a discrete set of images along a defined walkthrough route, while video analytics receives a continuous stream from fixed cameras.
This leads to different requirements for the sales floor, recognition accuracy, and the cost of changing the display scheme.
In a static image, geometry is repeatable: the same capture point, a known angle, predictable exposure. Detectors from the YOLO family produce robust results on such frames, a disputable zone can be rechecked by rerunning, and only divergences are sent to the operator.
Video wins with something else — it catches the moment: a price tag swap, an empty spot on the shelf, movement of a product between zones.
Video has stricter capture constraints. Cameras are mounted at an angle, some shelves are occluded by customers, distant rows lose detail, and evening lighting changes packaging color.
Stream processing partially smooths this out — the decision is made based on accumulated observations — but it requires deduplication: without it, the same empty shelf produces an avalanche of detections.
In practice, the choice is this: photos — for auditing planogram compliance and inventory accounting, video — for events on the sales floor and reaction in real time.
When both are needed, we build a combination: cameras raise alerts, and scheduled walkthroughs with images provide the reference and evidence base for disputable cases.
Common technical doubts before launching the system
Doubts before launch are usually engineering-related: whether there is enough data, what to do with similar packaging, how often to recapture references, and whether the sales floor will require equipment replacement.
We test this in a pilot zone — several shelving units and one display format, where the system’s behavior is visible under real lighting and real customer flow.
What amount of data is needed at the start?
There is no need to label the entire assortment. Reference images for each display block and a few hundred frames of the real sales floor are enough — after that, the model is fine-tuned on the divergences it finds itself. The narrower the pilot zone, the faster it becomes clear whether the capture quality is sufficient.
How does the system distinguish similar packaging?
We look not at the silhouette but at a set of features: package profile, order of neighboring units, color zones of the label, position relative to dividers and price tags.
Positions that are objectively indistinguishable in the frame go to the “requires confirmation” list rather than a false positive. We mark such categories as difficult and configure a separate sensitivity threshold for them.
What happens when layouts are updated and during the season?
The layout is stored as a version with an effective date, not as a single file. A new planogram is loaded as a separate version, and the previous one is retained — divergences can be compared and you can roll back.
Seasonal extensions — pallet displays, promo zones, temporary inserts — are described by tolerance rules so they are not read as violations.
What equipment will be required on the sales floor?
Usually the cameras already installed on the sales floor are enough, plus capture along the walkthrough route. What matters is not so much the optics as zone coverage and a stable capture point: we calculate how many shelves one pass covers.
With uneven lighting or a deep shelf, we add a capture point — this is what gets clarified during the pilot.
Let’s discuss the task: where to start and what is required
The estimate and plan come not from a questionnaire but from analyzing the current process: how the display is checked today, who records violations, what has already been attempted to automate.
Therefore, the first step is to discuss the task and collect the inputs that determine both the scope of work and the implementation timeline.
What is required from you: planograms in a working format — tables, PDF, or an export from a merchandising system — with mandatory and variable positions marked.
Plus sales floor capture from the angles where the system will operate, and a history of real violations: it shows which display errors are critical, and setup becomes more precise.
Next, the work proceeds in steps:
- Review of inputs and task boundaries: categories, shelf types, what the system should do and what it should not do.
- Data audit: whether capture and labeling are sufficient for detection to work robustly, rather than guessing from one lucky example.
- Acceptance criteria: what share of detected violations and what false positive rate are considered normal for your chain.
- Pilot on one category or group of locations: we measure accuracy on real walkthroughs and show what the system misses and why.
- Fine-tuning for the chain: we close the divergences that surfaced in the pilot and add new shelf types and capture conditions.
- Solution handover: documentation, regulations for handling signals, team training, accesses, and permission separation.
- Post-launch support: we review disputable detections and update the model as the display changes.
We state the implementation timeline for the entire chain after the pilot — it depends on the number of categories and locations, not on the model size.
Describe the task and the current display control process — we will come back with an estimate, a plan, and the pilot boundaries.
