Product Parser: Why the Catalog Diverges from Suppliers
The catalog diverges from suppliers not because someone is doing a poor job, but because of how the process is structured: product cards are assembled manually from price lists, emails, and spreadsheets where each supplier has its own field logic.
One writes "weight 1.2 kg," another writes "1200 g," a third puts the mass in the description rather than in the specification.
A product parser removes this manual work: it pulls data from the source, brings it to a unified schema, and delivers ready-made cards rather than raw text.
The cost of manual entry shows up not in labor effort but in data quality. Inconsistency in specifications breaks filters: a buyer sets a range by weight or power and cannot find a product that physically fits.
Comparison within a category stops working because values sit in different formats and units. Export to marketplaces fails validation — the platform expects a numeric field but receives a string with a unit of measurement inside it.
Errors in the catalog accumulate quietly. While the assortment is small, they are caught by eye; at hundreds of SKUs and regular price list updates, manual reconciliation cannot keep up with the changes, and divergence from the supplier becomes a constant background.
Specification normalization closes this at the input: units are brought to a single system, values are split into number and measure, field synonyms are mapped to a common reference dictionary, and duplicates are collapsed by SKU and name.
Collecting product descriptions, meanwhile, is not reduced to copying text. Descriptions are cleaned of the supplier's service notes, brought to a unified structure, and checked for completeness — so that the card passes the platform's requirements without manual rework.
We break down how this is covered: from analyzing sources and the normalization schema to regular catalog updates and divergence control.
What data normalization provides and how approaches compare
Data normalization is the process of bringing heterogeneous descriptions into a single format of characteristics: each product card gets a predictable set of fields, units of measurement, and values from a reference dictionary.
Without it, a catalog assembled from multiple sources turns into a set of incompatible records: "weight 1.2 kg", "1200 g", and "1.2kg" are three different values for search and filtering.
After normalization, the share of cards with a complete set of key fields grows from 40–60% to 95%+, and the share of manual edits when updating the catalog drops severalfold.
What changes in catalog quality
A unified format of characteristics delivers three measurable effects. Filters and product comparison start working across all items, not just those where a manager manually filled in the fields.
Repeat imports stop creating duplicates: matching is done by a normalized key rather than by the title string. Card automation speeds up — new items enter the catalog with ready-made attributes instead of a queue for manual review.
How data collection approaches compare
| Approach | When it fits | Limitations |
|---|---|---|
| Ready-made template-based parser | Stable source markup, a single site | Breaks when markup changes, does not scale to dozens of sources |
| Custom parser with extraction rules | Many sources, on-the-fly normalization needed | Requires maintaining rules for each source |
| Manual entry and proofreading | Small catalog, unique items | Does not scale, update time grows |
| Hybrid: parsing + manual review of disputed fields | Critical characteristics where an error is costly | Requires a process for resolving discrepancies |
The choice is determined not by catalog size, but by how often sources change and how costly an error in a characteristic is. For rapidly changing sources, extraction rules are kept separate from the code — then adapting to new markup takes minutes instead of a release.
Schema.org provides a vocabulary for describing products and characteristics, so normalized data fits into the markup without reinventing the field structure.
How we implement a parser: stages of engineering work
Implementing a product parser begins not with code, but with an inventory of sources: which marketplaces and supplier catalogs provide data, in what form the description arrives — HTML markup of a product page, an XML export, or a response from an internal service — which fields are mandatory for normalization (SKU, brand, specifications, units of measurement), and how often the page structure changes.
At this stage we identify failure points: captchas, rate limits, unstable selectors, duplicate products across sources.
- Audit of data sources. We compile a list of marketplaces, analyze the structure of product pages and exports, and determine which fields are extracted directly and which require matching. The result is a source map with update frequency and access rules.
- Designing the collection schema. We choose the method of obtaining data for each source: markup parsing, working with exports, or calling an internal interface. We build in address rotation and pauses between requests in advance so that collection is not blocked on the marketplace side.
- Setting up collection and normalization. We write field extraction and bring specifications to a unified reference: brands, units of measurement, categories. Products are matched across sources by SKU and name, and duplicates are collapsed.
- Quality check. We run a sample of product pages and compare the result against a reference: the share of correctly parsed fields, the number of gaps, conflicts during matching. We set up alerts for a drop in the share of successful parses — a markup breakdown is visible immediately, not a week later.
- Stabilization and monitoring. We add retries on failures, request logging, and collection speed control. We fix the reference update schedule so that new categories do not break normalization.
- Handover of the solution. We deploy the parser on your infrastructure or ours, hand over access credentials, source configurations, and normalization rules.
- Documentation and training. We hand over a description of the data schema, instructions for adding a new source, and a procedure for responding to failures — the team can develop the solution without us.
Each stage ends with a verifiable result: a source map, a collection schema, a quality report, a working pipeline. Describe your task — we will come back with an estimate and a plan for your set of marketplaces.
How does catalog parsing differ from a one-time export?
A one-time export is a snapshot of the catalog at a specific moment: a script goes through a list of pages, collects the cards, and returns a file.
Catalog parsing is structured differently — it is a continuously running pipeline where a normalization layer sits between the source and your database, rather than just a collector. The difference shows up not in the volume of data, but in what happens to it afterward.
With a one-time export, you get a static set: prices, availability, and specifications are fixed as of the collection date. A week later, some items have been discontinued, some have changed parameters, and new ones have appeared — and the file begins to diverge from reality.
As long as the divergence is tolerable, it is used; when it becomes noticeable, the export is run again, and the cycle repeats.
Catalog parsing removes this problem: regular updates run on a schedule, changes arrive incrementally, and maintaining currency becomes a standard mode rather than a separate task.
The key difference is in normalization. With one-time collection, it is enough to pull the values as they are: a person reviews them afterward.
With continuous updating, the data must fit into a single schema — units of measurement, size formats, and synonyms for colors and materials are brought to a common form at the input.
Otherwise, each new export will breed duplicates and conflicts, and the catalog will quickly turn into a dump.
The parser's operating modes also differ. A one-time export is a single run over a fixed list. Catalog parsing works in several modes: a full crawl for initial population, an incremental one — only for changed items, and a targeted one — for specific cards or sections.
Plus tracking of disappeared items: they are not silently deleted but flagged, so that the history remains.
The choice of mode is determined by the task. If you need a slice for analysis or a one-time comparison, an export is enough.
If you need a living catalog that feeds a storefront, search, or recommendation system, without a continuous pipeline with normalization it will fall apart within a matter of weeks.
Case study: how normalization accelerated product listing on marketplaces
The task was typical for e-commerce: a supplier delivered a price list in a format that could only be pushed to marketplaces manually.
A manager would open a card, transfer characteristics from the supplier's table into the platform's form, and verify units of measurement and category.
At a volume of several thousand items, this created a publication queue of weeks and a steady percentage of errors in attributes.
What was changed
We built a product parser that pulls descriptions and characteristics from supplier sources and brings them to a unified model: attribute type, unit of measurement, allowed values, and linkage to the platform's category.
Normalization was not done "by eye" but according to a synonym dictionary and rules — "diagonal 15.6"" and "15,6 inches" were turned into a single value rather than two different attributes.
Then the card was assembled automatically: description based on the category template, characteristics laid out across fields, mandatory platform attributes filled in. The manager checked the finished result instead of typing it from scratch.
Metrics before and after
| Indicator | Before | After |
|---|---|---|
| Manual work per card | 12-15 minutes | 2-3 minutes for review |
| Share of errors in attributes | about 18% | below 3% |
| Time to list a new item | 5-7 days | within a day |
| Catalog coverage | selectively, by priority | the supplier's entire price list |
We separately closed out repeat publications: if the supplier changed the price or description, the update went into the card without manual reassembly. Previously, such edits simply got lost in the queue.
Schema.org was used as a reference for the attribute structure — this provided predictable markup and simplified verification against platform requirements.
What the parsing case study delivered in numbers: manual work reduced severalfold, the share of errors dropped, and listing products on marketplaces stopped being a bottleneck. The team switched from transferring data to working with the assortment.
Compatibility, access, and rollback concerns — let's discuss your task
A product parser almost never lives in a vacuum: descriptions and specifications need to be reconciled with an accounting system, catalog, or price list, which means deciding how data gets in and what happens when there's a mismatch.
In practice, the main concerns boil down to three: will we break the current accounting, what access will be needed, and can changes be rolled back if normalization doesn't go as expected.
What if the accounting system is closed and doesn't expose data directly? We work through exports in an agreed format or an intermediate layer that writes to the system using its standard tools.
We don't write directly to the production database — first we populate a test environment, verify the result, and only then enable the exchange.
What access is needed and how safe is it? We request the minimum necessary set: read access to the catalog, write access to a separate area for normalized cards, access to the exchange queue or log. We store secrets in a secure vault, not in code, and hand them over to you upon delivery.
How are changes rolled back? Each batch of data is tagged with a run identifier, so a rollback means deleting or deactivating a specific batch, not restoring the entire database. Before the first production run, we fix a return point and test the rollback on the test environment.
What if some cards are normalized incorrectly?
Disputed records don't go into the catalog automatically — they go into a review queue with the source and the reason the rule was triggered. You can see which rule fired and on which fields, and you can fix it without rewriting the parser.
What the client receives:
- parser configuration and specification normalization rules;
- the exchange scheme with the accounting system and a description of the formats;
- configured access policies and secrets handling;
- a procedure for rolling back a data batch with verification on the test environment;
- run logging and a discrepancy report;
- an operations manual and handover of the solution to your team.
Describe your task — which accounting system, where the data comes from, and what you've already tried. We'll get back to you with an estimate and a work plan.
Parser implementation timeline by stage
Timelines depend not on catalog size, but on how predictable the sources are. A stable API or feed delivers data right away — the collection stage takes days. Markup that changes from page to page requires a separate layer of rules and stretches out preparation.
Below are approximate timelines for a typical project: three to five sources, normalization into a single schema, regular updates.
The stages partially overlap: the data schema is refined while collection is underway, and normalization starts on the first collected cards.
That is why the total timeline is not equal to the sum of the table rows — for a typical project it is three to six weeks from start to a working, updatable pipeline.
We separately set aside time for sources with protection against automated requests: there, selecting the request mode and pauses takes longer than parsing the markup itself.
If a source only delivers data through a browser, we add an emulation stage — it runs in parallel with collection and does not critically extend the overall plan.
Describe your sources and the fields you need in the card — we will get back to you with an estimate and a stage-by-stage plan.
