Data Transformation Configuration for Parsing in 1C-Bitrix

Our data transformation and parsing for 1C-Bitrix ensures proper catalog data normalization and import validation, making filters work correctly. A common scenario is: products load, but filters don't work, prices display with currency, and categories are scattered across the tree. Without a transformation layer between the parser and importer, data enters 'as is' — and breaks the catalog structure. For example, a price string with extraneous characters ends up in the price field, and sorting by price stops working. We configure end-to-end transformation and guarantee that after import the catalog remains clean and consistent. We'll evaluate your project in 1 day — just contact us.

Official 1C-Bitrix documentation emphasizes the importance of pre-validation of data during import.

What transformations are needed during parsing?

Text normalization — standard cleaning from extra characters and unification to a single case. For example, SHURUPOVERT BOSCH GSR 18V becomes Шуруповёрт Bosch GSR 18V. We use mb_convert_case() with a whitelist for SKUs and brands. We also remove HTML entities (& → &) and non-breaking spaces. Name errors cause up to 30% of product rejection during manual verification.

Price normalization — extract the number from a price string → numeric value (e.g., 1299.00). Regex: preg_replace('/[^\d,.]/', '', $price) followed by comma replacement. Convert currency using the Central Bank rate or a fixed value. Round to two decimals.

Property normalization — units of measurement are parsed with /^([\d.,]+)\s*([а-яА-Яa-zA-Z]+)$/u, boolean values are converted to Y/N, list properties are mapped to XML_ID via a configuration array.

Image processing — often parsers download images with duplicate names. We compute the md5 hash of each file and compare with already uploaded ones. If the hash matches, the image is not added. This prevents file storage from growing and speeds up import. We also configure automatic thumbnail generation via Bitrix Imaging.

How to map categories without risk?

The source category structure rarely matches the information block sections. We use a mapping table:

$categoryMap = [
    'Электроинструмент/Дрели'          => 15,
    'Электроинструмент/Шуруповёрты'     => 16,
    'Ручной инструмент/Отвёртки'        => 22,
];

$sectionId = $categoryMap[$externalCategory] ?? DEFAULT_SECTION_ID;

For new categories not in the mapping, products are placed into an 'Uncategorized' section with a log entry. Automatic section creation is dangerous — one error in the source data and garbage sections appear in the catalog. Our experience shows this approach reduces catalog maintenance time significantly.

Validation is mandatory before import

Even after transformation, data may contain errors: empty name, invalid XML_ID, negative price. Validation is a separate pipeline stage between transformation and import:

All rejected elements are saved in the parser_rejected table with the reason. This allows error analysis without re-importing.

What's included in turnkey transformation setup?

• Audit of source data and identification of typical issues.
• Design of rule chains for each field.
• Implementation in PHP using \Bitrix\Main\ORM and events.
• Integration with existing parser or importer.
• Testing on a real product sample (at least 1000 items).
• Documentation of rules and instructions for making changes.
• 12-month warranty on code and support when the source changes.

The rule configuration allows flexible logic changes without modifying code. Example rule configuration:

$transformRules = [
    'NAME' => [
        ['type' => 'trim'],
        ['type' => 'mb_title_case'],
        ['type' => 'max_length', 'value' => 255],
    ],
    'PRICE' => [
        ['type' => 'extract_number'],
        ['type' => 'multiply', 'value' => 1.2],  // Markup 20%
        ['type' => 'round', 'value' => 2],
    ],
    'PROPERTY_WEIGHT' => [
        ['type' => 'extract_number'],
        ['type' => 'convert_unit', 'from' => 'kg', 'to' => 'g'],
    ],
];

Comparison: configuration vs hardcoded logic

The configurable approach allows changing transformation without involving a developer or modifying the parser. For catalogs with frequent source changes, this is significantly cheaper in the long run. Such setup typically pays off in 2–3 months by saving on manual processing. Our configurable approach allows changes up to 10 times faster than hardcoded logic.

Process and timeline

1. Analytics — study source structure, identify critical points (1–2 days).
2. Rule design — create configuration for each field type (1–2 days).
3. Implementation — write transformer and validator classes (2–3 days).
4. Testing — run on real data, fix errors (1 day).
5. Deployment — deploy on production server, document (1 day).

Estimated timeline — from 5 to 8 business days. Final cost is calculated individually and depends on the number of fields and transformation complexity. Get a consultation on transformation setup today.

Why we do not recommend automatic section creation?

Once we witnessed a parser creating 500 sections due to an incorrect category field in the source. Recovery took a week. Our approach — only manual or semi-automatic mapping with administrator notification. This increases reliability and maintains catalog cleanliness.

Our experience — 10+ years working with 1C-Bitrix and over 500 projects in integration and parsing. We have a 98% success rate in filter functionality after transformation. Official Bitrix documentation recommends using information blocks version 2.0 and tagged caching for large catalogs. We follow these recommendations and guarantee that your catalog will run fast and without failures.

For example, our clients typically save $300–$500 per month in manual data correction after implementing our transformation rules. Contact us — we'll evaluate your project and prepare a transformation configuration in 1–2 days. Order transformation setup and forget about import problems.

Parser Development for 1C-Bitrix: Where to Start?

XMLReader, not SimpleXML — the choice of tool determines the project's fate. SimpleXML loads the entire XML into memory, and with an 800 MB supplier file, PHP will crash with a fatal error on a 512 MB limit. XMLReader processes streamingly, node by node, consuming 20–30 MB — 30 times more efficient. This detail starts any parser development for Bitrix. With over 10 years of Bitrix development and 50+ parser projects delivered, we know the pitfalls. Contact us to start your parser development today.

What Problems Does Parsing Solve?

• Primary catalog filling — 15,000 cards with descriptions, characteristics, photos. Manually, that's three months of content manager work; a parser takes a week with debugging.
• Competitor price monitoring — collecting data from Ozon, Wildberries, competitor sites. A competitor drops the price on a hot item — you find out in two hours, not two weeks.
• Supplier aggregation — five price lists in different formats (CSV with CP1251, XML in CommerceML, Excel with merged cells) become a single catalog with a unified property system.
• Card enrichment — pulling characteristics, instructions, 3D models from manufacturer sites. Without this, a product card is an SEO empty shell.
• Assortment update — products missing from the supplier feed are deactivated via CIBlockElement::Update($ID, ['ACTIVE' => 'N']). New ones are created. The catalog stays synchronized.

What Tools Do We Use in Parser Development?

Static websites — PHP (Goutte, Symfony DomCrawler) or Python (Scrapy, lxml). Speed: 50–100 pages/sec. Sufficient for catalogs without JS rendering.

SPA and dynamic websites — Puppeteer or Playwright. Infinite scroll, AJAX filters, lazy-load images — headless browser handles it all. Speed drops to 1–10 pages/sec, but there is no alternative: data exists only after JavaScript execution.

Supplier files:

• Excel (XLS, XLSX) — PhpSpreadsheet. Beware of merged cells and formulas — they break automatic mapping.
• CSV — fgetcsv() with correct encoding. Suppliers love CP1251, BOM in UTF-8, and semicolons instead of commas. All need detection and handling.
• XML/YML — XMLReader for large files, SimpleXML for feeds up to 50 MB.
• CommerceML — standard exchange format with 1C. We parse import.xml and offers.xml, map to information block structure.

API — Supplier REST endpoints, marketplace APIs (Ozon Seller API, Wildberries API). We work within rate limits, handle pagination.

How Is the Auto-Population Pipeline Structured?

Four stages. Each can break in its own way.

1. Collection. Parser crawls sources via cron schedule. Raw data goes to an intermediate table — not directly into b_iblock_element. Log everything: pages visited, elements parsed, where we got 403 or timeout. Without logs, debugging a parser is like fortune-telling.

2. Normalization. Main work here:

   • Clean HTML tags, extra spaces, Unicode garbage
   • Units: "mm" → "mm", "millimeters" → "mm", "миллиметр" → "mm"
   • Map supplier categories to Bitrix information block sections. One supplier has "Notebooks", another "Notebooks and tablets", third "Laptops" — all into one section
   • Deduplication by SKU, EAN/GTIN. One product from three suppliers should not appear three times

3. Load into Bitrix. Via CIBlockElement::Add() for new elements, CIBlockElement::Update() for existing. Images: download, resize via CFile::ResizeImageGet(), convert to WebP. Properties via CIBlockElement::SetPropertyValuesEx(). SEO meta via \Bitrix\Iblock\InheritedProperty\ElementValues. SEF URLs generated from name transliteration.

4. Update. Key point — not overwrite manual edits by content manager. Update only price, stock, activity. Description and photos manually edited are flagged with UF_MANUAL_EDIT property and skipped during import. Products missing from feed are deactivated, not deleted.

Why Is Competitor Price Monitoring Necessary?

A separate subsystem with its own specifics:

Result — dashboard: your product vs competitors, price history, trends. The manager sees where to raise price without losing position, and where to react.

CSV/XML Import Module: Customization for Your Format

For supplier files — custom module with admin panel:

• Configurable mapping: "column B in file → BRAND property of information block"
• Auto-detect encoding (CP1251, UTF-8, UTF-16) via mb_detect_encoding() with validation
• Download images from URL with queue — to avoid channel saturation
• Incremental update by row hash: row changed — update, no — skip
• Cron schedule, report: created 145, updated 892, errors 3 (with details)

Large files: CSV processed in batches of 1000 rows via fgetcsv() (10 times faster than row-by-row), XML streamed via XMLReader, background execution via Bitrix agent queue — no PHP timeouts.

Legal Aspects to Consider

• robots.txt — respect it. Crawl-delay — comply.
• Request frequency — 1–2 per second, no more. Don't DDoS someone else's site.
• Manufacturer content — use it. Unique author texts — don't copy.
• Personal data — don't collect.

What Is Included in a Turnkey Parser Development?

How We Work and Deadlines

1. Prototype — parser for 1–2 sources in 2–3 days. Assess data quality, pitfalls (Cloudflare protection, captcha, dynamic loading).
2. Development — full pipeline: parser → normalization → import into Bitrix → admin panel for management.
3. Testing — run on full catalog volume, check edge cases (empty fields, malformed HTML, broken images).
4. Launch — configure cron, error monitoring via Telegram bot.
5. Support — competitor changed layout? Update CSS selectors in parser.

Get in touch for a free consultation — we will analyze your data sources and propose the optimal parser architecture. Request a project assessment today and get a fixed deadline. We guarantee stable parser operation and full support throughout the usage period.