Auto-Populating Product Attributes from External Sources in Bitrix

Our company is engaged in the development, support and maintenance of Bitrix and Bitrix24 solutions of any complexity. From simple one-page sites to complex online stores, CRM systems with 1C and telephony integration. The experience of developers is confirmed by certificates from the vendor.
Showing 1 of 1All 1626 services
Auto-Populating Product Attributes from External Sources in Bitrix
Medium
~1-2 weeks
Frequently Asked Questions

Our competencies:

Development stages

Latest works

  • image_website-b2b-advance_0.webp
    B2B ADVANCE company website development
    1361
  • image_bitrix-bitrix-24-1c_fixper_448_0.webp
    Website development for FIXPER company
    949
  • image_bitrix-bitrix-24-1c_development_of_an_online_appointment_booking_widget_for_a_medical_center_594_0.webp
    Development based on Bitrix, Bitrix24, 1C for the company Development of an Online Appointment Booking Widget for a Medical Center
    695
  • image_bitrix-bitrix-24-1c_mirsanbel_458_0.webp
    Development based on 1C Enterprise for MIRSANBEL
    834
  • image_crm_dolbimby_434_0.webp
    Website development on CRM Bitrix24 for DOLBIMBY
    733
  • image_crm_technotorgcomplex_453_0.webp
    Development based on Bitrix24 for the company TECHNOTORGKOMPLEKS
    1076

Properly filled attributes are the foundation of the faceted filter in 1C-Bitrix. Without them, the bitrix:catalog.smart.filter component does not display parameters, and the buyer cannot narrow down their choice. We have repeatedly seen catalogs where all attributes are dumped into one text property via line breaks—in such a filter, half the items are lost. Incorrect or missing attributes lead to a 20–30% drop in conversion—the buyer cannot filter the desired models. Automating import from external sources solves this problem but requires a well-thought-out schema and data normalization. Over 5 years, we have implemented more than 50 projects for catalog automation on Bitrix and know the typical bottlenecks. Switching to auto-fill pays for itself on average in 2–3 months, significantly saving team resources. The implementation cost depends on the catalog size, but in most cases it pays off through reduced manual labor.

Why Attribute Automation Is Critical for an Online Store

Manual filling of 1000 products takes 30–40 working days and yields 10–15% errors. Attribute errors mean not only lost sales but also product returns due to mismatched expectations. Automation reduces time to 7–12 days and errors to 1%. Comparison of approaches:

Parameter Manual Fill Auto-Fill
Time for 1000 products 30–40 working days 7–12 working days
Data errors 10–15% of items <1% after setup
Scaling Requires hiring managers No additional resources
Accuracy Depends on the executor Automatic updates

Automation is 3–4 times faster than manual filling and reduces errors by 90%. Data accuracy increases 10-fold—this directly affects buyer trust.

For example, for a home appliance online store, we connected Icecat and the manufacturer's API. Previously, managers spent 2 days a week updating 300 products; after automation, it took 1 hour for monitoring. Errors dropped from 12% to 0.5%.

How to Design a Property Schema for Automation?

Before starting, conduct an audit of the current infoblock properties and identify issues. Typical mistakes in legacy catalogs that need fixing before automation:

  • Attributes are stored in a single text property "Description" via line breaks instead of separate properties.
  • The same property is created multiple times with different CODE, leading to duplicates in the filter.
  • Numeric values are stored in string properties—the interactive filter does not work because it cannot compare values.

For successful auto-fill, a clean schema is required: each attribute is a separate property with the correct data type. Use types: Numbers – type N, categories (brand, color, material) – type L (list values), text descriptions – type S. For numeric attributes, always specify the unit of measurement in the property settings (field UNIT). This simplifies conversion between sources and automatic calculation.

Attribute Sources

  • Icecat XML — the most complete source for electronics and home appliances. Search by EAN via https://icecat.us/api/. Data is structured, attribute names standardized according to international standards. In one project, we connected Icecat for a catalog of 50,000 products—setup took 2 days, after which 90% of attributes were filled automatically. Integration with Icecat is particularly effective for tech products: monitors, laptops, smartphones, where many precise parameters are needed.
  • GS1 / GEPIR — barcode database containing basic product attributes, manufacturer, and valid value ranges. Useful for validating data from unofficial sources.
  • Manufacturer API — if the manufacturer provides partner access to specifications (e.g., official catalogs of Bosch, LG, Samsung). The most reliable source, but requires individual agreements.
  • Site parsing of the manufacturer's site — fallback when there is no API. Parse specification tables carefully, filtering out HTML junk and checking for changes in page structure.

Normalization and Quality Control

The main problem is that the same parameters from different sources may have different names and units of measurement. This leads to catalog desynchronization and reduces buyer trust. Our solution is based on a three-level approach:

  1. Canonical name dictionary — a table property_canonical_map mapping all variants of parameter names to a single form:
    source: 'icecat', source_name: 'Screen Size', canonical: 'display_diagonal', unit: 'inch'
    source: 'supplier_a', source_name: 'Диагональ экрана', canonical: 'display_diagonal', unit: 'sm'
    source: 'supplier_b', source_name: 'Размер дисплея', canonical: 'display_diagonal', unit: 'inch'
    
  2. Unit converter — automatically converts cm to inches (or vice versa) by canonical during import. Supports metric and English conversion systems.
  3. Range validator — numeric values are checked for realism (phone weight 5000 g is a clear error, laptop weight 15 kg is impossible). Configured for each product category respecting real attribute ranges.
Example unit converter code ```php class UnitConverter { public static function convert($value, $fromUnit, $toUnit) { // Implementation via coefficient array } } ```

How to Manage Enum Values for List Properties?

Properties of type L require pre-creating values in b_iblock_prop_enum. During auto-fill, new values constantly appear. A strategy is needed:

  • Auto-creation — a new value is automatically added to the enum. Risk: garbage values from parsing errors.
  • Moderation queue — the new value is placed in a queue, and the manager approves or rejects it. Safer but requires attention.

We recommend a combination: auto-creation with filtering (length 2–100 characters, no HTML, no special characters) + notification to the manager about new values. For each product category, it makes sense to predefine an allowed dictionary of values—this prevents cluttering the reference with duplicates, spelling errors, or non-standard abbreviations.

Incremental Updates

Attributes change less often than prices—once a week is enough for most catalogs. Process optimization:

  • Store a hash of the attribute set for an element: md5(serialize($properties)).
  • When updating from a source, compare the hash—if unchanged, skip the DB write.
  • This reduces the load on the b_iblock_element_property table for large volumes and speeds up the synchronization cycle.

Practice shows that this approach reduces the full update cycle time by 60–70%. For a catalog of 100,000 products, a full update takes 15–20 minutes instead of an hour, which is critical for frequent synchronizations. When updating from multiple sources simultaneously, control the order of applying data: attributes from the priority supplier overwrite data from secondary ones. The priority order is configured in each provider's configuration and can be changed without code modification.

Implementation Process

  1. Audit of infoblock property schema — 1–2 days. Identify duplicates and incorrect types.
  2. Development of providers — 2–4 days. Connect Icecat, GEPIR, parsing if needed.
  3. Creation of canonical name dictionary — 2–3 days. Map source names to infoblock properties.
  4. Setup of unit conversion and validation — 1 day.
  5. Implementation of incremental update — 1–2 days.

What Is Included

  • Audit and restructuring of infoblock property schema.
  • Development of providers for each source.
  • Creation of canonical name dictionary and unit converter.
  • Setup of validation and enum value management.
  • Implementation of incremental update with hashing.
  • Testing on 1000+ products, manager training.
  • Warranty on work — 3 months free support.

Timeline

Stage Duration
Audit and restructuring of infoblock property schema 1–2 days
Development of source providers 2–4 days
Normalizer, canonical name dictionary 2–3 days
Enum value management 1 day
Incremental update, monitoring 1–2 days

Total: 7–12 working days. The cost is calculated individually based on catalog volume and source complexity. Order a preliminary audit of your catalog—we will assess the property schema and data sources. Get a consultation by contacting us.

Source: CommerceML standard, description of data exchange with 1C.

Parser Development for 1C-Bitrix: Where to Start?

XMLReader, not SimpleXML — the choice of tool determines the project's fate. SimpleXML loads the entire XML into memory, and with an 800 MB supplier file, PHP will crash with a fatal error on a 512 MB limit. XMLReader processes streamingly, node by node, consuming 20–30 MB — 30 times more efficient. This detail starts any parser development for Bitrix. With over 10 years of Bitrix development and 50+ parser projects delivered, we know the pitfalls. Contact us to start your parser development today.

What Problems Does Parsing Solve?

  • Primary catalog filling — 15,000 cards with descriptions, characteristics, photos. Manually, that's three months of content manager work; a parser takes a week with debugging.
  • Competitor price monitoring — collecting data from Ozon, Wildberries, competitor sites. A competitor drops the price on a hot item — you find out in two hours, not two weeks.
  • Supplier aggregation — five price lists in different formats (CSV with CP1251, XML in CommerceML, Excel with merged cells) become a single catalog with a unified property system.
  • Card enrichment — pulling characteristics, instructions, 3D models from manufacturer sites. Without this, a product card is an SEO empty shell.
  • Assortment update — products missing from the supplier feed are deactivated via CIBlockElement::Update($ID, ['ACTIVE' => 'N']). New ones are created. The catalog stays synchronized.

What Tools Do We Use in Parser Development?

Static websites — PHP (Goutte, Symfony DomCrawler) or Python (Scrapy, lxml). Speed: 50–100 pages/sec. Sufficient for catalogs without JS rendering.

SPA and dynamic websites — Puppeteer or Playwright. Infinite scroll, AJAX filters, lazy-load images — headless browser handles it all. Speed drops to 1–10 pages/sec, but there is no alternative: data exists only after JavaScript execution.

Supplier files:

  • Excel (XLS, XLSX) — PhpSpreadsheet. Beware of merged cells and formulas — they break automatic mapping.
  • CSV — fgetcsv() with correct encoding. Suppliers love CP1251, BOM in UTF-8, and semicolons instead of commas. All need detection and handling.
  • XML/YML — XMLReader for large files, SimpleXML for feeds up to 50 MB.
  • CommerceML — standard exchange format with 1C. We parse import.xml and offers.xml, map to information block structure.

API — Supplier REST endpoints, marketplace APIs (Ozon Seller API, Wildberries API). We work within rate limits, handle pagination.

How Is the Auto-Population Pipeline Structured?

Four stages. Each can break in its own way.

  1. Collection. Parser crawls sources via cron schedule. Raw data goes to an intermediate table — not directly into b_iblock_element. Log everything: pages visited, elements parsed, where we got 403 or timeout. Without logs, debugging a parser is like fortune-telling.

  2. Normalization. Main work here:

    • Clean HTML tags, extra spaces, Unicode garbage
    • Units: "mm" → "mm", "millimeters" → "mm", "миллиметр" → "mm"
    • Map supplier categories to Bitrix information block sections. One supplier has "Notebooks", another "Notebooks and tablets", third "Laptops" — all into one section
    • Deduplication by SKU, EAN/GTIN. One product from three suppliers should not appear three times
  3. Load into Bitrix. Via CIBlockElement::Add() for new elements, CIBlockElement::Update() for existing. Images: download, resize via CFile::ResizeImageGet(), convert to WebP. Properties via CIBlockElement::SetPropertyValuesEx(). SEO meta via \Bitrix\Iblock\InheritedProperty\ElementValues. SEF URLs generated from name transliteration.

  4. Update. Key point — not overwrite manual edits by content manager. Update only price, stock, activity. Description and photos manually edited are flagged with UF_MANUAL_EDIT property and skipped during import. Products missing from feed are deactivated, not deleted.

Why Is Competitor Price Monitoring Necessary?

A separate subsystem with its own specifics:

Parameter How It Works
Frequency From once a day to every 2 hours — depends on market volatility
Matching By SKU, EAN, fuzzy name comparison via Levenshtein distance
Storage Separate vendor_price_monitor table with history, not information blocks
Alerts Telegram/email when competitor price deviation exceeds X%
Auto-rules "Keep price 3% below competitor minimum, but not below cost + 15%"

Result — dashboard: your product vs competitors, price history, trends. The manager sees where to raise price without losing position, and where to react.

CSV/XML Import Module: Customization for Your Format

For supplier files — custom module with admin panel:

  • Configurable mapping: "column B in file → BRAND property of information block"
  • Auto-detect encoding (CP1251, UTF-8, UTF-16) via mb_detect_encoding() with validation
  • Download images from URL with queue — to avoid channel saturation
  • Incremental update by row hash: row changed — update, no — skip
  • Cron schedule, report: created 145, updated 892, errors 3 (with details)

Large files: CSV processed in batches of 1000 rows via fgetcsv() (10 times faster than row-by-row), XML streamed via XMLReader, background execution via Bitrix agent queue — no PHP timeouts.

Legal Aspects to Consider

  • robots.txt — respect it. Crawl-delay — comply.
  • Request frequency — 1–2 per second, no more. Don't DDoS someone else's site.
  • Manufacturer content — use it. Unique author texts — don't copy.
  • Personal data — don't collect.

What Is Included in a Turnkey Parser Development?

Component Description
Prototype Parser for 1–2 sources in 2–3 days to assess data quality
Main parser Full data collection from one source (static/dynamic)
Bitrix import module Normalization, loading, update, mapping admin panel
Price monitoring If needed — collection and alert system (up to 10 competitors)
Documentation Architecture description, selector update instructions
Support 3-month guarantee for uninterrupted operation, fix for donor layout changes

How We Work and Deadlines

  1. Prototype — parser for 1–2 sources in 2–3 days. Assess data quality, pitfalls (Cloudflare protection, captcha, dynamic loading).
  2. Development — full pipeline: parser → normalization → import into Bitrix → admin panel for management.
  3. Testing — run on full catalog volume, check edge cases (empty fields, malformed HTML, broken images).
  4. Launch — configure cron, error monitoring via Telegram bot.
  5. Support — competitor changed layout? Update CSS selectors in parser.
Task Deadlines
Single site parser (static HTML) 3–5 days
SPA site parser (Puppeteer/Playwright, bypass protection) 1–2 weeks
CSV/XML import module for Bitrix 1–2 weeks
Price monitoring system (5–10 competitors) 2–4 weeks
Comprehensive auto-population system 4–8 weeks
Parser support and adaptation by subscription

Get in touch for a free consultation — we will analyze your data sources and propose the optimal parser architecture. Request a project assessment today and get a fixed deadline. We guarantee stable parser operation and full support throughout the usage period.