Optimizing the import of large volumes of 1C-Bitrix data

Optimization of large data import 1C-Bitrix

Standard 1C exchange via CommerceML processes 50,000 items in 4 hours. For a catalog of 200,000 items with 30 properties and 5 price types — exchange doesn't complete: hits max_execution_time, exhausts memory, or locks tables for hours. Import optimization — work at three levels: PHP and DB tuning, exchange procedure enhancement, and transition to incremental import.

Why standard exchange is slow

CommerceML exchange works this way: 1C exports full XML file (import.xml + offers.xml), Bitrix parses it and for each element calls CIBlockElement::Add() or CIBlockElement::Update(). Each call:

1. Checks access rights
2. Calls OnBefore* event handlers
3. Writes element to b_iblock_element
4. Writes properties to b_iblock_element_property (separate INSERT/UPDATE for each property)
5. Writes prices to b_catalog_price
6. Updates search index
7. Invalidates cache
8. Calls OnAfter* event handlers

Per element — 10–20 SQL queries. For 200,000 elements — 2–4 million queries. Plus facet index recreation on each change.

PHP tuning

These parameters set globally in php.ini or for specific exchange script via .htaccess or virtual host. For PHP-FPM — separate pool with increased limits for /bitrix/admin/1c_exchange.php.

MySQL tuning

During import:

• Disable sync_binlog (if not using replication) — each INSERT won't be synced to disk: SET GLOBAL sync_binlog = 0;
• Increase innodb_log_file_size to 1–2 GB — reduces checkpoint frequency and speeds writes
• innodb_flush_log_at_trx_commit = 2 — transaction log written once per second instead of per transaction
• innodb_autoinc_lock_mode = 2 — speeds bulk INSERT (interleaved lock)

After import revert settings to production values. For automation — wrapper script changing parameters before import and reverting after.

Disabling handlers and indexes

Main speedup — disable unnecessary work during import:

• Event handlers. If OnAfterIBlockElementUpdate sends notifications or updates external system — on 200,000 items this is 200,000 HTTP requests. Disable via flag: check defined('CATALOG_IMPORT_RUNNING') in handler.
• Search index. Disable search module during import: \Bitrix\Main\ModuleManager::deActivate('search') — radical but effective. Alternative — disable catalog infoblock indexing in search module settings.
• Faceted index. Recreate on each item update — useless on mass import. Disable auto-update, after import recreate once.
• URL transliteration and slug generation. If CODE generated on element save — extra computations. Pass CODE from 1C or generate batch after import.

Batch import via D7 API

Instead of element-by-element CIBlockElement::Update() — direct table work via D7 ORM:

// Batch price update
$connection = Application::getConnection();
$connection->queryExecute("
    INSERT INTO b_catalog_price (PRODUCT_ID, CATALOG_GROUP_ID, PRICE, CURRENCY)
    VALUES (1, 1, 100.00, 'RUB'), (2, 1, 200.00, 'RUB'), ...
    ON DUPLICATE KEY UPDATE PRICE = VALUES(PRICE), CURRENCY = VALUES(CURRENCY)
");

INSERT ... ON DUPLICATE KEY UPDATE batches 1000 rows in one query instead of 1000 separate UPDATEs. Speedup — 10–50x.

Limitation: direct table writes don't trigger event handlers, don't invalidate cache, don't update facet index. All must be done manually after import.

Incremental import

Full import of 200,000 items nightly — wasteful if only 500 changed. Incremental approach:

1. On 1C side — export only changed elements (by modification date)
2. On Bitrix side — accept delta via REST API or custom endpoint
3. Hash comparison — store MD5 of key fields for each item. On import compare hash — if unchanged, skip

For exchanges more than once per hour incremental import is the only viable option.

Monitoring and diagnostics

• Import log — Bitrix writes to /bitrix/catalog_export/. Analyze timing of each step.
• Slow query log — enable on MySQL during import (long_query_time = 1). Shows problematic queries.
• Profiling — Bitrix\Main\Diag\SqlTracker counts SQL query qty and time. Enable on test run, find bottlenecks.

Optimization timeline

Key principle: standard CommerceML exchange is development convenience at cost of performance. For catalogs over 50,000 items need to consciously decide where standard mechanism is acceptable and where — direct SQL and incremental logic.