The product catalog fills automatically via a parser, but one morning you discover prices haven't updated for two days. The parser failed in the first hour, and notifications didn't arrive — the standard Bitrix mail queue is clogged, alerts not configured. Revenue loss over two days — hundreds of thousands of rubles. Sound familiar? We've solved this on 30+ projects.
Experts agree: "A parser without alerts is a ticking time bomb." Our clients typically achieve savings of 50,000 to 100,000 RUB per month after implementation.
Without an alert system, any parser is a ticking time bomb. We've encountered this dozens of times: the client loses revenue because the parser silently crashes, and notifications don't come. The solution is to configure alerts so you learn of a problem the minute it fails, not after the fact.
Why Standard Bitrix Notifications Don't Solve the Problem
Standard Bitrix mail events (CEvent::Send) are often unsuitable for parsers: an email can take several minutes due to the b_event queue, and with mass errors the mailbox fills to capacity. There is no deduplication, no routing by urgency. A custom wrapper is needed that considers error type, source, and frequency. Idempotent notification handling ensures no duplicate alerts, and exponential backoff for network retries prevents notification floods.
Types of Errors to Catch
Before configuring notifications, classify errors:
-
Network — connection timeout, HTTP 403/429/503, connection reset. Source is unavailable or blocking. In 30% of cases the problem is temporary, but 5+ in a row is systemic.
-
Parsing DOM — source markup changed: CSS selectors or XPath return empty. Most common type (60% of incidents).
- Data validation — price = 0, name empty, SKU not in format. Parser received garbage.
- Catalog import —
CIBlockElement::Add() error, memory limit, missing properties. Apply circuit breaker pattern to temporarily disable notifications for non-critical errors.
Each type requires its own urgency. Network — 5-minute delay and retry; selector breakage — developer intervention.
Notification Architecture
Create a mail event PARSER_ERROR_NOTIFY with macros #ERROR_TYPE#, #SOURCE_URL#, #ERROR_MESSAGE#, #TIMESTAMP#. Register the template in admin section. Send from parser code:
CEvent::SendImmediate('PARSER_ERROR_NOTIFY', SITE_ID, [
'ERROR_TYPE' => 'DOM_CHANGED',
'SOURCE_URL' => $url,
'ERROR_MESSAGE' => 'Selector .price-block returned empty result',
'TIMESTAMP' => date('Y-m-d H:i:s'),
]);
SendImmediate sends mail immediately, bypassing the queue — correct for critical errors. Add deduplication: before sending, check in Redis (or b_option) if the same notification was sent in the last N minutes. For network errors — 30-60 minute suppression, for validation — no suppression. Our Bitrix parser monitoring ensures real-time alerts, and for 1C exchange error handling we prioritize timely notifications.
How to Deduplicate Notifications?
Deduplication is key to avoid spamming channels. We use Redis on each node: key — hash of ERROR_TYPE + SOURCE_URL, value — timestamp. If such hash exists and hasn't expired, notification is not sent. For network errors TTL 30 minutes; for DOM parsing errors — no suppression (each failure unique). Alternative: b_option table with serialized array, but Redis is faster and doesn't load the database.
Channels Besides Email
Email is slow (delivery 2-5 minutes). For critical errors we integrate Telegram Bot API or Bitrix24 webhooks. Telegram sends in 1-2 seconds, Bitrix24 chat — instantly to mobile app. Configure routing table by urgency:
| Error Type |
Email |
Telegram |
Bitrix24 Chat |
| Single network error |
— |
— |
— |
| Network (>5 in a row) |
+ |
+ |
— |
| DOM change |
+ |
+ |
+ |
| Invalid data (>10%) |
+ |
+ |
— |
| Import error |
+ |
+ |
+ |
Telegram is 10x faster than email for delivery — comparison not in favor of standard mail.
| Channel |
Typical Latency |
Drawbacks |
| Email (CEvent::Send) |
2-5 min |
Depends on queue |
| Telegram Bot API |
1-2 sec |
Requires bot |
| Bitrix24 webhook |
0.5-1 sec |
On-premise only |
Logging as Foundation for Notifications
Notifications are an overlay on logs. Write each error to b_event_log via CEventLog::Add() with AUDIT_TYPE_ID='PARSER_ERROR'. This provides history in the standard log, filtering, and rotation. An agent counts errors hourly and sends a digest if threshold exceeded. This parser control automation reduces manual oversight.
Example fault emulation for testing
To test alerts, we simulate a parsing error: send a POST request with deliberately wrong data to the parser. For example, specify a non-existent URL or feed HTML without required selectors. Catch the exception, and the system sends a notification. If it arrives within 5 seconds — everything works.
How to Set Up Alerts in 1 Day?
The process takes one working day:
- Create mail event PARSER_ERROR_NOTIFY and template with macros.
- Write
ParserNotifier::send() class with deduplication and channel selection.
- Integrate Telegram Bot API or Bitrix24 webhook.
- Configure error logging to
b_event_log.
- Emulate a fault and verify alert delivery.
Our error alert system is proven across 30+ projects, leading to significant parsing cost savings.
What's Included in the Setup
- Mail event type and template creation
- Notification wrapper with deduplication logic
- Telegram Bot API or Bitrix24 webhook integration
- Error logging to b_event_log with rotation
- Fault emulation test to verify delivery
- Operation manual for maintenance
After setup, you get a ready-made notification system with an operation manual. Order setup — we'll analyze your parser and configure alerts in 1 day. Get a consultation: our engineers will help determine optimal channels and thresholds.
Parser Development for 1C-Bitrix: Where to Start?
XMLReader, not SimpleXML — the choice of tool determines the project's fate. SimpleXML loads the entire XML into memory, and with an 800 MB supplier file, PHP will crash with a fatal error on a 512 MB limit. XMLReader processes streamingly, node by node, consuming 20–30 MB — 30 times more efficient. This detail starts any parser development for Bitrix. With over 10 years of Bitrix development and 50+ parser projects delivered, we know the pitfalls. Contact us to start your parser development today.
What Problems Does Parsing Solve?
- Primary catalog filling — 15,000 cards with descriptions, characteristics, photos. Manually, that's three months of content manager work; a parser takes a week with debugging.
- Competitor price monitoring — collecting data from Ozon, Wildberries, competitor sites. A competitor drops the price on a hot item — you find out in two hours, not two weeks.
- Supplier aggregation — five price lists in different formats (CSV with CP1251, XML in CommerceML, Excel with merged cells) become a single catalog with a unified property system.
- Card enrichment — pulling characteristics, instructions, 3D models from manufacturer sites. Without this, a product card is an SEO empty shell.
- Assortment update — products missing from the supplier feed are deactivated via
CIBlockElement::Update($ID, ['ACTIVE' => 'N']). New ones are created. The catalog stays synchronized.
What Tools Do We Use in Parser Development?
Static websites — PHP (Goutte, Symfony DomCrawler) or Python (Scrapy, lxml). Speed: 50–100 pages/sec. Sufficient for catalogs without JS rendering.
SPA and dynamic websites — Puppeteer or Playwright. Infinite scroll, AJAX filters, lazy-load images — headless browser handles it all. Speed drops to 1–10 pages/sec, but there is no alternative: data exists only after JavaScript execution.
Supplier files:
- Excel (XLS, XLSX) — PhpSpreadsheet. Beware of merged cells and formulas — they break automatic mapping.
- CSV —
fgetcsv() with correct encoding. Suppliers love CP1251, BOM in UTF-8, and semicolons instead of commas. All need detection and handling.
- XML/YML — XMLReader for large files, SimpleXML for feeds up to 50 MB.
- CommerceML — standard exchange format with 1C. We parse
import.xml and offers.xml, map to information block structure.
API — Supplier REST endpoints, marketplace APIs (Ozon Seller API, Wildberries API). We work within rate limits, handle pagination.
How Is the Auto-Population Pipeline Structured?
Four stages. Each can break in its own way.
-
Collection. Parser crawls sources via cron schedule. Raw data goes to an intermediate table — not directly into b_iblock_element. Log everything: pages visited, elements parsed, where we got 403 or timeout. Without logs, debugging a parser is like fortune-telling.
-
Normalization. Main work here:
- Clean HTML tags, extra spaces, Unicode garbage
- Units: "mm" → "mm", "millimeters" → "mm", "миллиметр" → "mm"
- Map supplier categories to Bitrix information block sections. One supplier has "Notebooks", another "Notebooks and tablets", third "Laptops" — all into one section
- Deduplication by SKU, EAN/GTIN. One product from three suppliers should not appear three times
-
Load into Bitrix. Via CIBlockElement::Add() for new elements, CIBlockElement::Update() for existing. Images: download, resize via CFile::ResizeImageGet(), convert to WebP. Properties via CIBlockElement::SetPropertyValuesEx(). SEO meta via \Bitrix\Iblock\InheritedProperty\ElementValues. SEF URLs generated from name transliteration.
-
Update. Key point — not overwrite manual edits by content manager. Update only price, stock, activity. Description and photos manually edited are flagged with UF_MANUAL_EDIT property and skipped during import. Products missing from feed are deactivated, not deleted.
Why Is Competitor Price Monitoring Necessary?
A separate subsystem with its own specifics:
| Parameter |
How It Works |
| Frequency |
From once a day to every 2 hours — depends on market volatility |
| Matching |
By SKU, EAN, fuzzy name comparison via Levenshtein distance |
| Storage |
Separate vendor_price_monitor table with history, not information blocks |
| Alerts |
Telegram/email when competitor price deviation exceeds X% |
| Auto-rules |
"Keep price 3% below competitor minimum, but not below cost + 15%" |
Result — dashboard: your product vs competitors, price history, trends. The manager sees where to raise price without losing position, and where to react.
CSV/XML Import Module: Customization for Your Format
For supplier files — custom module with admin panel:
- Configurable mapping: "column B in file → BRAND property of information block"
- Auto-detect encoding (CP1251, UTF-8, UTF-16) via
mb_detect_encoding() with validation
- Download images from URL with queue — to avoid channel saturation
- Incremental update by row hash: row changed — update, no — skip
- Cron schedule, report: created 145, updated 892, errors 3 (with details)
Large files: CSV processed in batches of 1000 rows via fgetcsv() (10 times faster than row-by-row), XML streamed via XMLReader, background execution via Bitrix agent queue — no PHP timeouts.
Legal Aspects to Consider
-
robots.txt — respect it. Crawl-delay — comply.
- Request frequency — 1–2 per second, no more. Don't DDoS someone else's site.
- Manufacturer content — use it. Unique author texts — don't copy.
- Personal data — don't collect.
What Is Included in a Turnkey Parser Development?
| Component |
Description |
| Prototype |
Parser for 1–2 sources in 2–3 days to assess data quality |
| Main parser |
Full data collection from one source (static/dynamic) |
| Bitrix import module |
Normalization, loading, update, mapping admin panel |
| Price monitoring |
If needed — collection and alert system (up to 10 competitors) |
| Documentation |
Architecture description, selector update instructions |
| Support |
3-month guarantee for uninterrupted operation, fix for donor layout changes |
How We Work and Deadlines
- Prototype — parser for 1–2 sources in 2–3 days. Assess data quality, pitfalls (Cloudflare protection, captcha, dynamic loading).
- Development — full pipeline: parser → normalization → import into Bitrix → admin panel for management.
- Testing — run on full catalog volume, check edge cases (empty fields, malformed HTML, broken images).
- Launch — configure cron, error monitoring via Telegram bot.
- Support — competitor changed layout? Update CSS selectors in parser.
| Task |
Deadlines |
| Single site parser (static HTML) |
3–5 days |
| SPA site parser (Puppeteer/Playwright, bypass protection) |
1–2 weeks |
| CSV/XML import module for Bitrix |
1–2 weeks |
| Price monitoring system (5–10 competitors) |
2–4 weeks |
| Comprehensive auto-population system |
4–8 weeks |
| Parser support and adaptation |
by subscription |
Get in touch for a free consultation — we will analyze your data sources and propose the optimal parser architecture. Request a project assessment today and get a fixed deadline. We guarantee stable parser operation and full support throughout the usage period.