RSS Auto-Posting for News Sections in 1C-Bitrix

Regularly updating the news section is critical for SEO and audience retention. Manually collecting and publishing news from industry sources is inefficient: site owners spend up to 10 hours per week on this routine, diverting resources from business development. RSS aggregation solves the problem: the system automatically fetches fresh entries, removes duplicates, and publishes them with proper attribution. We are a team of engineers with many years of experience in 1C-Bitrix, having completed over 50 content automation projects. We implement such a solution turnkey in 3–5 days, ensuring content uniqueness and copyright compliance. Get a consultation for your project — just contact us.

Why Automate News Filling?

Manually copying news from feeds leads to content duplication, which is penalized by search engines and reduces audience trust. Automation not only saves time but also guarantees that each news item is unique and properly attributed. With an RSS aggregator, you get fresh content daily without editor involvement.

How to Fetch and Parse RSS?

RSS is an XML format with a standard structure. Each entry (<item>) contains title, link, description, pubDate, author. Atom entries (<entry>) use different tags but the logic is similar.

Parsing with SimpleXML:

$rss = simplexml_load_file($feedUrl);
foreach ($rss->channel->item as $item) {
    $this->processItem([
        'title'   => (string)$item->title,
        'link'    => (string)$item->link,
        'content' => (string)$item->children('content', true)->encoded ?: (string)$item->description,
        'pubDate' => strtotime((string)$item->pubDate),
        'guid'    => (string)$item->guid,
    ]);
}

<content:encoded> contains the full article text (if the source provides it), <description> is usually an excerpt. SimpleXML is 2 times faster than ready-made DOM parsers. The RSS specification defines feed formats.

How to Avoid Duplicate Entries?

The same material may appear in multiple feeds or be republished. The primary method is deduplication by guid (unique identifier in RSS):

$existing = CIBlockElement::GetList([], [
    'IBLOCK_ID' => NEWS_IBLOCK_ID,
    '=PROPERTY_RSS_GUID' => $item['guid']
])->Fetch();
if ($existing) continue; // already imported

The RSS_GUID property is of type S with IS_REQUIRED = N. An alternative for speed is to store processed GUIDs in a separate table or Redis Set. We tested both approaches: Redis yields a 30% performance gain for streams over 10,000 entries.

Storage in the News Infoblock

Standard news infoblock with additional properties for RSS aggregation:

• RSS_GUID — entry GUID for deduplication
• RSS_SOURCE — source ID or name (for attribution)
• ORIGINAL_URL — link to original (for canonical and source link)
• AUTO_IMPORTED — auto-import flag (Y/N) to distinguish from manual posts

Publication date from RSS → ACTIVE_FROM of the element. This is important for correct news sorting.

Why Content Processing Matters?

Direct publication of RSS content without processing is duplication that leads to search engine penalties. We offer three processing levels:

Minimum: legitimate aggregation — not duplication, as only part of the text is published. Medium level: we clean HTML (HTMLPurifier), remove internal source links, rephrase the introduction and title. Full rewrite via AI: send content:encoded to GPT with instructions to rewrite in a different style. This is expensive for high-frequency feeds but justified for key materials. The costs of manual publication can be significant, and our solution pays off in 2–4 months.

How to Configure Multiple Feeds?

Source configuration in a Highload-block RssSources:

• UF_URL — feed URL
• UF_NAME — source name
• UF_IBLOCK_ID — target infoblock
• UF_SECTION_ID — section for imported materials
• UF_ACTIVE — enabled/disabled
• UF_INTERVAL — check interval in minutes
• UF_LAST_CHECK — last check time
• UF_PROCESSING — processing type (excerpt / full / ai_rewrite)

How We Set Up an RSS Aggregator?

1. Technical specification and solution architecture.
2. Develop RSS reader supporting RSS 2.0 and Atom.
3. Deduplication, storage in infoblock, attribution flags.
4. Administrative interface for source management.
5. Content processing according to selected scheme.
6. Agent scheduling and error monitoring.
7. Documentation and access transfer.

What Is Included in the Work?

• Documentation: data schema, agent settings, admin manual.
• Access: login credentials, SSH keys, API credentials (if required).
• Training: walkthrough of feed management interface, answering questions.
• Support: warranty maintenance for 30 days after delivery.

Timeline

Total: 3–5 working days. Adding AI-rewrite — plus 1–2 days.

We'll evaluate your project in 1 day — just contact us. Order turnkey development with quality guarantee and post-implementation support. Get in touch for your project estimate.

Parser Development for 1C-Bitrix: Where to Start?

XMLReader, not SimpleXML — the choice of tool determines the project's fate. SimpleXML loads the entire XML into memory, and with an 800 MB supplier file, PHP will crash with a fatal error on a 512 MB limit. XMLReader processes streamingly, node by node, consuming 20–30 MB — 30 times more efficient. This detail starts any parser development for Bitrix. With over 10 years of Bitrix development and 50+ parser projects delivered, we know the pitfalls. Contact us to start your parser development today.

What Problems Does Parsing Solve?

• Primary catalog filling — 15,000 cards with descriptions, characteristics, photos. Manually, that's three months of content manager work; a parser takes a week with debugging.
• Competitor price monitoring — collecting data from Ozon, Wildberries, competitor sites. A competitor drops the price on a hot item — you find out in two hours, not two weeks.
• Supplier aggregation — five price lists in different formats (CSV with CP1251, XML in CommerceML, Excel with merged cells) become a single catalog with a unified property system.
• Card enrichment — pulling characteristics, instructions, 3D models from manufacturer sites. Without this, a product card is an SEO empty shell.
• Assortment update — products missing from the supplier feed are deactivated via CIBlockElement::Update($ID, ['ACTIVE' => 'N']). New ones are created. The catalog stays synchronized.

What Tools Do We Use in Parser Development?

Static websites — PHP (Goutte, Symfony DomCrawler) or Python (Scrapy, lxml). Speed: 50–100 pages/sec. Sufficient for catalogs without JS rendering.

SPA and dynamic websites — Puppeteer or Playwright. Infinite scroll, AJAX filters, lazy-load images — headless browser handles it all. Speed drops to 1–10 pages/sec, but there is no alternative: data exists only after JavaScript execution.

Supplier files:

• Excel (XLS, XLSX) — PhpSpreadsheet. Beware of merged cells and formulas — they break automatic mapping.
• CSV — fgetcsv() with correct encoding. Suppliers love CP1251, BOM in UTF-8, and semicolons instead of commas. All need detection and handling.
• XML/YML — XMLReader for large files, SimpleXML for feeds up to 50 MB.
• CommerceML — standard exchange format with 1C. We parse import.xml and offers.xml, map to information block structure.

API — Supplier REST endpoints, marketplace APIs (Ozon Seller API, Wildberries API). We work within rate limits, handle pagination.

How Is the Auto-Population Pipeline Structured?

Four stages. Each can break in its own way.

1. Collection. Parser crawls sources via cron schedule. Raw data goes to an intermediate table — not directly into b_iblock_element. Log everything: pages visited, elements parsed, where we got 403 or timeout. Without logs, debugging a parser is like fortune-telling.

2. Normalization. Main work here:

   • Clean HTML tags, extra spaces, Unicode garbage
   • Units: "mm" → "mm", "millimeters" → "mm", "миллиметр" → "mm"
   • Map supplier categories to Bitrix information block sections. One supplier has "Notebooks", another "Notebooks and tablets", third "Laptops" — all into one section
   • Deduplication by SKU, EAN/GTIN. One product from three suppliers should not appear three times

3. Load into Bitrix. Via CIBlockElement::Add() for new elements, CIBlockElement::Update() for existing. Images: download, resize via CFile::ResizeImageGet(), convert to WebP. Properties via CIBlockElement::SetPropertyValuesEx(). SEO meta via \Bitrix\Iblock\InheritedProperty\ElementValues. SEF URLs generated from name transliteration.

4. Update. Key point — not overwrite manual edits by content manager. Update only price, stock, activity. Description and photos manually edited are flagged with UF_MANUAL_EDIT property and skipped during import. Products missing from feed are deactivated, not deleted.

Why Is Competitor Price Monitoring Necessary?

A separate subsystem with its own specifics:

Result — dashboard: your product vs competitors, price history, trends. The manager sees where to raise price without losing position, and where to react.

CSV/XML Import Module: Customization for Your Format

For supplier files — custom module with admin panel:

• Configurable mapping: "column B in file → BRAND property of information block"
• Auto-detect encoding (CP1251, UTF-8, UTF-16) via mb_detect_encoding() with validation
• Download images from URL with queue — to avoid channel saturation
• Incremental update by row hash: row changed — update, no — skip
• Cron schedule, report: created 145, updated 892, errors 3 (with details)

Large files: CSV processed in batches of 1000 rows via fgetcsv() (10 times faster than row-by-row), XML streamed via XMLReader, background execution via Bitrix agent queue — no PHP timeouts.

Legal Aspects to Consider

• robots.txt — respect it. Crawl-delay — comply.
• Request frequency — 1–2 per second, no more. Don't DDoS someone else's site.
• Manufacturer content — use it. Unique author texts — don't copy.
• Personal data — don't collect.

What Is Included in a Turnkey Parser Development?

How We Work and Deadlines

1. Prototype — parser for 1–2 sources in 2–3 days. Assess data quality, pitfalls (Cloudflare protection, captcha, dynamic loading).
2. Development — full pipeline: parser → normalization → import into Bitrix → admin panel for management.
3. Testing — run on full catalog volume, check edge cases (empty fields, malformed HTML, broken images).
4. Launch — configure cron, error monitoring via Telegram bot.
5. Support — competitor changed layout? Update CSS selectors in parser.

Get in touch for a free consultation — we will analyze your data sources and propose the optimal parser architecture. Request a project assessment today and get a fixed deadline. We guarantee stable parser operation and full support throughout the usage period.