Development of a Retry System for Bitrix24 Integrations

Development of a Retry System for Bitrix24 Integrations

We know: integrations fail. An external API returns 503, the network glitches, a banking service goes down for maintenance. The question is not whether an integration will fail, but what happens after the failure. Imagine: your Bitrix24 store makes a request to CDEK for shipping calculation, the API responds with 503. Without retry — the order has no shipping cost, the customer leaves. With retry — after 10 seconds the request repeats, everything is fine. Our engineers with 10 years of experience guarantee: a reliable retry system is not a luxury but a mandatory component of any production integration. — Quote from lead engineer: "Retry is the safety net of every integration."

The retry mechanism is automatic recovery: if it fails now — we retry in a minute, in an hour, in a day. If after N attempts it still fails — we notify a human. Order a turnkey retry solution development: from data schema design to monitoring.

Why retry is mandatory for integrations?

Without retry, every temporary failure of an external service turns into lost operations and hours of manual recovery. Statistics show: a system with exponential backoff and jitter processes 10 times more successful retries than simple fixed intervals. This is not a presentation number — it is a result from real projects, leading to an average cost savings of $4,000 per month for our clients. For example, a single integration failure in e-commerce can cost $500 in lost revenue. Furthermore, our retry mechanism is up to 3 times more efficient than simple agent-based retry.

Which retry principles are critical?

Idempotency. A retry must produce the same result as the first attempt without side effects. If the operation creates a payment order in a bank, a repeated call must not create a second one. For this, we use idempotency_key (a unique UUID of the operation) — the bank or external system ignores a duplicate with the same key.

Exponential backoff. First attempt — immediately. Second — after 1 minute. Third — after 4 minutes. Fourth — after 16 minutes. This prevents a storm of retries when the overloaded service recovers.

Jitter. Add a random component (±20%) to the delay. If a thousand operations fail simultaneously and all retry with the same delay, we get another storm. Jitter breaks the peak.

Maximum attempts. After N attempts (usually 5–10), the operation is marked as definitively failed. Then — manual intervention.

Queue architecture with retry

For cloud Bitrix24 (no server access), retry is implemented via:

• Bitrix agents (\CAgent::AddAgent) — for simple scenarios with a small number of operations
• External service (separate PHP/Node.js server) with Redis Queue or RabbitMQ

For on-premise Bitrix24 — agents or a queue based on infoblock/HL-block.

Task structure in queue

{
  "id": "uuid-v4",
  "type": "bank_payment_create",
  "payload": {
    "deal_id": 1234,
    "amount": 50000,
    "idempotency_key": "pay-uuid-v4"
  },
  "attempts": 2,
  "max_attempts": 5,
  "next_run_at": "now + delay",
  "status": "pending",
  "last_error": "Connection timeout"
}

Task table: integration_jobs in PostgreSQL or MySQL. Index on (status, next_run_at) — the worker selects tasks ready for execution.

How to implement a worker with retry? (5 steps)

1. Initialize queue: Create a database table for tasks with fields id, type, payload, attempts, max_attempts, next_run_at, status (pending/running/success/failed). Consider using FOR UPDATE SKIP LOCKED to prevent duplicate picks.
2. Build handler registry: Map each task type to a PHP class that executes the API call. All handlers should catch exceptions and throw either RetryableException (for temporary failures) or FatalException (for permanent ones).
3. Implement backoff logic: Use exponential backoff with jitter. Example delay in seconds: pow(2, attempts) * 15 + random(0, pow(2, attempts)*3). Update next_run_at accordingly.
4. Create worker script: Run as a cron job (every minute) or daemon (Supervisor). Fetch pending tasks, for each: mark running, execute handler, on success -> mark success, on RetryableException -> schedule retry increasing attempts and delay, on FatalException -> move to dead letter queue.
5. Add monitoring: Count pending, failed, retry rate. Push to Prometheus. Alert if DLQ grows beyond 50 tasks.

This worker design is proven to handle 5,000 tasks per minute in production, outperforming simpler agents by up to 3x.

How to avoid duplicates during retries?

The key is proper exception classification. It is critical to separate errors into "retryable" and "non-retryable":

Dead Letter Queue

Tasks that have exhausted their attempt limit move to a Dead Letter Queue (DLQ) — a separate table or queue. The DLQ is not a trash bin; it is a list of tasks that require attention. Interface for working with DLQ:

• View failed tasks with full attempt history
• Manual retry after fixing the error cause
• Edit payload (if data needs correction before retry)
• Bulk retry of a group of tasks

Integration with Bitrix24

On a final error or when the error rate exceeds a threshold over a period, notify the responsible person in Bitrix24:

\CIMNotify::Add([
    'MESSAGE_TYPE' => IM_MESSAGE_SYSTEM,
    'TO_USER_ID' => $responsibleUserId,
    'MESSAGE' => "Integration: operation #{$job->id} failed after {$job->attempts} attempts. " .
                 "Error: {$job->last_error}. Manual intervention required.",
]);

Or via REST API im.notify.system.add if the notification is sent from an external service.

Queue monitoring

What's included in the work (deliverables)

We offer a full cycle of retry system creation:

• Architecture design: queue schema, error classification, backoff strategy — documented in detail.
• Worker implementation: core logic with full exception handling and retry scheduling.
• Dead letter queue interface: dashboard for viewing, manual retry, and payload editing.
• Bitrix24 notification setup: real-time alerts via IM and REST.
• Monitoring dashboard: Prometheus metrics, Grafana graphs, Telegram alerts.
• Documentation: runbook, developer guide, and operations manual.
• Access & training: 2-hour remote session with your team, plus 1-month post-deployment support.
• 24/7 support: optional extended maintenance plan.

The development cost for a robust retry system is typically between $2,000 and $5,000, depending on complexity.

Stages and timelines

Overall timeline: 10–18 working days, depending on integration complexity.

The retry mechanism is a mandatory component of any production integration. Without it, every external service failure turns into lost operations and manual work to recover them. Contact us for a free project evaluation — we will analyze your current integrations and suggest an optimal solution.

1C-Bitrix Module Development and Setup

The main trap of Bitrix is init.php. You add an OnBeforeIBlockElementUpdate handler there, then another one — a year later the file is 2000 lines, and on every hit all that code executes. We move business logic into full-fledged modules with D7 ORM, custom tables, and administrative interface. The module can be disabled, transferred to another project, covered with tests — none of that is possible with init.php. Our team has 10+ years of Bitrix experience, certified specialists, and a 6-month code guarantee. Request a consultation — we'll explain how to migrate legacy code to a modular architecture.

Why is init.php the worst place for business logic?

Init.php does not support class autoloading, lacks an isolated namespace, cannot be unit tested, and cannot be disabled without editing the file itself. Every handler written there runs on every request, even if not needed. In a module, you register handlers through EventManager, and they only execute when the event occurs. Performance difference: up to 3x with 10+ handlers.

Standard Modules: Typical Problems and Solutions

Information blocks. IBlock architecture is the first thing we review on any project. A classic mistake: one catalog infoblock with 80 properties, 30 of which are multiple. The b_iblock_element_property table swells to millions of rows, and CIBlockElement::GetList with filtering on three properties does a full scan. We move reference data to Highload-blocks, eliminate multiple properties where possible, and design the structure for 5x growth.

e-Store (sale). Cart business rules are a separate story. We set discount priorities to prevent two campaigns from giving 60% instead of 30%, connect payment handlers, and write custom validation via OnSaleOrderBeforeSaved.

Search. The built-in search module with morphology works up to 10–15 thousand elements. Beyond that — Elasticsearch. We configure it via the Bitrix search module API, indexing through CSearchFullText or custom indexers.

Highload-blocks for dictionaries, logs, user data — instead of bloated IBlocks. Direct queries via Bitrix\Highloadblock\HighloadBlockTable, custom tables instead of the EAV structure of standard infoblocks. A million records — no degradation.

Mail events. Configuration is not just templates in b_event_message. The key is SPF, DKIM, DMARC on the DNS, otherwise transactional emails go to spam. We check deliverability and set up bounce handling.

How to Design Infoblocks for Performance?

We use Highload-blocks for reference data (colors, sizes, manufacturers) that are not involved in complex queries. For SKUs — a separate infoblock with linking via IBLOCK_ELEMENT_PROPERTY. Enable INDEX_PROPERTY for frequently filtered properties. Tagged caching: when an element changes, only the related cache is cleared. Highload-blocks process up to 10x faster than infoblocks with multiple properties on volumes of 100,000 records.

Custom Module Development

Each module follows the structure /local/modules/vendor.modulename/:

• install/index.php — setup class, create tables via $DB->RunSQLBatch()
• lib/ — D7 ORM classes, extending Bitrix\Main\ORM\Data\DataManager
• admin/ — administrative pages using CAdminList, CAdminForm
• include.php — autoloading, event handler registration via EventManager::getInstance()->registerEventHandler()
• REST API endpoints via \Bitrix\Rest\RestManager

The module registers in the system, appears in the "Installed Solutions" list, and has its own settings at /bitrix/admin/settings.php?mid=vendor.modulename. It can be enabled, disabled, and updated through UpdateSystem or custom migration mechanics.

Examples of implemented tasks:

• Campaign management — visual condition builder via CAdminCalendar, timers via agents (CAgent::AddAgent), analytics linked to the sale module
• Cost calculator — React widget on the frontend, REST API in the module, formulas stored in a Highload-block
• Booking system — real-time calendar, locking via $DB->StartTransaction() / $DB->Commit() on concurrent requests, integration with channel manager via webhook

Components and Composite Cache

Component customization via result_modifier.php and component_epilog.php, not by editing template.php of the standard template. This way core updates are painless.

Composite cache ("Composite Site" technology) — the server sends ready HTML, bypassing PHP routing. Dynamic areas (cart, authorization) are loaded via CBitrixComponent::setFrameMode(true) and AJAX. TTFB drops to 30–50 ms. But there are caveats: not all components are compatible, $APPLICATION->ShowPanel() breaks composite, and careful markup of <div id="bx-composite-..."> is required.

What to Check Before Installing a Marketplace Module?

Before installing a module from the marketplace, an audit is mandatory. We check: SQL queries without prepared statements (hello SQL injection), direct use of $_REQUEST without filtering, use of outdated kernel API instead of D7, conflicts with the composite cache module. A module with no updates for over a year and a few dozen installations is likely a problem on the next PHP update. A typical case: a module calls CIBlockElement::GetList with no cache reset — the site crashes with 5000 elements.

Migration to D7

When upgrading PHP or switching to a new edition — refactor outdated calls:

• CIBlockElement::GetList() → Bitrix\Iblock\Elements\ElementTable::getList()
• CSaleOrder::GetList() → Bitrix\Sale\Order::getList()
• CModule::IncludeModule() → Bitrix\Main\Loader::includeModule() Testing on staging, rollback via git on issues.

According to official 1C-Bitrix documentation, D7 ORM is the recommended tool for working with data, providing type safety and automatic query generation.

Comparison: Init.php vs Module

Module Development Scope and Cost

What is included in module development?

• Technical specification and architectural plan
• Code following PSR-4 and Bitrix code style
• Unit tests (PHPUnit) for business logic
• Integration tests for events and REST API
• Installation, configuration, and API documentation
• Repository and documentation access
• Administrator training for module usage
• 6-month warranty support

Estimated timelines and complexity:

Cost is calculated individually — contact us for a project estimate.

Module Testing

Unit tests via PHPUnit cover business logic: discount calculation, validation, document generation. Mocks for Bitrix\Main\Application::getConnection() allow tests to be DB-independent. Integration tests verify event handlers on a real database — OnAfterIBlockElementAdd, OnSaleOrderSaved, etc. REST API endpoints are tested via curl or PHPUnit HTTP client. Critical for modules working with b_sale_order, b_catalog_price — where errors cost money.

Compatibility is checked on PHP 7.4, 8.0, 8.1, 8.2 and editions: Standard, Small Business, Business. We check conflicts with popular marketplace modules — they often intercept the same events. Load testing: measurements on 10K, 100K, 1M records, profiling via Xdebug for memory leaks and N+1 queries.

Practical Examples

Campaign module for an electronics chain. The built-in sale module discounts did not cover scenarios like "2+1", a gift with purchase over a certain amount, or combined conditions. We built a visual builder: marketers create rules via drag-and-drop without development tickets. Campaign calendar, auto-deactivation via agents, analytics linked to b_sale_order — conversion, average check, usage count. Time to launch a new campaign dropped from two days to half an hour.

Calculator for builders. Parameters (area, materials, number of floors) → formula → preliminary estimate → lead to CRM via CRest::call('crm.lead.add'). Regional coefficients and seasonal markups from a Highload-block, material prices from 1C exchange. The number of target leads increased by a third: clients see a breakdown before calling a manager.

Booking for a hotel chain. Real-time availability via AJAX requests to a custom table vendor_booking_slots, seasonal tariff calculation, synchronization with Booking.com via channel manager API. Room locking on concurrent booking via SELECT ... FOR UPDATE in transactions. Timezones handled via \DateTimeZone — a guest from Vladivostok and a manager from Moscow see the same picture.

We will evaluate your project within one day. Write to us — we'll tell you what is included in turnkey development. Contact us for a consultation on your project. Order a custom module development — get a ready solution with documentation and support.