Accurate A/B Testing for Bitrix24 Email Campaigns

Imagine: you launch an A/B test for a mailing in Bitrix24, pick option A as the winner, but a week later sales haven't grown. Sound familiar? We see such cases every week: the test is on autopilot, the sample is too small, the wait window is too short — the result is statistically unreliable. Regardless of your marketer's experience, Bitrix24's built-in tools do not guarantee correct statistics. We set up A/B testing for mailings so that each test provides objective data for decision-making. Our methodology includes hypothesis testing, p-value calculation, and integration with Google Analytics to track conversions. Save up to 30% of your mailing budget by choosing effective variants.

A/B testing in the sender module allows you to compare subject lines, sender names, or templates. When creating a mailing, activate the "A/B test" mode and set audience percentages: 15% get variant A, 15% get variant B, and the remaining 70% wait for the winner. The winner is determined by Open Rate or Click Rate. Data is collected via tracking pixels and redirect links — the pixel is embedded automatically, clicks are tracked through redirects with UTM.

How to Prepare the Base for an A/B Test?

Audience segmentation is key. The base must be cleaned of duplicates and inactive addresses. Use behavior-based segments: opens in the last 30 days, clicks, subscription recency. When exporting data via the Bitrix24 REST API, choose only representative segments. The test group size must be at least 100 contacts per variant. If your base is small, increase the test group share to 30%.

Statistical Significance: What Is Often Ignored

The main mistake is too small a test sample or short wait window. If the base is 500 people and the test group is 10%, each variant has 25 recipients. A single open gives a 4% difference in Open Rate — statistically meaningless. Calculating the sample size for the target effect size is mandatory. Use an online calculator or export data from the b_sender_mailing_chain_table and compute a z-test manually.

How Many Recipients Are Needed for a Reliable A/B Test?

For minimal statistical significance, you need about 100 recipients per variant. For results with p<0.05, aim for 300+. The built-in tools do not show p-value, so we export data and calculate the z-test manually or via a third-party tool. This is the only way to filter out random fluctuations.

How to Interpret A/B Test Results in Bitrix24?

A/B testing in Bitrix24 increases decision accuracy by 3X compared to manual post-send comparison. But interpretation requires context: Open Rate is good for subject lines, Click Rate for content. We use UTM tags like utm_source=email&utm_medium=newsletter&utm_campaign=promo_march&utm_content=variant_a for variant A and utm_content=variant_b for B. This allows us to compare not only opens and clicks within Bitrix24 but also behavior on the site in Google Analytics and Yandex.Metrica — conversions, session depth, goal completion.

Compared to intuitive choice, A/B testing with our setup improves forecast accuracy by 2-3 times.

What Is Included in A/B Test Setup

We do the work end-to-end, from base analysis to test report. The scope includes:

1. Checking subscriber base for validity, duplicates, and segmentation
2. Defining the test hypothesis and calculating sample size
3. Preparing variants A and B with custom UTM tags
4. Configuring the campaign in the sender module (group percentages, wait time, winning criterion)
5. Verifying tracking: test send with pixel and redirect verification
6. Monitoring and analyzing results with p-value calculation
7. Documentation of setup and optimization recommendations

Why Trust Certified Specialists?

We have been working with Bitrix24 for over 5 years and have completed more than 100 projects configuring marketing tools. Our engineers are certified Bitrix24 specialists. We guarantee your test results will be statistically sound, not random.

Get a consultation on A/B test setup — we will evaluate your base and help choose the right scenario. Contact us for an audit of your current mailings and receive a personalized recommendation. Order setup, and let your mailings work with mathematical precision.

Duplicate Products on Page 3: A Bug That Goes to Production

A real case: an online store with pagination through bitrix:catalog.section duplicates products on page three after every second visit. Cache clearing helps for a day, then the duplicates return. Root cause: a custom sort handler collides with PAGEN_1, and under a specific filter combination CIBlockElement::GetList returns identical IDs. Code review missed it; only testing caught it. We build QA for 1C-Bitrix projects that catches such bugs before they hit production: manual functional, automated E2E, load, and acceptance testing. With over a decade of Bitrix experience, we have a library of typical pitfalls and test scenarios that prevent these issues from the start.

How Does a Standard Bitrix Project Break Without Dedicated Testing?

1C-Bitrix is not a landing page. Behind the frontend lie dozens of modules, external integrations, and non‑obvious dependencies. A discount change in sale.discount breaks a promo code in sale.basket.discount — the discount module is one of the most fragile in the platform. Interchange with 1C via catalog.import.1c or REST fails when property mapping is off, resulting in products without price or stock. Core updates — bitrix:main updated, a custom component uses the deprecated CModule::IncludeModule. Without regression testing, every deployment is Russian roulette. Cross‑browser: sale.order.ajax renders differently in Safari and Chrome; the “Place Order” button can move off‑screen on an iPhone. These are not edge cases — they are daily realities for Bitrix teams.

What Does Functional Testing Cover?

We check every business scenario — not just “works or doesn’t work”, but all boundary cases.

Catalog (catalog.section, catalog.element)

• Smart filter catalog.smart.filter: all property combinations, reset, result counting. Filters by SKUs break most often.
• Sorting + pagination — the duplicate bug described above.
• Comparison via catalog.compare.list — add, remove, display differences.
• Quick view — modal window, cart from modal.

Cart and Order (sale.basket.basket, sale.order.ajax)

• Adding from catalog, product page, quick order.
• Discounts: by quantity, by amount, by coupon, cumulative. Discount intersection — at least eight test combinations.
• Delivery calculation: handlers sale.delivery.services, cost, time, pickup points on map.
• Payment: sale.paysystem — processing, handling declines, refunds.
• Order placement: email via main.mail.event, CRM recording, transmission to 1C via sale.export.1c.

Personal Account (sale.personal.section)

• Registration, authorization, password recovery — including Cyrillic email edge cases.
• Order history, repeat order.
• Subscriptions, bonus program.

Forms and Search

• form.result.new / iblock.element.add.form — submission, validation, file fields.
• search.page — relevance, morphology, typo handling via search.title.

Why Is Regression Testing Critical for Bitrix?

After every deployment we verify that nothing previously working is broken.

• Smoke tests — main page loads, catalog shows products, order completes. 5 minutes, run after every deploy. If smoke fails — roll back immediately.
• Regression suite — 40–80 test cases covering main scenarios before every release.
• Visual testing — screenshot comparison (Percy or Playwright). A button shifted 20px, font changed after update — test shows the diff.
• Module checklists — structured lists for sale, catalog, iblock, search. Each module has its own checklist.

What Happens During Load Testing?

The question is not “will the site handle it” but at how many concurrent users catalog.section starts returning 500 errors.

Tools:

• k6 — JavaScript scripting.
• Apache JMeter — classic, for complex scenarios with cookie authorization.
• Yandex.Tank — real‑time visualization, integration with Overload.

Output: peak RPS, response times by percentiles p50/p95/p99, bottlenecks (CPU, RAM, MySQL slow queries on b_iblock_element, file cache). Recommendations: which index to add, which query to rewrite with D7 ORM, where to enable composite cache.

What Deliverables Do You Receive After Testing?

• Test plan with scope, priorities, and quality criteria.
• Test case suite — functional, regression, load.
• Defect report in a tracker (Jira/YouTrack) with severity classification.
• Auto tests (Playwright/Cypress) — basic smoke suite for CI/CD.
• Load testing protocol with graphs and recommendations.
• Acceptance certificate after UAT — confirming readiness for launch.

After delivery, we provide free consultation for a month — answering questions on test improvements and process adaptation. Contact us to receive a full package of documents and auto tests.

Cross‑Browser Testing

We test where buyers actually are. Statistics from your Metrica are more important than general market data.

Minimum set:

• Chrome (last 2 versions) — main traffic.
• Safari on iOS — critical for mobile checkout, sale.order.ajax often behaves unpredictably.
• Yandex.Browser — significant share in Russia, Chromium‑based but with extension quirks.
• Samsung Internet — mobile Android, often forgotten.

Devices:

• Desktop: 1920×1080, 1366×768.
• iPhone: 375×812, 390×844 — checkout must be verified.
• Android: 360×800, 412×915.

Tools: BrowserStack for real devices, Playwright for automation on Chromium/Firefox/WebKit.

Automation

Playwright — primary choice for E2E on Bitrix:

• Cross‑browser: Chromium, Firefox, WebKit.
• Parallel execution, automatic waits.
• Works well with dynamic forms sale.order.ajax.
• Supports mobile viewports and geolocation.

Cypress:

• Runs in browser — more stable for SPA‑like interfaces.
• Excellent visual runner for debugging.
• Limitation: only Chromium‑based browsers.

PHPUnit for custom code:

• Unit tests for custom Bitrix components and modules.
• Tests business logic without frontend dependency.
• Integration with CI/CD — GitLab CI, GitHub Actions.

UAT – Acceptance Testing

Final check with the client on a staging environment with real data:

• Jointly compile a list of critical scenarios — 15–20 key customer paths, not 200 test cases.
• Staging with a copy of the production database (anonymized personal data).
• Quick bug tracking — Jira/YouTrack, prioritization by severity.
• Acceptance protocol — document with results, signatures, and launch readiness.

Order UAT support and we guarantee a release without surprises.

QA Process – Integrated, Not Tacked On

1. Requirements analysis — QA participates in task discussions, catches ambiguities. “Does the discount apply to the product or the order?” — such a question upfront saves two days of debugging.
2. Test cases before development — scenarios ready before the first line of code.
3. Code review — checks for typical Bitrix mistakes: uncleared component cache, direct SQL queries instead of ORM, missing $USER‑>IsAuthorized() check.
4. Functional → regression → deploy.
5. Post‑release monitoring — errors in bitrix/error.log, metrics in Metrica, alerts for 500 errors.

We have been working with Bitrix for over 10 years and have tested more than 300 projects of various scales — from small online stores to corporate portals with 1C and Bitrix24 integration.

Timelines

Testing cost is calculated individually for your project. Get a free consultation — we will assess the scope within one business day and provide a preliminary estimate and test plan. Contact us to discuss your project and schedule a call.