Setting up A/B testing on 1C-Bitrix

A/B Testing Setup for 1C-Bitrix

Redesigned product card, new "Buy" button, changed block order on landing — without A/B test, this is guessing. Intuition misleads: the variant the "director likes" often loses on conversion. 1C-Bitrix has built-in A/B testing in module abtest, but few use it — documentation is sparse and limitations unclear. Let's examine native mechanism and alternatives.

Native abtest Module

Module abtest available in "Business" and "Enterprise" editions. Located at Marketing → A/B Testing. Mechanics:

1. Create test with name and description.
2. Specify traffic percentage for variant B (usually 50%).
3. Select test type: site template, component template, included area, or arbitrary PHP code.
4. Bitrix assigns user to group (A or B) on first visit via cookie BITRIX_SM_ABTEST_{ID}. Cookie lives until test end.
5. Depending on group, corresponding variant substitutes.

Creating test via API:

\Bitrix\ABTest\ABTestManager::addTest([
    'NAME' => 'Buy button: red vs green',
    'SITE_ID' => 's1',
    'DURATION' => 14,  // days
    'PORTION' => 50,   // traffic percentage for variant B
    'TEST_DATA' => [
        'type' => 'template',
        'original' => '/local/templates/main/',
        'modified' => '/local/templates/main_test/',
    ],
]);

For component template test: specify original and test template, Bitrix substitutes TEMPLATE_NAME parameter on $APPLICATION->IncludeComponent() call.

Native Module Limitations

• No analytics integration. Module counts conversion by Yandex.Metrica or Google Analytics goals, but link is unreliable — data diverges, attribution imprecise.
• No segmentation. Can't run test only for mobile, only Moscow, only new users.
• No multivariate tests. Only A vs B, no C, D variants.
• Caching. If component caches result — both variants get same HTML. Must disable cache for tested component or cache separately by test group.

Caching solved by adding cookie BITRIX_SM_ABTEST_{ID} to component cache key:

$this->setResultCacheKeys(['AB_GROUP']);
$cacheDir = '/catalog/element/' . $arParams['ELEMENT_ID'] . '/' . $_COOKIE['BITRIX_SM_ABTEST_1'];

Google Optimize as Alternative (Client-Side)

Google Optimize closed in 2023, but client-side A/B testing approach lives in other tools: VWO, Optimizely, ABTasty. Principle: JavaScript snippet replaces DOM on client. Bitrix serves identical HTML, script modifies elements.

Pros: no need to touch server code. Cons: flickering (FOUC — original appears briefly before substitution), doesn't work for server logic (different prices, sort algorithms).

Server-Side A/B via Custom Code

For serious tests (different discount calculation logic, different recommendation algorithm), need server approach:

function getABGroup(string $testName, int $percentB = 50): string
{
    $cookieName = 'ab_' . md5($testName);
    if (isset($_COOKIE[$cookieName])) {
        return $_COOKIE[$cookieName];
    }
    $group = (mt_rand(1, 100) <= $percentB) ? 'B' : 'A';
    setcookie($cookieName, $group, time() + 86400 * 30, '/');
    return $group;
}

// Usage
if (getABGroup('discount_algorithm') === 'B') {
    // New discount algorithm
} else {
    // Current algorithm
}

For tracking results — pass group to dataLayer:

dataLayer.push({
    'ab_test': 'discount_algorithm',
    'ab_group': '<?= getABGroup("discount_algorithm") ?>'
});

In Yandex.Metrica or GA4, create custom parameter, build report by conversion with split by groups.

Statistical Significance

Typical error — stop test after 2 days seeing 0.5% difference. For reliable result need sample size. Rough estimate: with base conversion 2% and desired minimum detectable effect 20% (relative), requires ~20,000 visits per variant. On site with 1000 visits/day — test runs 40 days.

Don't stop test until p-value drops below 0.05. Calculators: Evan Miller's AB Test Calculator, ABTestGuide.

What We Configure in One Day

1. Native abtest module — creating test with component template substitution.
2. Solving caching problem (separate cache by group).
3. Passing test group to dataLayer for analytics.
4. Report setup in Metrica/GA4 with A/B group split.
5. Calculating required traffic volume and test duration.