A/B testing with Multi-Armed Bandit: implementation and optimization

Smart A/B Testing: Multi-Armed Bandit Turnkey

A classic A/B test requires waiting for statistical significance — often weeks at moderate traffic, losing conversions on a suboptimal variant. Multi-Armed Bandit adapts in real time: automatically redistributes traffic in favor of the better variant while data accumulates.

For example, for an online store with 50,000 visitors per day, each day of using a suboptimal variant costs hundreds of lost orders. We implement MAB turnkey — from choosing the algorithm to integration with your experiment management platform.

When Is MAB More Profitable Than Classic A/B?

High-frequency decisions (email subject lines, push notifications, UI elements). When the cost of error is high — for example, losing 10% conversion for a week means tens of thousands in lost revenue.

When there are more than two variants — the classic test requires exponential sample growth, while MAB scales easily to dozens of variants.

How Do MAB Algorithms Work?

Thompson Sampling — a Bayesian approach: for each variant we maintain a beta distribution of conversion probabilities. On each request: sample from the distributions → show the variant with the highest sample → update the distribution based on the result.

The exploration/exploitation balance is built in mathematically. Epsilon-Greedy is simpler to implement: with probability ε — a random variant (exploration), with probability 1-ε — the current best (exploitation). ε decreases over time (ε-decay).

Contextual Bandit extends MAB by accounting for user context: device, traffic source, on-site behavior. We use LinUCB, NeuralLinear — each user sees the optimal variant for their profile.

Comparison of MAB and Classic A/B

How Do We Implement MAB?

We use Python (Vowpal Wabbit — instant performance, up to 1 million requests per second on a single core) or custom code in PyTorch. Redis for storing statistics with session timeouts. Feature flags platform (Unleash, LaunchDarkly) for managing variants and rollback.

Monitoring: cumulative regret (losses from suboptimal choices), conversion by variant over time, traffic distribution.

One of the projects: for an e-com site with 500k visits/day, we reduced regret by 37% in two weeks by switching traffic away from an ineffective banner that was shown 60% of the time. The Thompson Sampling algorithm provided the optimal balance.

How to Implement MAB: Step-by-Step Instructions

1. Audit of the current experimentation infrastructure: analysis of traffic, goals, existing A/B tests.
2. Selection of the MAB algorithm (Thompson Sampling, Epsilon-Greedy, Contextual Bandit) taking your data into account.
3. Integration via API, SDK, or feature flags: we support Python, Node.js, Go.
4. Monitoring setup: dashboards with cumulative regret, conversion by variant, exploration rate.
5. Launch and optimization: parameter adjustment, A/B validation.

Implementation timeline: 3–5 weeks depending on integration complexity.

What Is Included in the Work?

• Audit of the current experimentation infrastructure: analysis of traffic, goals, existing A/B tests.
• Selection of the MAB algorithm for your task and stack.
• Integration via API or SDK: Python, Node.js, Go — for any backend.
• Monitoring and dashboards: metrics, regret, conversion, exploration rate.
• Documentation and team training: how to interpret results, how to add new variants.
• Support during the first month: parameter adjustment, help with interpretation.

Why Should You Choose Experienced Engineers?

Errors in MAB setup are costly: the wrong choice of ε leads to excessive exploitation of a bad variant, and ignoring context leads to incorrect conclusions. Our engineers are certified ML specialists with 10+ years of experience in production ML.

We have implemented MAB for 20+ projects, including fintech and e-com with million-strong audiences. Request a consultation — we will analyze your project and propose the optimal solution.