Firebase A/B Testing Setup: Reliable Experimentation Without Errors

Suppose you launched an A/B test for a new onboarding, and a week later conversion dropped — and it's unclear whether it's random or an effect. We've seen many times how incorrect Remote Config activation breaks the experiment: the user sees both variants in one session, making data useless. To get reliable results, you need to control several technical aspects. Our implementation experience — over 5 years, dozens of projects — ensures that experiments yield statistically significant conclusions.

Why Firebase A/B Testing needs preparation

Firebase A/B Testing is an overlay on Remote Config. You create an experiment in the console: define a parameter, a control group (current value) and variants (new ones). Firebase distributes users on the server, and with fetchAndActivate each gets their value. But if activation happens after screen render — the user sees a switch, and the experiment gets contaminated.

A typical mistake is overlapping experiments: two tests change the same screen independently. Firebase allows parallel experiments, but the responsibility for avoiding conflicts lies with the team. We maintain a table of active experiments with parameters being changed.

How to avoid experiment overlap?

Overlap is a common cause of unreliable data. Solution: before launching a new experiment, check that no active test changes the same Remote Config parameter. Use a single table (e.g., in Confluence) with columns: experiment name, changed parameters, start date, expected end date. We guarantee that after auditing your active experiments, conflicts will be eliminated.

Why statistical significance is critical

Without sufficient data volume, the experiment result is just noise. Firebase uses Bayesian statistics: it computes the probability that the variant is better than control. But stopping the test early can give a high probability by chance. For a 5% conversion rate, you need at least 500 conversions per group. With smaller sample sizes, decisions lead to metric degradation. Our engineers always calculate the required sample size before launch.

How we set up experiments

1. Formulate the hypothesis: what we change, which metric we impact, what effect we expect.
2. Create a parameter in Remote Config with type and default value.
3. Verify that client code reads the parameter before rendering the target screen.
4. Launch the experiment in Firebase console with the target Analytics event.
5. Monitor statistical significance: for conversions below 5%, we need at least 500–1000 conversions per group.
6. When sufficient data is reached, decide: lock the variant or revert to control.

Implementation on iOS

// Config already set up via RemoteConfig
// In the experiment: parameter "paywall_position" = "bottom" (control) / "center" (variant)

remoteConfig.fetchAndActivate { [weak self] _, _ in
    let position = RemoteConfig.remoteConfig()["paywall_position"].stringValue
    DispatchQueue.main.async {
        self?.paywallViewModel.position = position == "center" ? .center : .bottom
    }
}

It's mandatory to log the trigger event — Firebase A/B Testing uses it to mark "experiment seen":

Analytics.logEvent("experiment_paywall_viewed", parameters: [
    "variant": RemoteConfig.remoteConfig()["paywall_position"].stringValue ?? "unknown"
])

Implementation on Android (Kotlin + Jetpack Compose)

val remoteConfig = FirebaseRemoteConfig.getInstance()
remoteConfig.fetchAndActivate().addOnCompleteListener { task ->
    if (task.isSuccessful) {
        val position = remoteConfig.getString("paywall_position")
        // Apply to UI
        viewModel.position = if (position == "center") Position.Center else Position.Bottom
    }
}

On Flutter (via firebase_remote_config)

final remoteConfig = FirebaseRemoteConfig.instance;
await remoteConfig.setConfigSettings(RemoteConfigSettings(
  fetchTimeout: const Duration(seconds: 10),
  minimumFetchInterval: const Duration(hours: 1),
));
await remoteConfig.fetchAndActivate();

final paywallPosition = remoteConfig.getString('paywall_position');

Activation strategy comparison

What's included in our work

• Setting up Remote Config with parameters specific to the experiment.
• Typed access to experimental parameters (to eliminate typos).
• Integration at the trigger point (before rendering the target screen).
• Configuring target events in Firebase Analytics for conversion measurement.
• Consultation on experiment design: hypothesis, metric, minimum sample size.

Timeline and cost

From 1 day (if Remote Config is already integrated) to 3 days (from scratch, including analytics and consultation). Cost is calculated individually — we estimate the workload after reviewing your project. Request an audit of your experiment and get free consultation.

Example experiment metrics

Compared to a custom solution: Firebase A/B Testing is 3x simpler — no need to write server-side distribution logic, and Bayesian statistics are built in. Our experience — over 5 years in mobile development — guarantees correct setup. Contact us for consultation — we'll help you avoid typical mistakes and get reliable results. Order Firebase A/B Testing integration for your app.

Mobile App Analytics: Firebase, Amplitude, AppsFlyer and Attribution

Our team regularly encounters projects where analytics is already "set up" but yields no real insights. A typical example is a startup with 50k DAU: tracking dozens of events without a single answer to the question "why don't users reach payment?". In two weeks we built a basic funnel and found that 70% of users drop off at the phone number verification screen. After fixing the bug, retention increased by 12%. The takeaway: analytics should start with specific questions, not tracking everything indiscriminately.

Why Event Taxonomy is the Foundation of Mobile App Analytics?

Firebase Analytics, Amplitude, Mixpanel — technically similar. The difference lies in what you put into them. A common mistake: events like screen_view, button_tap_1, button_tap_2 without context. A month later, no one remembers what button_tap_2 means.

Proper taxonomy: object + action + context. product_viewed, checkout_started, payment_completed with parameters product_id, category, price, source. This allows building funnels, cohort analysis, and retention without additional tracking.

We document the naming convention in a tracking plan — a document (Google Sheet or Amplitude Data Catalog) describing every event, its parameters, and triggering conditions. The tracking plan is synced with the analytics team before development begins, not after. This approach ensures that data remains interpretable months later and doesn't become a dump. Experience from 50+ projects confirms: without a tracking plan, analytics maintenance costs increase 2-3 times due to rework.

What Should You Choose for Mobile App Analytics: Firebase, Amplitude, or Mixpanel?

The table below highlights key differences between the three popular platforms. Choice depends on budget, traffic, and tasks.

Firebase Analytics — free, deep integration with Google Ads, BigQuery export for raw data. Limitations: data latency up to 24 hours, limited funnels. For startups with Google Ads traffic, it's the first choice.

Amplitude — product analytics focused on cohorts and user journeys. Journeys (formerly Pathfinder) shows actual paths between events — not assumed funnels but real routes. Session Replay records sessions for UX analysis. The free tier up to 10M events/month is enough for most products at launch.

Mixpanel — close to Amplitude, stronger in real-time segmentation. Insights, Funnels, Retention cover 90% of product analysts' tasks.

How to Solve Multi-Channel Attribution with AppsFlyer?

Knowing where a user came from is a separate task. Firebase Attribution works only within the Google ecosystem. For multi-channel attribution (Facebook Ads, TikTok, Apple Search Ads, programmatic), an MMP (Mobile Measurement Partner) is needed.

AppsFlyer is the market leader. OneLink — universal deep link working on iOS and Android, correctly attributing installs from any channel. Protect360 — built-in fraud protection (fake installs, click injection on Android). Adjust and Branch are competitors with similar features. Branch excels in deep linking; Adjust is popular in gaming.

According to Apple, with iOS 14.5, apps must obtain user permission via ATT before collecting IDFA for tracking. AppsFlyer uses probabilistic matching (IP + user agent + timing) for these users — accuracy is lower but better than nothing. SKAdNetwork and Privacy Preserving Attribution provide aggregated data from Apple with a 24-72 hour delay.

How to Set Up Crash Analytics to Not Miss Bugs?

Firebase Crashlytics is the standard for crash reporting. It automatically groups crashes by stack trace, shows affected users %, and sends velocity alerts when crash rate increases by more than 10% per hour.

Important: symbolication. On iOS, .dSYM files must be automatically uploaded with each build — via Fastlane upload_symbols_to_crashlytics or Xcode Cloud built-in. Without symbols, crashes in Crashlytics appear as memory addresses. This happens more often than expected when switching to a new CI — in one project with 500k users, we found that 40% of crashes remained unsymbolicated due to a missing CI/CD step. After automation, bug response time dropped from 3 hours to 15 minutes.

For React Native and Flutter, @sentry/react-native and sentry_flutter provide additional context: breadcrumbs, network requests before the crash, Redux/Provider state.

Below is a comparison of popular crash analytics tools to choose according to your needs.

Environment Setup

Three environments with separate Firebase projects: dev, staging, production. Mixing analytics from test sessions and production is a common mistake that skews all metrics. On iOS via GoogleService-Info.plist per scheme, on Android via google-services.json in each flavor folder.

Timelines: basic analytics with Firebase + Crashlytics — 3-5 days. Full tracking plan + Amplitude/Mixpanel with funnels and cohorts — 2-3 weeks. Attribution via AppsFlyer with deep linking and fraud protection — 1-2 weeks. Cost is calculated individually based on integration complexity.

What Is Included in Our Work

As part of analytics implementation, we provide:

• Development and approval of a tracking plan with product and marketing teams.
• SDK integration (Firebase, Amplitude, Mixpanel, AppsFlyer) considering your stack (Swift/Kotlin/Flutter/React Native).
• Setup of funnels, cohorts, dashboards, and alerts.
• Automation of symbolication and .dSYM upload via Fastlane.
• Documentation of events and parameters.
• Team training on the analytics platform.
• Two weeks of post-release support and tracking adjustments.

Our experience: 7 years of analytics implementation and over 80 successful projects in mobile development. We guarantee data correctness and transparency at every stage.

Contact us for a consultation on setting up analytics for your app. Request an audit of your current analytics — and we will show you which metrics you are losing.