Configuring Real Device Testing with Firebase Test Lab

Configuring Real Device Testing with Firebase Test Lab

Emulators lie. Not maliciously, but systematically: GPU rendering differs, Bluetooth is unavailable, the camera is simulated, memory behavior is not like a real device. At TrueTech, with over 5 years of experience on mobile projects, we've encountered this dozens of times. Firebase Test Lab provides access to physical devices — Pixel 8, Samsung Galaxy S24, tablets, old Xiaomi units with custom firmware. A test that passes on an emulator can crash on Samsung One UI due to a custom WebView or modified permissions system. Order a turnkey setup — we'll prepare the matrix, Robo Script, and CI integration in 2–3 days.

Why Emulators Fail and Real Devices Don't

Emulators don't reproduce hardware specifics: different memory behavior, lack of real sensors (accelerometer, gyroscope), emulated camera. Real devices expose issues with push notifications (APNs/FCM), background execution (Background fetch), and deep links (Universal Links / App Links). According to our data, about 30% of crashes found on physical devices are not reproducible on an emulator. Using Firebase Test Lab ensures your app is ready for real users.

What Firebase Test Lab Can Do

Two testing modes:

Instrumentation Tests — runs your Espresso / XCUITest tests on selected devices. Full control over what is checked.

Robo Test — automatic crawler. Launch an APK without writing a single test; Robo navigates the UI via the accessibility tree, taps buttons, fills fields with random data, and looks for crashes. Useful for quick checks of new builds before release.

Supported platforms: Android (Espresso, UI Automator, Appium) and iOS (XCUITest). Firebase Test Lab for iOS has a smaller device pool, but real iPhones and iPads are still more valuable than a simulator.

Setup: From Build Upload to Results

Android

Build APK and test APK:

./gradlew assembleDebug assembleAndroidTest 

Upload and run via gcloud:

gcloud firebase test android run \ --type instrumentation \ --app app/build/outputs/apk/debug/app-debug.apk \ --test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk \ --device model=Pixel8,version=34,locale=ru,orientation=portrait \ --device model=a54xnsxx,version=13,locale=ru,orientation=portrait \ --results-bucket gs://my-project-test-results \ --results-dir "run_$(date +%Y%m%d_%H%M%S)" 

model=a54xnsxx is a Samsung Galaxy A54. List available models: gcloud firebase test android models list.

Device Matrix

Choosing a matrix is a separate analytical task. No point in running tests on 50 devices. Principle:

A realistic matrix for most projects: 3–5 devices. More is costlier and slower, but not necessarily more informative.

iOS

For iOS, you need an .ipa with a development signature (not distribution). Upload:

gcloud firebase test ios run \ --test MyAppTests.zip \ --device model=iphone15pro,version=17.4,locale=ru_RU,orientation=portrait 

MyAppTests.zip is an archive with the .xctestrun file and test products. Build via xcodebuild build-for-testing.

Robo Test with Custom Script

The plain Robo crawler sometimes gets stuck on the login screen — it doesn't know credentials. Robo Script allows you to define initial actions:

[ { "eventType": "VIEW_TEXT_CHANGED", "replacementText": "[email protected]", "elementDescriptors": [{"resourceName": "com.example.app:id/email_input"}] }, { "eventType": "VIEW_CLICKED", "elementDescriptors": [{"resourceName": "com.example.app:id/login_button"}] } ] 

After authentication, Robo continues crawling the authenticated part of the app. This is a quick way to check regressions without writing tests.

CI Integration

GitHub Actions:

- name: Set up gcloud uses: google-github-actions/setup-gcloud@v2 with: service_account_key: ${{ secrets.GCLOUD_SERVICE_ACCOUNT_KEY }} project_id: my-firebase-project - name: Run tests on Firebase Test Lab run: | gcloud firebase test android run \ --app app-debug.apk \ --test app-debug-androidTest.apk \ --device model=Pixel8,version=34 \ --timeout 10m 

A service account with the Firebase Test Lab Admin role is needed. Results are automatically saved to Cloud Storage — screenshots, videos, logcats, XML reports.

Analyzing Results

In the Firebase Test Lab console, for each run you get:

• Video recording of the test
• Logcat with tag filters
• Screenshots at key points
• Coverage report (if jacoco enabled)
• XML results in JUnit format (for Allure / Jenkins)

What's Included in Turnkey Setup

We offer a full cycle:

• Analysis of your device park based on analytics data
• Configuration of a 3–5 device matrix
• Writing a Robo Script for authentication (if needed)
• CI integration (GitHub Actions, GitLab CI, Bitrise)
• Team training on working with results
• Guarantee of correct operation for 5 days after delivery

Our engineers have experience with 30+ projects using Firebase Test Lab. Contact us for a project assessment — we'll find the optimal configuration.

Timeline

2–3 days — setting up Firebase Test Lab, configuring the device matrix, CI integration, writing Robo Script for authentication. Plus time for initial analysis of results and fixing device-specific issues. Cost is calculated individually.

How Firebase Test Lab Saves Time on Regressions

Robo tests without a single line of code find crashes and ANRs on real devices faster than manual testing. According to our data, this saves up to 40% of QA cycle time. Combined with instrumentation tests, you get full coverage in a matter of hours.