Developing a Mobile AR App for Museums

Developing a Mobile AR App for a Museum

Imagine: a visitor points their camera at a display case, but the glass glares, the marker isn't detected, the AR object jitters. In the museum—uneven lighting, old iPhone 8s with limited ARKit, a queue of visitors overloading the Wi-Fi. Our approach is to build an AR app that remains reliable under these conditions. Otherwise, the visitor becomes disappointed, and the museum loses its investment.

Over 7 years, we have completed 15+ projects for museums and exhibitions. Each required an individual solution: from calibrating markers under display case glass to optimizing 3D models for weak devices. Properly designed museum AR is not just an animation on an exhibit. It's integration with the exposition, navigation through halls, offline mode, and accessibility for all ages. We guarantee the app will work on visitor devices, including models five years old (iPhone 8 and up), and will not lose stability under congested Wi-Fi. Statistics show about 30% of visitors abandon the app due to poor AR performance—we aim to reduce this churn to zero.

How to Choose Tracking Technology for a Museum?

Two basic scenarios determine the architectural choice: Image Tracking—point at an exhibit or sign, and augmented content appears. ARImageTrackingConfiguration is simpler, does not require plane detection, and works through display case glass (if there are no strong reflections). Limitation: physical markers or highly textured images are needed. Location + Plane Detection—navigation through the hall with an AR guide. ARWorldTrackingConfiguration + ARGeoAnchor (for outdoors) or custom indoor positioning via Bluetooth Beacon / UWB. More complex, but gives freedom of movement.

In practice, museums use a hybrid: image tracking to activate content at specific exhibits, plane detection + beacons for navigation between halls.

Image Tracking: Implementation Details

Each AR marker is an ARReferenceImage with a physicalSize parameter (real size of the image in meters). Incorrect physicalSize → ARKit incorrectly estimates distance → content appears in the wrong position.

let referenceImage = ARReferenceImage(cgImage, orientation: .up, physicalWidth: 0.15) // 15 cm

ARImageTrackingConfiguration.maximumNumberOfTrackedImages—limit on simultaneously tracked images. Maximum: 4–8 (depends on device). In a hall with 50 exhibits, you cannot load all 50 at once. Solution: proximity detection via Bluetooth Beacons—activate tracking only for the nearest 5–6 markers.

Marker quality: ARReferenceImage.validate() checks the contrast histogram. Signs with fine text on a white background are poor markers. Solution: a dedicated QR code next to the sign (high uniqueness) or applying an invisible UV marker on the exhibit frame.

Why Offline Work Is Critical for Museum AR?

Museums often have no Wi-Fi in halls or it's overloaded. All AR resources must be available offline. Strategy: on first launch (or when connected to Wi-Fi)—background download of content for the current exhibition. BackgroundTasks framework for iOS, WorkManager for Android.

Content package size: 20–100 MB per exhibition—acceptable for a one-time download before the visit.

Content: From 3D Models to Video

Museum AR content is not just a 3D object. Typical set for one exhibit:

• 3D reconstruction (how it looked fully, in color—e.g., an ancient statue)
• Annotations with facts (hotspots)
• Animation (mechanism movement, creation process)
• Video clip (archival footage, curator interview)

AR video over the exhibit—via VideoMaterial in RealityKit:

let videoURL = Bundle.main.url(forResource: "artifact_story", withExtension: "mp4")!
let player = AVPlayer(url: videoURL)
let videoMaterial = VideoMaterial(avPlayer: player)
planeEntity.model?.materials = [videoMaterial]
player.play()

How We Ensure Accessibility for All Devices?

A museum is a public institution. The app must work for an audience from 8 to 80 years old, on visitors' own devices (not museum-issued), including old ones. Minimum device requirements: iOS 14+ (ARKit 4), Android 8.0+ with ARCore. Real minimum—iPhone 8 / iOS 14. Dynamic Type for captions in AR—via UIFont.preferredFont(forTextStyle:). Users with system large font should see readable annotations. Audio guide as an AR alternative—for users without compatible devices or with limited vision: a "Listen" button at each exhibit without requiring AR.

Step-by-Step AR App Development Process

1. Exhibit audit and tracking type selection—analysis of lighting, display cases, hall size, number of exhibits.
2. AR architecture design—creating marker schematics, configuring ARReferenceImage, testing in museum conditions.
3. iOS and Android version development—Swift + ARKit for iOS, Kotlin + ARCore for Android. Unified logic in C++ for calculations.
4. Integration with museum CMS—REST API to Axiell, MuseumPlus, or custom CMS. Admin panel for content updates without app re-release.
5. Setting up offline download and caching—BackgroundTasks / WorkManager, asynchronous download of 3D models and videos.
6. Testing on visitor devices—renting iPhone 8, 11, 13 and Android devices from different manufacturers.
7. Publishing to App Store and Google Play—preparing metadata, screenshots, passing reviews.
8. Documentation and curator training—instructions for updating content and using the admin panel.
9. 6-month warranty support—bug fixes, updates for new OS versions.

Integration with the Museum System

The museum's CMS (usually Axiell, MuseumPlus, or custom) stores exhibit data: inventory numbers, texts, images. Integration via REST API: on marker scan—request by inventory number → get text, multimedia. Cache locally.

For editing AR content without developers—admin panel: curator uploads 3D model, video, text → they are linked to inventory number → on next app sync, visitors see updated content.

Case Study: History Museum, 120 Exhibits

History museum, 3 thematic halls, 120 exhibits. Image tracking on QR codes next to signs (100 markers active simultaneously via lazy loading). Content: 3D reconstructions for 30 key exhibits, text annotations for the rest. Integration with MuseumPlus via REST API. Offline cache on hall entry via BLE beacons. Main issue: display case glass with anti-reflective coating distorted AR tracking. Solution: markers were placed on the outer side surfaces of display cases, not behind the glass. The core framework is ARKit from Apple, providing accurate image tracking and plane detection. Apple ARKit Documentation

What's Included in the Work

• AR tracking architecture design tailored to the specific exposition
• Creation of AR markers and testing in museum conditions
• Development of iOS (Swift, ARKit) and Android (Kotlin, ARCore) versions
• Integration with museum CMS (REST API) and admin panel for content
• Setting up offline download and caching
• Testing on visitor devices (iPhone 8+, Android 8+)
• Publishing to App Store and Google Play
• Documentation and curator training
• 6-month warranty support

Indicative Timelines

Development cost is calculated individually after analyzing the exposition. Image tracking is faster to implement but requires physical markers; location-based navigation takes longer but offers more interactivity. A hybrid approach often proves optimal in terms of cost and effect. Contact us for a consultation—we will evaluate your project and propose a solution. Order turnkey AR app development to make your exhibits come alive and attract more visitors.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.