Image Tracking Implementation for AR Apps

Why tracking markers gets lost on glossy packaging?

Image tracking — anchoring AR content to a physical image: product packaging, poster, business card, book page. User points the camera — the image 'comes alive.' Technically it's a well-studied problem, but in production you regularly hit the same issues: tracking 'shakes' on reflective surfaces, gets lost under partial occlusion, doesn't scale to large marker catalogs. We are a team with 5+ years of AR development experience (iOS/Android) and solve these problems turnkey. 50+ AR projects completed, each with its own nuances. Contact us to evaluate your project — we guarantee stable tracking and optimal quality.

ARKit Image Tracking: how it works

ARImageTrackingConfiguration — configuration for tracking without world tracking. Initializes faster, lower CPU load, but no plane detection or world anchors.

ARWorldTrackingConfiguration with detectionImages — marker tracking in the context of full world tracking. Needed when AR content must exist in world space between frames or when plane detection is needed simultaneously with image tracking.

Reference image preparation: ARReferenceImage with physical size (physicalSize). Size is mandatory — ARKit calculates distance and scale from it. Incorrect size → object at wrong scale.

let image = UIImage(named: "marker"),
// get cgImage

let referenceImage = ARReferenceImage(cgImage, orientation: .up, physicalSize: CGSize(width: 0.15, height: 0.10))
referenceImage.name = "product_label"
config.detectionImages = [referenceImage]
config.maximumNumberOfTrackedImages = 4

maximumNumberOfTrackedImages — critical parameter. ARKit A12+ tracks up to 100 images simultaneously (detection), but active position tracking — up to 4 on older chips, up to 8 on A14+. Difference: detected — we know marker exists; tracked — we know exact real-time position.

Marker quality and why 'any image' doesn't work

ARKit evaluates quality score for each reference image. Images with low quality score track unstably or not at all. Check: add image to Xcode AR Resources group → inspector shows warning for low quality.

Bad markers:

• Solid colors or large uniform areas (logo on white background)
• Symmetric patterns (ARKit confused about orientation)
• Low contrast, faded images
• Text without other visual elements

Good markers:

• High contrast, heterogeneous patterns (magazine covers, detailed illustrations)
• Asymmetric — ARKit unambiguously determines orientation
• Physical size from 10 cm — small markers track from distance less than 30 cm

Apple recommends using images with diverse details and avoiding uniform areas.

Tracking on reflective surfaces

Packaging with glossy coating, holographic stickers, foil elements — all produce reflections that change marker appearance depending on lighting angle. ARKit loses tracking because feature points 'float.'

Solution at physical product level: matte lamination on the marker area. At code level: hysteresis for tracking loss — do not hide AR content immediately when trackingState == .limited, but with a 0.5-1 second delay. Most brief losses recover themselves.

ARCore Image Tracking: comparison with ARKit

AugmentedImageDatabase — analogue of ARKit detection images. We compile the database in advance via arcoreimg utility (command line) or AugmentedImageDatabase(session:imageBytes:) at runtime. Precompiled database loads faster.

ARCore additionally provides AugmentedImage.getTrackingMethod(): FULL_TRACKING vs LAST_KNOWN_POSE. LAST_KNOWN_POSE allows keeping AR content at the last known position even when the marker temporarily leaves the frame.

How to scale Image Tracking to hundreds of markers?

For applications with a large catalog (100+ markers — e.g., all SKUs of a product line), you cannot bundle all reference images into the app. Architecture:

• Server stores reference images + AR content
• On detecting a new marker (via external ID in QR or cloud recognition) — load content for that specific marker
• Cloud Image Target (Vuforia Cloud, Wikitude Cloud): client sends frame to server, server returns marker ID and transform. Works for catalogs of 100k+ images

What's included in turnkey Image Tracking development?

1. Analysis of your markers — quality assessment, recommendations for improvement
2. Preparation of reference images (conversion, cropping, size adjustment)
3. Integration of ARKit or ARCore (configuration selection, state handling)
4. Testing on 5+ real devices (different cameras, lighting)
5. Performance optimization (CPU/GPU, battery)
6. Documentation and source code delivery
7. Post-launch support (2 weeks)

Contact us to get a consultation and timeline estimate.

How we work: step by step

• You send a task description and marker samples
• We analyze and prepare a commercial proposal
• After agreement — stages: analytics → design → implementation → testing → deployment
• You receive a ready solution with stable tracking guarantee

Timelines

Basic image tracking with 1-10 markers, static 3D content — from 3 to 5 days. Animated content, video overlay, catalog management via CMS — 2-3 weeks. Cloud solution for 100k+ markers — separate evaluation. Pricing determined individually.

Our experience: over 5 years in AR development, certified Apple and Google engineers. Write to us — we'll evaluate your project for free.

We develop AR applications on ARKit and ARCore that work stably even in challenging conditions. Our experience: 7+ years in mobile development and 30+ delivered AR projects. Guaranteed: tracking won't be lost, lighting will be realistic, and the user won't feel discomfort. Certified Apple and Google developers.

Why does tracking get lost and how to fix it?

ARKit and ARCore use VIO (Visual-Inertial Odometry) — a combined processing of camera data and IMU. Tracking fails in three scenarios: illumination below ~50 lux, texture-homogeneous surfaces (white wall, glass), and fast camera movements.

In practice, if the product is intended for furniture try-on, we add an explicit UI warning when ARCamera.TrackingState.limited(.insufficientFeatures). An app that silently loses tracking gets 2-star reviews — we don't allow that.

Plane detection is configured via ARWorldTrackingConfiguration.planeDetection = [.horizontal, .vertical]. Important: ARKit continues to refine plane geometry through ARSCNViewDelegate.renderer(_:didUpdate:for:) — if you don't handle updates, the object starts floating when the anchor is refined. Our team solves this at the architecture stage, not during testing.

AR Foundation: cross-platform with nuances

Unity AR Foundation is an abstraction layer over ARKit and ARCore. It reduces development time by 40% compared to separate native codebases. But some features (e.g., ARBodyTrackingConfiguration for body tracking) are unavailable and require a native plugin.

For React Native and Flutter, direct AR Foundation is missing. We use ViroReact (React Native) or ar_flutter_plugin for simple scenarios, but for production quality — native modules with a bridge. Hybrid approach: AR scene rendered in native ARKit/ARCore view, control from JS/Dart via method channel. Included in our standard delivery.

Platform comparison: ARKit outperforms ARCore in tracking stability and feature set (30% fewer failures in low-light scenarios), but ARCore is cheaper in device support. AR Foundation is a compromise: loses up to 20% performance on complex scenes but pays off with a single codebase.

Try-on: product fitting via AR

Fitting glasses, jewelry, cosmetics — a separate class of tasks. Here, face tracking is needed, not plane detection.

ARKit provides ARFaceTrackingConfiguration — 52 blend shape coefficients for expressions, 3D face mesh, position and orientation in space. Works only on devices with TrueDepth camera (iPhone with Face ID).

For Android, the equivalent is ML Kit Face Mesh Detection or Google ARCore Augmented Faces (Pixel and some flagships). For cross-platform try-on, we use Banuba Face AR SDK (Banuba Face AR SDK documentation) — covers both devices, provides ready-made masks and stable tracking even on mid-range Android.

Try-on quality critically depends on 3D product models. Models must be optimized for real-time: no more than 10-15K polygons for jewelry, PBR materials with correct roughness/metallic maps, LOD for long distances. Within our engagement, we provide ready-made optimization guides.

How to achieve realistic lighting in AR?

ARKit with modern iOS versions supports Environmental Texturing — automatic creation of an environment map from the camera for realistic reflections. Enabled via ARWorldTrackingConfiguration.environmentTexturing = .automatic. Without it, metallic and glass materials look plastic.

ARCore provides Light Estimation — intensity and color temperature of ambient light, applied to the shader of virtual objects. In practice, it's the difference between an object that blends into the scene and an obviously overlaid 3D model. We guarantee that the final image doesn't betray virtuality.

What's included

• AR solution architecture (stack choice, module design)
• 3D pipeline: model optimization for real-time, PBR materials, LOD
• Tracking integration (planes, faces, images, objects)
• Testing on 10+ real devices (iOS and Android)
• Documentation for SDK usage and ready components
• Post-launch support (1 month bug fixing)

Timeline and estimation

Simple AR scene with placing one 3D model on a plane — 1-2 weeks. Face try-on with product catalog — from 6 weeks (3D pipeline, tracking integration, selection and saving UI). Full AR shopping with cloud anchors and multiplayer — from 3 months. We'll estimate your project in 1 day — contact us to discuss your AR idea.