Imagine: you launch an AR game, enemies spawn on a real table, but after two seconds ARKit loses tracking – objects drift, physics breaks, and the player is furious. We've encountered this dozens of times over 5 years, having delivered over 30 AR projects. Developing stable AR mechanics requires accounting for many nuances: from RealityKit physics to optimization for devices with 3 GB RAM. Our team guarantees stable 60 FPS even on iPhone 11. Typical problems: tracking loss, unstable physics under changing lighting, gesture control anomalies, and pathfinding difficulties on AR surfaces. Payback for such a solution is under 6 months. Contact us to discuss your task and get a consultation.
How to choose a physics tool for AR?
RealityKit is the first choice for new projects. PhysicsBodyComponent + CollisionComponent provide native physics through ARKit's scene understanding:
var physicsBody = PhysicsBodyComponent(massProperties: .default, material: nil, mode: .dynamic)
physicsBody.isAffectedByGravity = true
entity.components[PhysicsBodyComponent.self] = physicsBody
var collision = CollisionComponent(shapes: [.generateBox(size: entity.visualBounds(relativeTo: nil).extents)])
entity.components[CollisionComponent.self] = collision
With arView.environment.sceneUnderstanding.options = [.physics] enabled, AR objects correctly interact with real surfaces. A ball rolls on a real table, a box stands on a real shelf.
The problem with RealityKit is limited control. You don't have access to individual physics engine steps or custom force generators. For simple mechanics, it's sufficient. For complex game physics (ragdoll, fluid simulation, rope physics), you need SceneKit with PhysicsWorld or Unity with AR Foundation.
| Tool | Simplicity | Control | Performance |
|---|---|---|---|
| RealityKit | ★★★★ | ★★ | ★★★★ |
| SceneKit | ★★★ | ★★★ | ★★★ |
| Unity AR Foundation | ★★ | ★★★★★ | ★★★★★ |
AR controls: gestures vs. aiming
Tap to place, pinch to scale, rotation – standard. But in AR, gestures compete with camera movement. The user wants to rotate an object – ARKit interprets it as hand tracking.
Solution via UIGestureRecognizer with shouldRequireFailure(of:): tap is recognized only if swipe has not started. For rotation – UIPanGestureRecognizer with a minimum displacement of 20 points before activation, so accidental camera movement doesn't trigger rotation.
A separate mechanic is 'aiming' via the screen center instead of tap. In shooters and strategies, it's more convenient: raycast from arView.center every frame, the object under the reticle is highlighted, action on button press. Implemented via ARView.raycast(from:allowing:alignment:) in a CADisplayLink callback.
How to handle tracking loss?
Tracking loss during gameplay. ARKit transitions to .limited tracking state – objects drift. For AR games this is a disaster: an enemy teleports 30 cm, a ball passes through a wall. Strategy: at .limited we freeze physics, show an overlay 'Point the camera at a surface', resume after .normal. We don't interrupt the game loop – only pause physics (set arView.scene.paused = true or scene.physicsWorld.speed = 0). We preserve body velocities for realistic resumption.
AR objects out of view. The player turns around, an AR enemy behind them continues moving and attacking. Frustum culling is needed not for rendering (ARKit does that automatically) but for game logic: AI of enemies outside the viewport can run simplified or be paused.
Spawning on uneven surfaces. Raycast returns the surface normal via ARRaycastResult.worldTransform – the fourth column of the matrix gives the point, X/Y/Z give orientation. If the surface is sloped 30°, the spawned object tilts with it. For games with 'flat' objects (tokens, chips), we fix the vertical: take only Y from the normal, construct transform with up = (0,1,0).
How to optimize AR game performance?
Optimization is critical for older devices. We use LOD for distant objects: simplified meshes, disable shadows beyond 0.5 meters. Reduce pathfinding graph update frequency from 60 to 20 times per second and cache results. On iPhone 11 this raised FPS from 45 to 60 with 6+ units – a 33% gain. For devices with 3 GB RAM, we additionally reduce shadow resolution by 50%.
Technical details of LOD
For iOS, we use `ModelIO` to generate LOD meshes. For each object we create 3 levels of detail: high (0-0.5 m), medium (0.5-1.5 m), low (>1.5 m). Switching by distance from camera. We attach a script to `ARView` that updates LOD in the `update` loop.Case study from our practice
A mobile AR strategy game: towers, enemies, real table as map. Our client wanted 6 unit types, pathfinding on mesh surface. The main problem: ARMeshAnchor updates asynchronously – while the mesh is rebuilt, the pathfinding graph becomes outdated. Our solution: navigation graph updated every 3 seconds on a background thread via GCD, and units used the cached graph. A unit in motion checked collision with the current mesh via raycast – thus we caught situations where the mesh changed and the unit walked through a wall. This reduced development time by 40%.
FPS stayed at 60 on iPhone 13. On iPhone 11 it dropped to 45 with 6+ units. Solution: LOD for distant units (simplified meshes), disable shadows beyond 0.5 meters. Result: stable 60 FPS on both devices.
Step-by-step AR mechanics development process
- Analytics and prototyping: Define key mechanics, test on real device.
- Architecture design: Choose stack (RealityKit/SceneKit/Unity), design physics and AI.
- Implementation: Create mechanics, integrate gestures, work with tracking.
- Optimization: LOD, caching, FPS profiling.
- Testing: Verify on different devices, under various lighting conditions.
- Deployment: Publish to App Store/Google Play.
Order turnkey AR game mechanics development – get a free consultation for your project.
What's included in the work
- Physics of AR objects interacting with real surfaces (RealityKit / SceneKit)
- Control system: gestures, aiming, UI buttons
- Game loop with ARKit tracking loss handling
- AI and pathfinding on AR surfaces
- LOD optimization for support of devices from iPhone 11
- Testing under real lighting conditions
- Guaranteed result and free support for one month after delivery
Timelines
| Mechanic | Timeline |
|---|---|
| Simple AR mini-games (tap, shoot) | 2–4 weeks |
| Strategy / tower defense with AI | 8–14 weeks |
| Full AR shooter with multiplayer | 16–24 weeks |
Cost is calculated after a detailed discussion of mechanics. Get a consultation – contact us for an assessment.