Integrating Pose Estimation into a Mobile App
While developing a fitness app, we found that the raw PoseNet model produced false positives on every second frame—insufficient temporal stability. The solution was to implement pose tracking based on MediaPipe with custom OneEuroFilter smoothing. We integrate body keypoint detection into mobile apps—real-time identification of skeleton landmarks. This is in demand in fitness (rep counting, technique analysis), medicine (rehabilitation, gait diagnosis), and AR. Technically, the task is more complex than object detection: it requires not only accurate localization but also temporal stability. Our team has 5+ years of experience in computer vision and has delivered over 30 projects with pose analysis. Cost is determined after analysis and depends on the number of exercises and platforms. Time savings compared to in-house development can be as high as 40%. Typical engagements start at $3,000.
Choosing a Model: MoveNet, BlazePose, or ML Kit?
| Model | Points | Speed (on iPhone 12) | Typical Scenario |
|---|---|---|---|
| MoveNet Lightning | 17 | 30+ FPS | Fitness rep tracking |
| MoveNet Thunder | 17 | ~15 FPS | Medical analysis with higher accuracy |
| MediaPipe BlazePose | 33 | ~25 ms (Pixel 7 GPU) | Gait analysis, AR with face/hand detail |
| ML Kit Pose Detection | 33 | ~30 FPS | Fast cross-platform integration |
MoveNet Lightning is the best balance for mobile: TFLite-optimized, available via PoseLandmarker. BlazePose provides z-coordinates for 3D angles. ML Kit is simple but slightly less accurate. In our case, MoveNet Lightning was 2x faster than BlazePose on iPhone 11 (35 FPS vs 17 FPS), critical for real-time feedback.
Correct Rep Counting Approach
The naive approach—tracking the hip’s Y-coordinate—breaks in reality. The correct solution is to compute the knee joint angle via the scalar product of vectors [HIP → KNEE] and [KNEE → ANKLE]. A squat is an angle below 120°, standing up above 160°. State machine: STANDING → DOWN → STANDING = 1 rep. Angles using z-coordinates are more stable if the camera is not strictly from the side. Landmark smoothing is mandatory: raw data jumps 3–5 pixels between frames. Additionally, we use a median smoothing filter to clip outliers.
Best Smoothing Methods
| Method | Latency | Stability | Application |
|---|---|---|---|
| EMA (α=0.6) | Low | Medium | Fast response, fitness |
| VelocityFilter (MediaPipe) | Medium | High | Medical, AR |
| OneEuroFilter | Tunable | High | Universal |
For fitness, EMA is enough; for rehabilitation, OneEuroFilter. In our projects, we customize OneEuroFilter parameters per exercise: for squats low cutoff (min_cutoff=0.5), for fast movements higher (min_cutoff=1.0).
Integration: iOS and Android—Step by Step
- Select model based on required accuracy and speed. MoveNet Lightning to start.
- Prepare stack: MediaPipe Tasks Vision (SPM or Gradle) version 0.10.0.
- Configure inference: PoseLandmarker with
runningMode = .liveStreamandPoseLandmarkerOptions. - Process results: normalized points (0..1) or
worldLandmarks(in meters). For angles use worldLandmarks—they are independent of camera crop. - Render skeleton: on iOS—CAShapeLayer with animation, on Android—Canvas.drawLine on SurfaceView or Compose Canvas.
- Test—check FPS and accuracy on target devices (iPhone 11, Pixel 6, Galaxy S21).
Common implementation mistakes:
-
Drawing the skeleton in normalized coordinates without transforming to preview coordinates—forgetting aspect ratio and crop. Use
convertNormalizedLandmarksToImageCoordinates. - Running inference on the main thread—drops FPS to zero. Use background queues (userInteractive on iOS, SingleThreadExecutor on Android).
- No landmark smoothing—visual jitter and incorrect rep counting. Even simple EMA (α=0.7) solves it.
What Is Included in the Work
- Consultation on model selection and feasibility assessment.
- Architecture design (inference, UI, history storage).
- Integration of body keypoint detection on iOS and/or Android.
- Implementation of repetition counting or movement analysis logic.
- Custom smoothing and optimization for specific devices.
- Deliverables: complete source code for the pose module, detailed documentation for usage and calibration, access to code repositories, training for your development team.
- Post-launch support (bug fixes, updates, guarantee stability).
Timeline
Basic skeleton on video stream + one exercise: 1–2 weeks. Full fitness module with multiple exercises, voice feedback, and history: 3–4 weeks. Cost starts from $2,000 for basic integration and goes up to $8,000 for a full module with multiple exercises.
Contact us for a project assessment—we will help you choose a model and design a turnkey solution. Get a consultation on pose estimation integration today.