AI for delivery robots: navigation, teleoperation, fleet monitoring
Imagine: a delivery robot gets stuck at a curb, doesn't see a puddle, or loses GPS under a bridge. In an unstructured urban environment, standard algorithms fail. We solve this problem with a stack of computer vision, reinforcement learning, and teleoperation.
Our experience — 5+ years in AI for robotics, over 30 implemented projects. The main challenges are uneven terrain, dynamic obstacles, and lighting changes.
We have developed a system that processes 30 frames per second with multiple data sources, ensuring localization accuracy up to 10 cm.
Unlike traditional approaches, our RL planner reduces manual interventions by 90% compared to rules, and AI implementation reduces delivery costs by up to 40%.
Project success depends on precise ODD definition and data quality. We help collect and label datasets for model training, and also conduct simulation testing before moving to the real environment. We will evaluate your project and offer a turnkey solution.
What technical problems we solve
Urban sidewalks are full of edge cases: unevenness, ramps, intersections, unpredictable pedestrians. A delivery robot must:
- Detect objects with latency less than 33 ms (30 FPS).
- Distinguish surface types (asphalt, lawn, puddle, snow).
- Plan trajectory in real time taking into account dynamic obstacles.
- Handle temporary obstacles (construction fences, crowds).
Standard SLAM algorithms produce errors when there is no texture. We use Visual SLAM with IMU fusion and SLAM for accuracy up to 10 cm.
Why is an AI system critical for delivery robots?
Without AI, the robot cannot adapt to environmental changes. We use a combination of detection, segmentation, and depth estimation. The sensor package includes:
- 9–12 cameras for 360° coverage.
- 2–4 LiDAR (Livox Mid-360 or solid-state).
- Ultrasonic sensors for the near zone.
- RTK GPS + Visual SLAM.
Detection is based on YOLOv8 and RT-DETR, segmentation — SegFormer. Everything is optimized for NVIDIA Jetson Orin NX via TensorRT, achieving 30+ FPS per stream.
How do we ensure a 95%+ success rate in urban environments?
The global route is built on an HD map of sidewalks (OSM + proprietary labeling). The segment graph contains attributes: width, surface type, curb presence, illumination.
The local planner uses RL (TD3) with a continuous velocity space. Input — BEV 64×64 m with semantic layers. The agent is trained in Isaac Sim (NVIDIA Omniverse) with photorealistic urban scenes over a 10-second horizon.
For non-standard situations — an OOD detector with safe-stop and operator request.
| Situation | Strategy |
|---|---|
| Curb without ramp | Detour using HD map / search for ramp |
| Puddle / snow | Speed reduction, detour |
| Construction fence | Re-planning of global route |
| Crowd of pedestrians | Stop, wait for passage |
| Dog off leash | Soft stop, detour |
Sensor package and model details
- Cameras: fisheye, 1–2 MP, 30 FPS. - LiDAR: Livox Mid-360, 360° x 90° FOV. - IMU: inertial module with 6 DOF. - Compute: NVIDIA Jetson Orin NX, 100 TOPS. - Models: YOLOv8l, SegFormer-B2, UniDepth.Comparison of approaches: LiDAR vs cameras
| Parameter | LiDAR | Cameras | Combination |
|---|---|---|---|
| Range | up to 200 m | up to 100 m (stereo) | 200+ m |
| Accuracy in darkness | 100% | low | high |
| Semantic information | no | yes | yes |
| Cost | high | low | medium |
YOLOv8 detection shows mAP 0.55 on Cityscapes, while segmentation gives IoU 0.78.
Human-in-the-loop teleoperation
Full autonomy is achievable with a clearly defined ODD. Initially, some edge cases are handled by teleoperators:
- Video stream from 4 cameras (WebRTC, < 200 ms latency).
- Control via gamepad.
- All sessions are logged for DAgger training.
- Intervention dynamics: first month 15–25%, after 6 months 1–3%.
Fleet management and monitoring
Centralized Fleet Controller:
- Order dispatching to the nearest available robot taking into account charge.
- Predictive charging with a 20% buffer.
- Kafka + TimescaleDB for real-time monitoring.
Metrics: Mission Success Rate >95%, Average Delivery Time deviation <10%, Intervention Rate, MTBF >200 hours.
Implementation process
- Analytics: environment audit, ODD definition, data collection (3–4 weeks).
- Design: sensor architecture, model selection, pipeline (2–3 weeks).
- Implementation: calibration, model training, integration (8–12 weeks).
- Testing: simulation, real tests, iterations (4–6 weeks).
- Deployment: fleet installation, fleet management setup (2–3 weeks).
Timelines: MVP with basic sidewalk navigation — 4–5 months. Full system with teleoperation and fleet management — 9–12 months. Cost is calculated individually. Order an analysis of your tasks — we will select the optimal configuration.
What is included in the work
- Documentation: architecture description, operation instructions.
- Training of the customer's team (2–3 days).
- Access to the monitoring dashboard.
- Support during operation (SLA).
We guarantee a Mission Success Rate >95% after stabilization. Get a consultation on implementing an AI system for your project. Contact us to discuss the details.
