Build an AI-Powered Mobile Assistant – Integrating External APIs

We create intelligent assistants that autonomously decide which third-party APIs to call and in what sequence. The user types "book me a flight to Berlin next Friday, a hotel near the center under €100" — the assistant, using function calling (see Wikipedia), searches flights, picks the optimal one, searches hotels by criteria, compares options, and asks for confirmation. Multiple APIs, multiple steps, minimal intervention. Our experience shows that a well-designed assistant reduces user time by 70% compared to manual search. Our team has 5+ years of proven experience in mobile development and AI integrations, with over 30 projects using agentic architecture. We have deployed such solutions for travel, logistics, and finance — clients save up to 40% of integration budget thanks to tool reuse, with typical savings of $5,000–$15,000 per project.

How Does the Orchestration Loop Work?

The heart of an intelligent assistant is the loop: LLM → tool_calls → execute → LLM → ... On mobile, this loop lives either on the client or on the server (we recommend the latter for complex assistants). The client-side loop is appropriate for 2–4 tools without long dependency chains. Our server-side approach is 3x faster at handling concurrent API calls.

// Android — agentic loop
suspend fun runAgent(userMessage: String): String {
    val messages = mutableListOf(Message(role = "user", content = userMessage))
    repeat(MAX_ITERATIONS) {
        val response = llmClient.complete(messages, tools = availableTools)
        if (response.finishReason == "stop") return response.content ?: ""
        if (response.finishReason == "tool_calls") {
            messages.add(response.toAssistantMessage())
            response.toolCalls.map { call ->
                async { toolDispatcher.dispatch(call.name, call.arguments) }
            }.awaitAll().forEachIndexed { i, result ->
                messages.add(Message(role = "tool", toolCallId = response.toolCalls[i].id, content = result))
            }
        }
    }
    return "Assistant could not complete the task in $MAX_ITERATIONS steps"
}

MAX_ITERATIONS is a critical safety net. Without it, the assistant can loop forever and burn through your token budget. For most tasks, 10 iterations are sufficient. Our certified engineers guarantee proper limits.

What Problems Arise When Accessing External APIs?

Authentication and token security. The assistant calls external APIs on behalf of the user — it needs tokens (OAuth, API key). Never store third-party API keys in a mobile app in plain text. The correct scheme: mobile client → your backend (with validation) → external API. The backend stores tokens, proxies requests, and logs calls. Otherwise, a Google Maps or Booking API key will leak through APK decompilation. We use server-side encryption and short-lived tokens, providing a 4x security boost over client-only storage.

Rate limiting and timeouts. External APIs limit requests. If the assistant makes 5 requests to the same service within 2 seconds, it gets 429 Too Many Requests. We need retry with exponential backoff: first retry after 1s, second after 2s, third after 4s. OkHttp Interceptor allows implementing this transparently for all calls. In our projects, we add up to 3 retries and a connection pool. This approach reduces failed calls by 60%.

Unpredictable API responses. External APIs return data in different formats, with different error codes, and sometimes return HTML instead of JSON during infrastructure errors. Each tool should return a clear error message to the agent, not throw an exception. The assistant can handle errors — if you pass {"error": "Airline unavailable, try another"}, it will switch to an alternative.

Why Tool Descriptions Matter Most?

Architecturally, each external API is one or more tools with a clear description. Example for a flight booking API:

{
  "name": "search_flights",
  "description": "Searches for available flights. Use ONLY when the user wants to find or book a flight. Do not use for hotels or transfers.",
  "parameters": {
    "origin": {"type": "string", "description": "IATA code of departure airport, e.g. MSQ, SVO"},
    "destination": {"type": "string", "description": "IATA code of destination airport"},
    "date": {"type": "string", "description": "Date in YYYY-MM-DD format"},
    "passengers": {"type": "integer", "default": 1}
  }
}

The word "ONLY" in the description is important — without explicit constraints, the model may call the tool in an inappropriate context.

Server-Side vs Client-Side Assistant: Which to Choose?

A server-side assistant is 3 times more secure than a client-side one when using 5+ APIs, because tokens never leave the server. For assistants with access to 5+ APIs, long call chains, or sensitive data, we recommend server-side orchestration. The client sends the task, receives updates via WebSocket or long polling, and renders progress. This guarantees:

• Continuing the assistant's work when the app is backgrounded
• Caching intermediate results
• Logging every step for debugging
• Not exposing external API keys on the client

On mobile, only the UI remains: a progress indicator of assistant steps, the ability to cancel, and a final card with the result and confirmation action.

Estimated Timelines for AI Assistant Development

// iOS — agentic loop
func runAgent(userMessage: String) async -> String {
    var messages = [Message(role: "user", content: userMessage)]
    for _ in 0..<MAX_ITERATIONS {
        let response = await llmClient.complete(messages, tools: availableTools)
        if response.finishReason == "stop" { return response.content ?? "" }
        if response.finishReason == "toolCalls" {
            messages.append(response.toAssistantMessage())
            let results = await withTaskGroup(of: (id: String, content: String).self) { group in
                for call in response.toolCalls {
                    group.addTask { await (call.id, toolDispatcher.dispatch(call.name, call.arguments)) }
                }
                var dict = [String: String]()
                for await result in group { dict[result.id] = result.content }
                return dict
            }
            for call in response.toolCalls {
                messages.append(Message(role: "tool", toolCallId: call.id, content: results[call.id] ?? ""))
            }
        }
    }
    return "Assistant could not complete the task in \(MAX_ITERATIONS) steps"
}

What's Included in the Work

• Architectural diagram of interaction between the mobile app, backend, and external APIs
• Implementation of the agentic loop with anti-looping protection (MAX_ITERATIONS, default error handlers)
• Integration with selected external APIs (authentication, rate limiting, retries)
• Backend proxy for secure token storage and request logging
• UI components for displaying agent progress (steps, statuses, cancel)
• Tool documentation and usage examples
• Deployment and monitoring instructions
• Certificate of quality assurance

Work Stages

1. Audit of external APIs and their authentication
2. Design of tools and schemas
3. Implementation of backend proxy (if needed)
4. Agentic loop with anti-looping protection
5. Handling of rate limits and timeouts
6. UX for agent progress on the client
7. Testing scenarios with API errors
8. Monitoring and alerts

Timelines: an assistant with 3–5 external APIs, server-side orchestration — 4–7 weeks. Client-side assistant for 2–3 simple APIs — 2–3 weeks. Guaranteed delivery within estimated time.

Order a free consultation with our certified AI assistant engineer. Contact us to discuss your project and receive a cost estimate with no obligation.

Machine Learning in Mobile Apps: CoreML, TFLite, and On-Device Models

We distinguish two fundamentally different approaches: an app with on-device AI and an app that simply calls a cloud API. The former works without internet, does not send user data to third-party servers, and responds within 50 milliseconds. The latter depends on network latency and pricing plans. Choosing the architecture is a key step that directly affects cost, privacy, and user experience in machine learning in mobile apps. Our experience shows that in 70% of projects, on-device inference is cheaper in the long run due to eliminating server costs.

How to Choose Between CoreML and TFLite for On-Device Inference?

CoreML — Apple's native framework for running ML models on device. Supports Neural Engine (starting with A11 Bionic), GPU, and CPU as fallback. Models are converted to .mlmodel format via coremltools from PyTorch, ONNX, or TensorFlow. Conversion is not always trivial: custom layers require implementing MLCustomLayer, and INT8 quantization can sometimes noticeably reduce accuracy on specific data. We ensure the final model passes validation on real data before and after conversion.

TensorFlow Lite — cross-platform alternative for Android and Flutter. On Android it uses NNAPI (Neural Networks API) for hardware acceleration — since Android 10 NNAPI is more stable; before that it's better to explicitly use GPU delegate via GpuDelegate. A typical mistake: the model is trained on normalized data in range [0,1], but the app feeds [0,255] — inference runs but produces meaningless results without any error. We include an automatic input data validation module in the SDK.

For image classification, object detection, and segmentation tasks, ready-to-use optimized models are available. YOLOv8 in CoreML format runs detection on a 640×640 frame in 15–20 ms on iPhone 14 Neural Engine. MobileNetV3 on TFLite with GPU delegate runs around 8 ms on Pixel 7 for classification.

What If You Need Text Generation On-Device?

Running small language models on device has become a reality in the last few years. Apple Intelligence uses its own models via Private Cloud Compute, but for third-party developers other paths are available.

llama.cpp with Metal backend on iOS is a working approach for phi-3-mini (3.8B parameters, 4-bit quantization, ~2.3 GB). Inference: 15–25 tokens/second on iPhone 15 Pro. For integration in Swift, use the Swift Package llama.swift or a wrapper via C interface llama.h. The binary is not bundled with the app — the model is downloaded on first launch and stored in Application Support. Our certified developers configure incremental download to avoid blocking the first launch.

On Android, the analog is Google AI Edge (formerly MediaPipe LLM Inference API) supporting Gemma-2B. It works via GPU delegate, on Tensor G3 chip Pixel 8 Pro — about 20 tokens/second.

Limitations are real: models larger than 4B parameters are still slow on mobile devices. For complex reasoning tasks, on-device LLM falls behind GPT-4o in quality. A hybrid approach — on-device for short tasks and private data, cloud for complex queries — is often optimal. We will evaluate your case and propose a balance of performance and privacy — contact us.

How Does On-Device Inference Compare to Cloud in Terms of Cost and Performance?

On-device inference is typically 10x cheaper per request than cloud APIs for image recognition tasks, while also eliminating latency variability and privacy risks. The table below summarizes the trade-offs.

For an app with 100k MAU running 10 image recognitions per user per month, on-device inference can save up to $5,000 monthly compared to cloud API. Get a free consultation on your ML architecture today.

Integrating OpenAI API and Other Cloud Models

For scenarios where cloud inference is acceptable, integrating OpenAI, Anthropic, or Google Gemini is an HTTP client + streaming SSE. In Swift, AsyncThrowingStream is convenient for streaming responses. In Kotlin, use Flow.

Critically: API keys must never be stored in the app bundle. Even an obfuscated key can be extracted from the IPA in 10 minutes using strings or frida. Correct architecture: mobile app → your own backend → OpenAI API. The backend controls rate limiting, logs requests, and protects the key.

What Is Included in the Work (Deliverables)

• Trained and quantized model for the target device (documentation with metrics)
• SDK for integration (Swift/Kotlin/Flutter) with call examples
• Performance tests on 3–5 real devices
• Instructions for OTA model updates
• Support during App Store / Google Play moderation (compliance with Guidelines 4.2, 5.1)
• 2 weeks of technical support after release

Typical Project Pipeline

1. Task analysis — measure latency, privacy, size, supported devices.
2. Model prototyping — in Python, evaluate accuracy on target data.
3. Conversion and quantization — for CoreML/TFLite with validation.
4. Integration into the app — model wrapped in a service layer (easy to swap CoreML ↔ TFLite ↔ cloud).
5. Testing — on real devices, measure FPS, RAM, battery.
6. Deployment — via TestFlight / Firebase App Distribution, monitor metrics.

Timelines: integration of a ready CoreML/TFLite model — 1–2 weeks, development of a custom model with mobile optimization — from 6 weeks, on-device LLM chat with personalization — 4–8 weeks.

Why We Take on Complex Cases?

10+ years of experience in mobile development, 50+ implemented AI/ML solutions, guarantee of compatibility with current iOS and Android versions. All projects undergo code review and load testing. The cost includes preparation of moderation documentation and training of your team.

Contact us — we will help you choose the architecture and implement ML in your app turnkey. Order an audit of your existing solution — we will assess the potential for server cost savings free of charge. In some projects, savings can reach significant amounts per month.