Implementing an AI Chatbot in a Mobile App
Integrating GPT-4o or Claude into a mobile chat isn't just "connect an SDK and you're done." The real complexity begins after the first working request: managing dialogue context, displaying streaming generation without UI jank, handling network issues on weak signals, and storing chat history between sessions without leaking personal data. We are a team with 8 years of mobile development experience, having delivered 40+ projects with AI features. We offer turnkey AI chatbot integration: from model selection to store publication. We'll evaluate your project in 1–2 days.
How to Manage Dialogue Context Without Losing Quality?
All LLMs are stateless. Each request to OpenAI, Anthropic, GigaChat, or YandexGPT sends the full dialogue history. This means: storing and truncating context is your job. A naive implementation after 20 messages can increase token cost by 3–4 times, and with a 128k context, you might wait 30+ seconds for a response.
A practical solution is a sliding window with summarization:
class ConversationManager { private var messages: [ChatMessage] = [] private let maxMessages = 20 private let summaryThreshold = 15 func addMessage(_ message: ChatMessage) { messages.append(message) if messages.count > summaryThreshold { Task { await compressSummary() } } } private func compressSummary() async { // Take messages before threshold, summarize with a separate LLM request let toCompress = Array(messages.prefix(10)) let summary = try? await llmClient.summarize(messages: toCompress) if let summary { messages = [ChatMessage(role: .system, content: "Context: \(summary)")] + Array(messages.suffix(10)) } } } The system prompt is a separate story. It must always remain the first message. Do not touch it when compressing context.
Streaming Generation and UI
Users shouldn't wait for a full response. Streaming via SSE is the standard for all modern LLM APIs. On iOS:
// Update SwiftUI View via @Published class ChatViewModel: ObservableObject { @Published var streamingText = "" func streamResponse(for prompt: String) { streamingText = "" Task { for try await chunk in llmClient.stream(prompt: prompt) { await MainActor.run { streamingText += chunk } } } } } On Android with Compose, use StateFlow<String> collected with collectAsState(). A typical mistake: calling notifyDataSetChanged() or recreating a RecyclerView adapter on each chunk — this causes visible flickering. Update only the last message's text, not the entire list.
Offline Mode: On-device vs Cloud
| Criteria | On-device model | Cloud LLM (GPT-4o) |
|---|---|---|
| Latency | ~15 tokens/sec (iPhone 15 Pro) | 50–200 ms to first chunk |
| Privacy | Data stays on device | Data sent to provider |
| Complexity | Requires chip-specific optimization | Ready-made API |
| Use case | FAQ, autocomplete | Creative responses, summarization |
For basic scenarios, use Apple Intelligence API (iOS 18+) or SmartReply from ML Kit. For more complex ones, use llama.cpp via Metal/CoreML. We guarantee correct operation with any stack.
Storing Dialogue History
Chat history contains personal data. Use SQLite/Core Data with encryption via SQLCipher or iOS Data Protection. Do not store history in UserDefaults — it syncs to iCloud without encryption. On Android, use Room with EncryptedSharedPreferences for encryption keys.
Cleanup strategy: auto-delete dialogues older than N days, or explicit deletion on user request — this is a requirement of GDPR and Russian Federal Law 152-FZ.
What's Included in the Work
- Architecture: LLM provider selection, on-device vs cloud, authorization scheme.
- Backend proxy with rate limiting, caching, logging.
- ConversationManager: sliding window, summarization, system prompt.
- Chat UI: bubble layout, streaming, typing indicator, reaction buttons.
- History storage: encryption, auto-cleanup, export.
- Moderation API: input and output filtering.
- Testing edge cases: network loss, long responses, concurrent requests.
- Documentation: README, flow diagram, deployment instructions.
- Support: 2 weeks after handover (consultation, bug fixes).
Typical Production Issues
Repetitive responses. GPT sometimes gets stuck on a pattern. Parameters presence_penalty: 0.6 and frequency_penalty: 0.3 reduce the likelihood. If stuck, implement client-side detection: if the last 3 bot messages contain >60% identical n-grams, reset the context.
Timeout on poor network. LLMs can generate slowly. Default URLSession timeout is 60 seconds, which is too short for long streaming responses. Set timeoutIntervalForResource: 120 and add an extra progress indicator "thinking..." after 5 seconds of no first chunk.
Moderation. OpenAI Moderation API before sending user input is mandatory for public apps. One POST /v1/moderations is cheaper than dealing with an App Store Review complaint.
Process
- Architecture design: LLM provider selection, on-device vs cloud, authorization scheme.
- Backend proxy development with rate limiting.
- ConversationManager implementation with context management.
- Chat UI: streaming, bubble layout, typing indicator.
- Dialogue history with encryption.
- Edge-case testing: network loss during generation, very long responses, concurrent requests.
Timeline Estimates
A simple chatbot with one LLM provider and no history: 5–7 days. A full-featured chatbot with history, context compression, offline mode, and moderation: 3–5 weeks. Cost is calculated individually. Contact us for a consultation and project estimate within 1–2 days.