LLM fine-tuning for mobile app tasks

Implementing LLM Fine-Tuning for Mobile App Tasks

Baseline GPT-4 or Llama 3 knows neither your domain, internal jargon, nor user specifics. Prompt engineering helps to a point: system prompt can inject context, but model still hallucinates on specialized terms, missets priorities, or generates wrong format answers. Fine-tuning is intervention level beyond: you modify model weights, not just instructions.

When Prompt Engineering Isn't Enough

Three scenarios justify fine-tuning:

Format determinism. Model must return strictly structured JSON with custom domain-specific fields. Even few-shot examples in prompt won't prevent base model periodically breaking schema or adding extraneous fields. After fine-tuning on 5,000–10,000 "request → correct JSON" pairs, format errors nearly disappear.

Domain terminology. Medical app with ICD-10 terms, legal assistant with article numbers, fintech with internal product codes — base model confuses or interprets abbreviations generically. Fine-tuning on your document corpus solves this.

Style and tone. Brand voice matters. If assistant must answer in specific character style or formality level, cheaper to bake into weights than inject into every request via system prompt.

Dataset Preparation — Most Critical Part

80% of fine-tuning success depends on training data quality, not hyperparameter choice.

Formatting for OpenAI Fine-Tuning API (gpt-4o-mini or gpt-3.5-turbo base):

{"messages": [
  {"role": "system", "content": "You are a medical app assistant. Answer symptom questions briefly and safely."},
  {"role": "user", "content": "What's resting HR of 45 bpm?"},
  {"role": "assistant", "content": "Bradycardia. Normal for trained athletes. With dizziness/fainting — see cardiologist."}
]}

Minimum volume for noticeable result — 50–100 examples (OpenAI allows), realistic for production — 500–2,000 pairs. Auto-generated datasets via GPT-4 require manual validation: auto-created examples reproduce base model errors.

For open-source models (Llama 3, Mistral, Gemma 2), datasets typically formatted as Alpaca or ShareGPT, fed to Hugging Face datasets.

Approach Choice: OpenAI vs Open-source

For most mobile products, OpenAI Fine-Tuning is fastest path to results. If data can't leave control (medical, finance) — open-source on own server or local llama.cpp/CoreML.

Fine-tuned Model Integration into Mobile App

After training, fine-tuned model gets ID like ft:gpt-4o-mini-2024-07-18:org:name:xxxxx. Mobile code change — substitute this ID:

// iOS — Swift, OpenAI SDK
let request = ChatCompletionRequest(
    model: "ft:gpt-4o-mini-2024-07-18:my-org:medical-assistant:abc123",
    messages: conversationHistory,
    maxTokens: 256,
    temperature: 0.3  // lower temperature = more deterministic answers
)

// Android — Kotlin, Retrofit
data class ChatRequest(
    val model: String = "ft:gpt-4o-mini-2024-07-18:my-org:medical-assistant:abc123",
    val messages: List<Message>,
    val max_tokens: Int = 256,
    val temperature: Double = 0.3
)

API-level — no difference. Same REST endpoint, same response format.

Quality Assessment and Iterative Improvement

Fine-tuning isn't one-time. Standard cycle:

1. Baseline measurement on test set (15–20% data, held before training)
2. Train → run A/B test in app on 10% traffic
3. Collect user feedback (likes/dislikes, answer corrections)
4. Augment dataset with problematic examples
5. Retrain

OpenAI Fine-Tuning Dashboard shows training loss and validation loss per epoch. Overfitting visible as divergence — validation loss grows, training continues falling. Reduce epochs or increase dataset.

Process

Current prompt audit and bottleneck identification → dataset collection and labeling → preparation in required format → training with control metrics → fine-tuned model integration → A/B test → iterative dataset augmentation.

Timeline Estimates

Dataset preparation from scratch (500–1,000 examples) — 2–4 weeks including validation. Training on OpenAI — 2–6 hours. Mobile app integration — 1–3 days. Full cycle from audit to production — 3–8 weeks. With ready annotated data — 1 week.