Meta Llama API Integration via Together AI Fireworks Groq

Meta Llama API Integration via Together AI, Fireworks, Groq

Llama 3 and 3.1/3.2 are the most powerful open-source LLMs from Meta, available through cloud providers without needing own infrastructure. Together AI, Fireworks AI, and Groq provide OpenAI-compatible APIs, simplifying integration and migration.

Together AI — Broadest Model Selection

from openai import OpenAI

# Together AI uses OpenAI-compatible API
together_client = OpenAI(
    api_key="TOGETHER_API_KEY",
    base_url="https://api.together.xyz/v1",
)

response = together_client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Explain how the attention mechanism works"}],
    temperature=0.1,
    max_tokens=2048,
)
print(response.choices[0].message.content)

# Available Llama models via Together:
LLAMA_MODELS = [
    "meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo",  # Maximum quality
    "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",   # Balance
    "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",    # Fast and cheap
    "meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo", # Multimodal
]

Groq — Extremely Fast Inference

from groq import Groq

groq_client = Groq(api_key="GROQ_API_KEY")

# Groq uses LPU (Language Processing Unit) — specialized hardware
# Speed: 500–800 tokens/sec vs 50–100 tokens/sec from GPU providers
response = groq_client.chat.completions.create(
    model="llama-3.1-70b-versatile",
    messages=[{"role": "user", "content": "Need a fast answer"}],
    temperature=0,
)

# Available models in Groq:
GROQ_MODELS = [
    "llama-3.1-70b-versatile",
    "llama-3.1-8b-instant",
    "mixtral-8x7b-32768",
    "gemma2-9b-it",
]

Fireworks AI — Optimized Inference

from openai import OpenAI

fireworks_client = OpenAI(
    api_key="FIREWORKS_API_KEY",
    base_url="https://api.fireworks.ai/inference/v1",
)

response = fireworks_client.chat.completions.create(
    model="accounts/fireworks/models/llama-v3p1-70b-instruct",
    messages=[{"role": "user", "content": "Request"}],
)

Provider Selection

Local Deployment (Ollama)

ollama pull llama3.1:70b
ollama pull llama3.2:3b  # For CPU

local_client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
response = local_client.chat.completions.create(model="llama3.1:8b", messages=[...])

Timeline

• OpenAI-compatible API integration: 0.5 day
• Provider comparative testing: 1–2 days
• Fallback setup between providers: 1–2 days