AI Observability Setup (LangSmith, LangFuse, Helicone, Weights & Biases)

Setting up AI observability with LangSmith and Langfuse

LangSmith (LangChain) and Langfuse are specialized platforms for observability of LLM applications: call chain tracing, request costs, quality assessment, and prompt regression testing.

LangSmith Setup

pip install langchain langsmith
export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=ls__xxx
export LANGCHAIN_PROJECT=my-llm-app

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

# Трассировка включается автоматически через env переменные
llm = ChatOpenAI(model="gpt-4o")
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant"),
    ("user", "{question}")
])
chain = prompt | llm

# Все вызовы автоматически логируются в LangSmith
result = chain.invoke({"question": "What is RAG?"})

What you can see in LangSmith: full call tree trace, each LLM call with prompt/response, latency of each step, cost (tokens × price), errors with full stack trace.

Langfuse Setup (self-hosted)

# Docker Compose для self-hosted
docker compose up -d  # из langfuse/langfuse репозитория

from langfuse import Langfuse
from langfuse.decorators import observe, langfuse_context

langfuse = Langfuse(
    public_key="pk-xxx",
    secret_key="sk-xxx",
    host="http://localhost:3000"  # self-hosted
)

@observe()  # автоматически создаёт trace
def process_user_query(query: str) -> str:
    # Каждая вложенная @observe функция — span внутри trace
    context = retrieve_context(query)
    response = generate_response(query, context)

    # Оценка качества прямо в коде
    langfuse_context.score_current_trace(
        name="relevance",
        value=0.9,
        comment="Context was relevant"
    )
    return response

@observe(name="retrieve_context")
def retrieve_context(query: str) -> str:
    # ... vector search
    pass

Cost monitoring

Both platforms automatically calculate the cost based on the model and the number of tokens. Alerts: daily budget exceeded, cost per request increased by >2x, abnormal increase in token consumption.

Comparison of platforms

Langfuse is preferred for data residency requirements and self-hosting.