AI Content Recommendations: Tracking, Personalization, and A/B Testing

A user lands on your site, sees a generic list of articles, and leaves within 10 seconds. The content isn't relevant; interest is lost. We've encountered dozens of such projects: content exists but engagement is low. AI recommendations solve this—they analyze behavior and deliver personalized content. Our experience: over 20 projects in this niche, guaranteeing at least a 30% increase in page depth. Implementation proceeds without performance degradation, with A/B testing at every stage.

Why AI Recommendations Increase Page Depth

Personalization directly impacts metrics. Content-based approaches boost engagement by 20–40%, collaborative filtering by 50–70%, and hybrid systems up to 80%. We use A/B testing to find the optimal algorithm for your site. The final improvement depends on data quality and the chosen model.

How to Set Up Event Tracking Without Performance Loss

Behavioral data is the foundation for any recommendation. We implement a client-side tracker that sends events via sendBeacon. This doesn't block rendering or affect Core Web Vitals. We analyze views, scrolls, reading time. All data is aggregated in PostgreSQL and used to train models. Tracking is configurable with flexible filtering—we collect only the needed signals, avoiding noise.

Comparison of Recommendation Approaches

Content-Based: Quick Start with Embeddings

The fastest way to get relevant recommendations is to find similar articles by vector distance. We use OpenAI's text-embedding-3-small model to create embeddings. Indexing happens on publication, and search is done via pgvector. Cosine similarity between vectors gives a similarity metric. Accuracy is 85-90% on test sets.

import OpenAI from 'openai';
import { sql } from '@vercel/postgres';

const openai = new OpenAI();

async function indexArticle(article) {
  const textToEmbed = [
    article.title,
    article.excerpt,
    article.tags.join(', '),
    article.body.slice(0, 2000),
  ].join('\n\n');

  const { data: [{ embedding }] } = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: textToEmbed,
  });

  await sql`
    UPDATE articles
    SET embedding = ${JSON.stringify(embedding)}::vector
    WHERE id = ${article.id}
  `;
}

async function getSimilarArticles(articleId, limit = 6) {
  const result = await sql`
    WITH source AS (
      SELECT embedding FROM articles WHERE id = ${articleId}
    )
    SELECT
      a.id, a.title, a.slug, a.excerpt, a.published_at,
      a.category, a.read_time,
      1 - (a.embedding <=> source.embedding) AS similarity
    FROM articles a, source
    WHERE a.id != ${articleId}
      AND a.published = true
      AND a.embedding IS NOT NULL
    ORDER BY a.embedding <=> source.embedding
    LIMIT ${limit}
  `;

  return result.rows;
}

Collaborative Filtering: When Data is Plentiful

Matrix factorization via implicit feedback (views, time on page). We use the implicit library with the AlternatingLeastSquares algorithm. The model is trained server-side and saved for fast predictions. Training frequency is once per day.

import implicit
import numpy as np
from scipy.sparse import csr_matrix
import pickle

def train_collaborative_model():
    events = fetch_events_from_db()
    users = {u: i for i, u in enumerate(events['user_id'].unique())}
    items = {a: i for i, a in enumerate(events['article_id'].unique())}
    rows = events['user_id'].map(users)
    cols = events['article_id'].map(items)
    data = events['weight']
    matrix = csr_matrix((data, (rows, cols)))
    model = implicit.als.AlternatingLeastSquares(
        factors=128, regularization=0.01, iterations=50, use_gpu=False
    )
    model.fit(matrix)
    with open('/models/collab_model.pkl', 'wb') as f:
        pickle.dump({'model': model, 'users': users, 'items': items}, f)

Hybrid System: Best of Both Worlds

We combine content-based, collaborative, and trending signals with weights. Content-based is faster, but hybrid yields 20% more clicks. Example implementation with weights 0.4, 0.5, 0.1:

async function getPersonalizedRecommendations(userId, currentArticleId) {
  const [contentBased, collaborative, trending] = await Promise.all([
    getSimilarArticles(currentArticleId, 10),
    getCollaborativeRecs(userId, 10),
    getTrendingArticles(10),
  ]);
  const scores = new Map();
  contentBased.forEach((article, i) => {
    scores.set(article.id, (scores.get(article.id) || 0) + (10 - i) * 0.4);
  });
  collaborative.forEach((article, i) => {
    scores.set(article.id, (scores.get(article.id) || 0) + (10 - i) * 0.5);
  });
  trending.forEach((article, i) => {
    scores.set(article.id, (scores.get(article.id) || 0) + (10 - i) * 0.1);
  });
  const allArticleIds = [...scores.keys()];
  const articles = await fetchArticlesByIds(allArticleIds);
  return articles
    .map(a => ({ ...a, score: scores.get(a.id) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, 6);
}

LLM Recommendations with Explanations

For personalized collections with understandable justifications, we use GPT-4o-mini. Users see not only the recommendation but also the reason: 'Because you read about React.' This boosts trust and click-through rate by 25%.

What's Included

Each implementation stage is completed with artifact delivery. The final package includes:

• Tracking schema and configured client-side tracker
• Recommendation API with documentation (OpenAPI)
• Metrics and A/B test dashboard (Grafana/Tableau)
• Access to the model and code repository
• Team training and text instructions
• 30-day support guarantee post-deployment

We don't just deploy code—we hand over a tool you can customize yourself.

Implementation Stages

Timelines and Cost

Timelines range from 3 days for a basic pgvector solution to 3 weeks for a full hybrid system with LLM. Cost is calculated individually after audit. We guarantee results contractually. Request AI recommendation implementation and get engagement growth. Contact us for a consultation—we'll assess your project and propose an optimal solution.

pgvector

AI Integration: Chatbots, RAG, Semantic Search, Recommendations

In 8 out of 10 projects, an "AI chatbot" turns out to be an expensive wrapper over GPT-4o with a system prompt. Without access to real company data. The user asks "how much does the Premium plan cost?" — the bot hallucinates a price out of thin air. Asks "when will my order arrive?" — gets a polite "contact support." This is not integration — it's imitation. We have implemented RAG solutions in 30+ projects over 5 years: from e-commerce stores to medical portals. We guarantee: useful AI assistance begins where the model reads your documents, not generic answers.

How do we build RAG systems?

Retrieval-Augmented Generation — standard architecture: query → find relevant fragments in a vector DB → insert found context → model response. But the devil is in the implementation details. Let's break down key components that determine quality.

Chunking. Cutting a document into 500-token pieces without regard for structure is a guarantee of losing meaning. If the cut lands in the middle of a paragraph, context breaks. Solution — recursive RecursiveCharacterTextSplitter with 10–15% overlap for documentation. For contracts and instructions, we use a semantic splitter: extract headings, lists, code blocks — each section becomes an independent chunk. Difference in search quality: on a medical project, precision increased from 0.55 to 0.84 just by proper chunking.

Embedding model. For Russian-language texts, intfloat/multilingual-e5-large gives a noticeable accuracy boost over outdated text-embedding-ada-002. In our measurements, NDCG@10 on a test set of 10,000 query-document pairs is 12% higher. OpenAI text-embedding-3-large is good for English content, but for Russian we recommend BAAI/bge-m3 or the mentioned e5-large.

Vector DB. If you already have PostgreSQL — pgvector saves resources. Install extension CREATE EXTENSION vector, add column vector(1024), create HNSW index. On a project with 80,000 support articles, p95 search time was 12 ms. That's enough. For catalogs with millions of items — Qdrant or Weaviate: native hybrid search and sharding out of the box.

What does hybrid search give?

Vector-only search is blind to exact matches: SKUs like "ABC-123", proper names, abbreviations are lost. Full-text-only search doesn't catch synonyms and paraphrasing. Combining via RRF (Reciprocal Rank Fusion) gives the best of both worlds: BM25 + vector search, results merged. In practice, recall@20 increases from 0.65 to 0.92 — the difference is noticeable to the user.

Reranking — final filter: top-20 candidates from hybrid search are run through a cross-encoder cross-encoder/ms-marco-MiniLM-L-6-v2. It adds 50–100 ms to response time, but relevance improves by another 5–10%. Without reranking, the chatbot may show irrelevant documents.

How to implement semantic search on a site?

A search for "comfortable leather armchairs" should find products described as "soft chairs made of natural leather" — ordinary LIKE search cannot do this. Our architecture: when adding a product/post, automatically generate an embedding via multilingual-e5-large, store it in pgvector. On query, embed it with the same model, search nearest neighbors via cosine distance with HNSW index. For a catalog of 100,000 items, index builds in 3 minutes, memory ~400 MB (1536-dimensional vectors). Average search time: 20 ms.

What about recommendation systems?

Collaborative filtering ("users like you bought X") requires history — at least 2–3 months of data with 1000+ active users. For startups or small projects, we use content-based: embedding of current product → search nearest neighbors by cosine similarity. When enough statistics accumulate (usually 15–20 interactions per user), we switch to a hybrid LightFM model. It combines behavior and product features. In our e-commerce project with 50,000 SKUs, the hybrid model increased conversion in the recommendation block by 18% (A/B test lasted 2 weeks).

How does streaming work?

Users shouldn't wait for the entire text to be generated — it kills UX. Server-Sent Events (SSE) is the protocol for token streaming. OpenAI SDK supports stream: true, returning an AsyncIterator. On frontend — Vercel AI SDK (useChat) or custom EventSource. Typical mistake: using WebSocket for unidirectional streaming — SSE is simpler (less code, built-in reconnect). Stack: Node.js + SSE + React.

How to orchestrate agents?

A simple chatbot answers. An agent performs actions: creates a Jira ticket, checks order status in CRM, books a calendar slot. For orchestration, we use LangGraph: state graph where each node is a model or tool call. Vercel AI SDK useChat + tools for Next.js allows adding integration in 10 lines of code. Main challenge — reliability: the model sometimes calls the wrong tool or passes malformed parameters. Protection — Zod schemas for each tool and structured outputs to guarantee JSON.

What does the work include?

Additionally: we hand over vectorizer source code, monitoring dashboards (Langfuse), admin panel access for knowledge base updates. Post-production support — 1 month free.

What are the timelines?

Pricing is calculated individually after project discovery. We'll evaluate your project in 1 day. Contact us — we'll show how to turn AI from a toy into a profit-driving tool.