AI-Powered CTR/CVR Prediction for Ad Campaigns

You launch an ad campaign with a million impressions per day. CTR prediction is underestimated by 15% — you lose top auction positions. Overestimated by 20% — you waste budget on irrelevant clicks. We solve this with custom end-to-end ML models. Our AI system for CTR/CVR prediction analyzes audience, context, and historical data to optimize bids in real time. A 20% error in CTR prediction can cost 15-25% of the ad budget — money lost on ineffective impressions.

CTR and CVR are fundamental signals for pricing in programmatic advertising. A 20% error in CTR prediction directly translates into overpayment or under-winning auctions. At the scale of hundreds of millions of daily impressions, even improving AUC from 0.76 to 0.78 means millions saved or earned. According to our data, LightGBM with calibration outperforms logistic regression by 0.03-0.05 AUC — a direct efficiency comparison.

How We Build CTR Models End-to-End

CTR prediction is binary classification with three key challenges: extreme class imbalance (CTR 0.1-2%), massive volume (billions of daily examples), and hidden conversions (CVR observed only for clicks, creating selection bias). We use LightGBM with scale_pos_weight tuning and subsequent calibration. Calibrated LightGBM yields Log Loss 1.3 times lower than raw probabilities.

import numpy as np
import pandas as pd
import lightgbm as lgb
from sklearn.calibration import CalibratedClassifierCV
from sklearn.metrics import roc_auc_score, log_loss

class CTRFeatureEngineer:
    """Features for CTR model in display advertising"""

    def build_features(self, bid_logs: pd.DataFrame) -> pd.DataFrame:
        """
        bid_logs: historical impression logs with clicked/converted flags
        """
        df = bid_logs.copy()

        # === User statistical features ===
        user_stats = df.groupby('user_id').agg(
            user_historical_ctr=('clicked', 'mean'),
            user_impression_count=('clicked', 'count'),
            user_conversion_rate=('converted', 'mean'),
        ).reset_index()

        # === Site statistical features ===
        site_stats = df.groupby('site_domain').agg(
            site_ctr=('clicked', 'mean'),
            site_conversion_rate=('converted', 'mean'),
            site_volume=('clicked', 'count'),
        ).reset_index()

        # === Cross features (user × ad) ===
        df['hour'] = pd.to_datetime(df['timestamp']).dt.hour
        df['is_weekend'] = pd.to_datetime(df['timestamp']).dt.dayofweek >= 5
        df['is_prime_time'] = df['hour'].between(18, 22)

        # Cross features are more important than individual ones
        df['ad_position_encoded'] = df['ad_position'].map({'atf': 1, 'btf': 0}).fillna(0.5)

        df = df.merge(user_stats, on='user_id', how='left')
        df = df.merge(site_stats, on='site_domain', how='left')

        # Smoothed CTR to handle sparsity (Wilson smoothing)
        alpha = 100  # Prior strength
        global_ctr = df['clicked'].mean()
        df['user_smooth_ctr'] = (
            df['user_historical_ctr'].fillna(global_ctr) * df['user_impression_count'].fillna(0) +
            global_ctr * alpha
        ) / (df['user_impression_count'].fillna(0) + alpha)

        feature_cols = [
            'user_smooth_ctr', 'user_impression_count',
            'site_ctr', 'site_volume',
            'hour', 'is_weekend', 'is_prime_time',
            'ad_position_encoded', 'banner_width', 'banner_height',
            'floor_price',
        ]

        return df[feature_cols].fillna(0)


class CTRModel:
    """LightGBM for CTR with proper calibration"""

    def __init__(self):
        self.model = lgb.LGBMClassifier(
            n_estimators=1000,
            learning_rate=0.03,
            num_leaves=255,
            min_child_samples=200,
            subsample=0.8,
            colsample_bytree=0.7,
            scale_pos_weight=50,  # Correction for imbalance: 1 click per 50 impressions
            random_state=42,
            n_jobs=-1,
        )
        self.calibrator = None
        self._is_calibrated = False

    def train(self, X_train: np.ndarray, y_train: np.ndarray,
               X_val: np.ndarray, y_val: np.ndarray):
        """Training with early stopping"""
        self.model.fit(
            X_train, y_train,
            eval_set=[(X_val, y_val)],
            eval_metric='auc',
            callbacks=[
                lgb.early_stopping(100, verbose=False),
                lgb.log_evaluation(200)
            ]
        )

        # Calibration — MANDATORY for use in bid price calculation
        # Raw LightGBM gives good ranking but poor probabilities
        self.calibrator = CalibratedClassifierCV(self.model, cv='prefit', method='isotonic')
        self.calibrator.fit(X_val, y_val)
        self._is_calibrated = True

    def predict_ctr(self, X: np.ndarray) -> np.ndarray:
        """Calibrated click probabilities"""
        if self._is_calibrated:
            return self.calibrator.predict_proba(X)[:, 1]
        return self.model.predict_proba(X)[:, 1]

    def evaluate(self, X_test: np.ndarray, y_test: np.ndarray) -> dict:
        raw_probs = self.model.predict_proba(X_test)[:, 1]
        cal_probs = self.predict_ctr(X_test) if self._is_calibrated else raw_probs

        return {
            'auc_raw': round(roc_auc_score(y_test, raw_probs), 4),
            'auc_calibrated': round(roc_auc_score(y_test, cal_probs), 4),
            'logloss_raw': round(log_loss(y_test, raw_probs), 4),
            'logloss_calibrated': round(log_loss(y_test, cal_probs), 4),
            'mean_predicted_ctr': round(float(cal_probs.mean()), 5),
            'actual_ctr': round(float(y_test.mean()), 5),
        }


class DelayedConversionCorrector:
    """
    Correction for delayed conversions in CVR models.
    Conversions can occur hours/days after a click.
    Truncating the training set by time creates bias.
    """

    def adjust_for_delayed_conversions(self, clicks: pd.DataFrame,
                                         observation_window_hours: int = 24) -> pd.DataFrame:
        """
        Discard recent clicks where the conversion window hasn't elapsed.
        Otherwise CVR will be underestimated for recent examples.
        """
        cutoff = pd.Timestamp.now() - pd.Timedelta(hours=observation_window_hours)
        return clicks[clicks['click_time'] < cutoff]

    def estimate_conversion_delay_distribution(self,
                                                 conversions: pd.DataFrame) -> dict:
        """Distribution of conversion delays"""
        delays = (conversions['conversion_time'] - conversions['click_time']).dt.total_seconds() / 3600

        return {
            'p50_hours': round(float(delays.quantile(0.50)), 1),
            'p90_hours': round(float(delays.quantile(0.90)), 1),
            'p99_hours': round(float(delays.quantile(0.99)), 1),
            'recommended_window': f"{int(delays.quantile(0.95))} hours",
        }

How to Evaluate CTR Model Quality?

Key metrics: AUC-ROC (>0.75 good), Log Loss (<0.10), and Calibration Error (<0.005). In production, also look at ΔAUC from new features — an increase >0.001 justifies engineering effort. Calibrated LightGBM yields Log Loss 1.3 times lower than raw probabilities.

Why Probability Calibration Is Critical?

An uncalibrated model can systematically overpay in auctions. Raw LightGBM gives excellent rankings but poor probabilities. We use isotonic regression, which reduces Calibration Error to <0.005. This is critical for bid price calculation: if the model predicts CTR 0.5% but actual is 0.8%, you overpay per click. Accurate calibration directly saves budget — savings from deploying a calibrated LightGBM model can reach up to 30% of ad spending.

Weekly Model Updates

The advertising landscape changes fast: new creatives, seasonality, shifts in audience behavior. We recommend weekly retraining. For this, we set up an automatic retraining pipeline with feature drift monitoring. During high volatility periods (e.g., Black Friday), daily updates may be needed, but this is usually excessive and introduces noise.

Delayed Conversions: Problem and Solution

In CVR models, conversions can occur hours or days after a click. If you simply truncate the training set by time, recent observations will have underestimated CVR. We use correction by discarding clicks with an unexpired conversion window (typically 24-72 hours depending on delay distribution). This improves CVR calibration by 10-20%.

CTR/CVR Model Quality Metrics

For CTR models, AUC 0.76 vs 0.74 is a significant difference at scale. Calibration is mandatory: an uncalibrated model can systematically overpay.

AI CTR/CVR Prediction Implementation Process

We work in stages:

1. Data Audit — collect impression logs, check quality and completeness. Estimate potential ML lift.
2. Feature Engineering — build features: smoothed CTR, site stats, temporal patterns. Use Wilson smoothing to handle sparsity.
3. Training and Calibration — LightGBM + isotonic calibration. Validate on a held-out test set.
4. A/B Testing — compare with baseline on revenue in production.
5. Documentation and Deployment — export model to ONNX or Triton, set up drift monitoring.

What's Included in Model Development

Deliverables:

• Data audit report with potential lift estimation
• Python package with inference pipeline
• API documentation for the bidder
• Training for the client's team
• 3-month support after deployment

Typical Mistakes in CTR/CVR MVP

• Ignoring delayed conversions — CVR underestimated by 10-20%.
• Lack of validation: using a single cross-validation instead of three time-based holdouts.
• Too frequent updates (daily) — noise from random fluctuations.
• Using only linear models — they miss nonlinear interactions.

Order a data audit of your advertising data — we'll determine the potential CTR lift. Get a consultation: we'll evaluate your data and tell you what CTR improvement is possible. Our experience: over 20 projects for programmatic platforms, 5+ years in ML ad optimization. Contact us for a free consultation on your data.