Churn Prediction: ML Model to Prevent Customer Churn

We faced a challenge: a SaaS product with $1M MRR had a 5% monthly churn. Each percentage point reduction means $120K additional ARR annually. But without an accurate model, retention efforts are blind—blanket discounts burn margins. Churn prediction solves this: a model identifies high-risk customers before they churn. We build systems that reduce churn by 20% in practice. Our guaranteed methodology delivers a proven track record with over 5 years of experience and 30+ successful projects.

Problems we solve

Blurred targeting. In non-contractual scenarios (e-commerce, games), there is no explicit churn label—you must define an inactivity threshold. For example, if a customer hasn't made a purchase for 90 days, consider them churned. Choosing the threshold is critical: at 30 days, 20% of customers get a label; at 90 days, only 5%.

Imbalanced classes. 2-10% churners vs. 90% non-churners. Without correction, a model achieves 90% accuracy but zero recall on churners.

Feature engineering. RFM metrics are the foundation, but you also need trends (activity change over 30 days), feature adoption rate, and support tickets. We use rolling window aggregations and diff features.

How we do it: stack and a case

Stack: LightGBM (baseline)—LightGBM is 10x faster than LSTM on tabular data with comparable quality. CatBoost for categorical features, LSTM if event sequences are critical. Feature store—PostgreSQL with pgvector for embeddings. MLflow for experiments, SHAP for interpretation.

Detailed case from our practice: Client—B2B SaaS with 50K users. Baseline LightGBM gave PR-AUC 0.31. After adding trend features (login frequency change over 30 days) — 0.41, +32%. Adding a sequence model (LSTM on event sequences) pushed it to 0.49, but with 4x latency. Production solution: an ensemble of LightGBM + LSTM with cascading scoring. Implementation cost: $15,000. Savings: $30,000–$50,000 per 10,000 customers, ROI of 10x in 6 months.

How to define churn in non-contractual scenarios?

Define an inactivity period after which a customer is considered churned. We choose X based on analysis of inter-purchase interval distribution. Typical values: 60-90 days for B2B SaaS, 90-180 for e-commerce. A wrong choice introduces noise into the target variable.

Why LightGBM is a good baseline for churn prediction?

LightGBM handles missing values, works with categorical features, and captures non-linear dependencies. On standard churn tasks, it beats logistic regression by 0.15–0.25 AUC-ROC and is 2-3x faster than XGBoost.

Development and deployment

Feature Engineering

RFM metrics (most important predictors):

• Recency: days since last action/transaction
• Frequency: number of sessions/purchases in 30/90/180 days
• Monetary: total spend over period

Behavioral features:

• Trend features: activity increase/decrease over last 30 days vs. previous 30
• Feature adoption rate: % of key product features used by the customer
• Support tickets: number, type, NPS after resolution

Contractual/demographic:

• Time since onboarding
• Plan type
• Segment (SMB / Enterprise)
• Acquisition channel

Algorithm selection

Recommendation: start with LightGBM as baseline, add Sequence Model if behavioral patterns are important.

Handling imbalanced classes

Methods to counter imbalance include using class weights (class_weight='balanced' in sklearn)—simplest fix; SMOTE generates synthetic minority examples but can introduce noise; Focal Loss in neural networks downweights easy examples; threshold tuning on Precision-Recall curve (not 0.5)—a free way to improve Precision@K. For evaluation, use weighted F1-score as primary metric, AUC-ROC for ranking, Precision@K for marketing—precision among top-K at-risk customers is most important. See Wikipedia on imbalanced classes.

Deployment and usage

Batch scoring: weekly model run on the entire customer base. Output: a table with churn probability for each customer. Segmentation: high risk (>0.7), medium risk (0.4-0.7), low risk (<0.4).

Real-time scoring: API endpoint POST /score, <100ms response, score updated in CRM in real time.

Retention by segment:

• High risk: personal call from Customer Success or a discount
• Medium risk: automated email campaign with value reminders
• Low risk: no action (don't waste resources)

Measuring business impact

Uplift modeling is the correct way to measure real system value. A standard A/B test: 50% of high-risk customers receive retention (treatment), 50% do not (control). Measure the difference in churn rate. Companies using churn prediction reduce churn by 15-20%. Average savings from implementation: $30–50K per 10K customers, with implementation costs starting at $15,000.

Process and timeline

Process

1. Analytics: collect and clean data, define churn, analyze distribution.
2. Feature engineering: RFM, trends, adoption, contractual data.
3. Modeling: baseline (LightGBM), experiments (CatBoost, LSTM), threshold tuning.
4. Testing: offline (AUC, F1, Precision@K), online A/B uplift test.
5. Deploy: weekly batch scoring, real-time API (<100ms), CRM integration.
6. Monitor: data drift, model drift, automatic retraining.

What's included

• Report on churn definition (target selection)
• Baseline model (LightGBM) + SHAP report
• Feature and pipeline documentation
• Batch scoring integration into your CRM
• Team training (2 hours)
• 3 months post-deployment support

Estimated timeline

First model with basic RFM features: 2-3 weeks. Full system with feature store, drift monitoring, and CRM integration: 8-10 weeks. We are a team with 5+ years of ML production experience and 30+ successful churn prediction projects. Contact us to evaluate your project and get precise timelines. Get a consultation on implementing churn prediction.

import pandas as pd

def rfm_features(transactions, as_of_date):
    """Compute Recency, Frequency, Monetary for each customer."""
    rfm = transactions.groupby('customer_id').agg(
        recency=('transaction_date', lambda x: (as_of_date - x.max()).days),
        frequency=('transaction_id', 'nunique'),
        monetary=('amount', 'sum')
    ).reset_index()
    return rfm

When does a time series forecasting model fail in production?

The CFO requests a quarterly sales forecast. An analyst builds SARIMA on three years of data, achieves MAPE 8.3% on the test set, and deploys. Two months later, the metric in production jumps to 23%. The root cause: the model was trained on pre‑COVID data, tested on a stable period, but production hit a promotion and supply chain disruption. Data leakage plus distribution shift—perfect notebook numbers, a broken forecast in reality. We have seen this pattern dozens of times across retail, fintech, and IoT. Our team has delivered more than 50 forecasting projects over 5+ years.

Incorrect cross-validation. Standard train_test_split for time series creates data leakage: the model sees future values during training. The correct approach is TimeSeriesSplit or walk‑forward validation with an expanding window.

Multiple seasonality. Hourly electricity consumption has three seasonalities: daily (24h), weekly (168h), yearly (8760h). SARIMA handles only one. Prophet can handle multiple but scales poorly to thousands of series.

Missing values and anomalies. A missing sensor reading is information (the sensor turned off), not NaN. Linear interpolation destroys this signal. Proper handling depends on the missingness mechanism.

Cold start. A new SKU in a 50,000‑item assortment has no history, yet a forecast is needed. Standard approaches fail; cross‑learning or feature‑based methods are required.

Why is model selection critical for your data?

Prophet (Meta) – a solid start for business data with clear seasonality and holidays. Fast setup, interpretable, built‑in outlier detection. Fails on irregular patterns and does not scale beyond ~10k series without parallelization.

Gradient boosting on features (LightGBM, XGBoost) – often underestimated. Engineer lags (t‑1, t‑7, t‑28), rolling means, day‑of‑week, holidays. The model trains on all series simultaneously, solving cold start via transfer learning. MAPE in retail often beats neural nets with proper feature engineering.

TFT (Temporal Fusion Transformer) – a transformer designed for interpretable forecasting with covariates. Built‑in variable selection, temporal attention, quantile outputs. Available in pytorch‑forecasting. Requires ~10,000+ records per series for stable training.

PatchTST – splits the series into patches (like ViT for images), capturing local patterns better than classic transformers. Excellent for long‑horizon forecasting (96–720 steps ahead).

N‑HiTS, N‑BEATS – attention‑free neural architectures, faster than TFT, competitive accuracy. N‑BEATS won the M4/M5 benchmarks for tasks without covariates.

How do we deploy TFT in production?

A typical pipeline via pytorch‑forecasting:

training = TimeSeriesDataSet(
    data,
    time_idx="time_idx",
    target="sales",
    group_ids=["store", "sku"],
    min_encoder_length=max_encoder_length // 2,
    max_encoder_length=max_encoder_length,  # 120 days
    min_prediction_length=1,
    max_prediction_length=max_prediction_length,  # 28 days
    static_categoricals=["store_type", "category"],
    time_varying_known_reals=["price", "promo_flag"],
    time_varying_unknown_reals=["sales"],
    target_normalizer=GroupNormalizer(groups=["store", "sku"], transformation="softplus"),
)

A common mistake: the default target_normalizer (StandardScaler) breaks predictions for series with zero values (no sales on weekends). GroupNormalizer with transformation="softplus" is the correct choice for count data.

Case study: retail demand forecasting

A chain of 120 stores, 8,000 SKUs, 28‑day forecast horizon. The original system: SARIMA per series, MAPE 18.4%, retraining cycle – 6 hours. We replaced it with TFT on PyTorch + pytorch‑forecasting: a single model for all series, MAPE 11.2%, retraining – 40 minutes on an A10G. Feature importance via variable selection revealed that day_before_holiday influences more than the holiday date itself. Annual savings on inference alone exceeded $50,000.

Step‑by‑step configuration

1. Data collection and preparation. Handle missing values (mark NaN, interpolate only for technical failures), aggregate to required frequency, engineer covariates (holidays, promotions, prices).
2. Create TimeSeriesDataSet. Set group_ids (store + SKU), time index, forecast horizon. Choose target_normalizer based on target distribution.
3. Train a baseline. Prophet or LightGBM first – to understand complexity.
4. Train TFT. Use TemporalFusionTransformer with loss=QuantileLoss(), tune learning rate and hidden layer sizes.
5. Validate and interpret. Walk‑forward test, analyze variable selection, build attention heatmaps.

How to properly evaluate forecast quality?

RMSE alone is misleading – it over‑penalizes large values. Our standard set:

• MAPE – interpretable, unstable near zero.
• sMAPE – symmetric, avoids division by small numbers.
• MASE (Mean Absolute Scaled Error) – normalized relative to a naive seasonal forecast, ideal for comparing series of different scales.
• Pinball loss – for probabilistic forecasting, inventory management.

We guarantee a model card with these metrics on the validation set and walk‑forward results on at least 6 months of history.

What deliverables do you receive?

• Documentation of chosen architecture and hyperparameter rationale.
• Reproducible training and inference pipeline (Docker + CI/CD + Airflow/Prefect).
• Committed code with unit tests for key components.
• Team training: retraining, output interpretation, deployment of new versions.
• 3 months of post‑delivery support (consultations, bug fixes, fine‑tuning).

The model is deployed via FastAPI or Triton Inference Server. Retraining is scheduled (e.g., weekly) via Airflow with drift validation and automatic rollback if metrics deteriorate.

Process and timeline

We start with EDA: visualization, ADF test, STL decomposition, analysis of missing values and outliers. This takes 2–3 days but often reveals systemic data issues that block forecasting. Then we build a baseline (naive seasonal, Prophet), engineer features for LightGBM, and select a neural architecture if needed. Walk‑forward validation with a realistic horizon. Deployment via API with automatic retraining scheduled via Airflow or Prefect.

Timeline: MVP forecast on one data type – 3–6 weeks. Hierarchical forecasting system with automation – 2–5 months. Cost is calculated individually based on data volume, number of series, and required accuracy.

Our team consists of certified ML engineers (AWS ML Specialty, GCP Professional ML Engineer) with 5+ years on the market and over 50 completed forecasting projects. Contact us for a free analysis of your data – we will assess the task and provide initial recommendations within 1–2 days. Request a consultation to ensure your forecasts work in production, not just in a notebook.