Predicting Subscriber Churn with >85% Accuracy: Implementation Experience

An operator spends millions on acquiring subscribers but loses up to 30% of the base within the first six months. The cost of acquisition is 5–7 times higher than retention — each lost unit is a direct loss. According to industry reports, operators with AI systems reduce churn by an average of 20%. On a project for an operator with 5 million subscribers, we cut churn by 18% in one quarter, retaining over 30,000 subscribers. We build AI systems that predict churn with >85% accuracy and suggest what to offer to whom. Request an analytical report on your data — we’ll show which segments are at risk.

Which Signs Influence Churn?

Key feature groups: service usage (voice, data, SMS), finances (ARPU, delays), support interactions, and competitive context. Trends are especially valuable — e.g., decreasing data consumption over the last 30 days compared to the previous period. Feature engineering from BSS/OSS systems includes:

features_usage = {
    'voice_outgoing_min_30d': sum(voice_outgoing_last_30d),
    'voice_incoming_min_30d': sum(voice_incoming_last_30d),
    'unique_called_numbers': len(unique_called_30d),
    'data_usage_gb_30d': sum(data_usage_last_30d),
    'data_usage_trend': data_30d - data_60_30d,
    'arpu': avg_monthly_revenue,
    'arpu_trend': arpu_30d - arpu_90d,
    'payment_delays': count(payment_delay > 5d),
    'last_payment_days_ago': days_since_last_payment
}

Interaction with the operator:

features_interaction = {
    'cs_contacts_30d': count(support_contacts_last_30d),
    'complaints_90d': count(formal_complaints_90d),
    'nps_score': last_nps_response,
    'app_logins_30d': mobile_app_logins_count
}

Competitive context: numbers ported to a competitor (MNP at segment level), price difference with analog tariff — the larger the gap, the higher the churn risk.

Why Uplift Modeling Outperforms Churn Classification

The naive approach is to give a discount to everyone who might leave. Problem: some subscribers will stay anyway (sure things), some will leave even with a discount (lost causes). The discount is only needed for persuadables — those whom the offer can tilt to stay. An uplift model (see Uplift modelling) estimates the causal effect:

Uplift = P(retained | treated) - P(retained | not treated)

Target those with uplift > 0. A meta-learner (T-Learner) builds two classifiers — for treatment and control:

model_treatment = LightGBMClassifier().fit(X_treated, y_treated)
model_control = LightGBMClassifier().fit(X_control, y_control)
uplift = model_treatment.predict_proba(X)[:, 1] - model_control.predict_proba(X)[:, 1]

This yields 20% more retained subscribers for the same budget than simply ranking by churn probability. Besides T-Learner, we use S-Learner (one classifier with a treatment feature) and X-Learner (cross-learning). In practice, T-Learner gives the best uplift with sufficient sample size, while X-Learner is more robust to imbalance. The choice depends on the volume of retention campaigns and the treatment share.

How to Personalize Retention Offers?

Predicting who will leave is not enough — you need to decide what to offer. Next Best Offer (NBO) — a multiclass model that selects the offer: discount, bonus traffic, tariff upgrade, or free roaming. It accounts for CLV, ARPU, consumption type, and offer history. Timing is crucial: 30–60 days before contract end, after a negative NPS (within 48 hours), after a complaint to CS (immediately).

Telecom Churn Specifics

Types of churn: voluntary churn (conscious departure), involuntary churn (disconnection for non-payment), early churn (first 90 days). For prepaid, churn is defined by inactivity: 30/60/90 days without top-up.

Contractual vs. prepaid: Postpaid has a clear termination date; the model predicts churn at renewal. Prepaid has no contract; churn is defined by inactivity.

Multi-Horizon Models

Steps to Build a Churn Model

1. Collect and aggregate data — export from BSS/OSS, CRM, CDR for 6–12 months.
2. Feature engineering — compute usage, financial, interaction, and trend features.
3. Train a baseline — LightGBM / XGBoost with time-based cross-validation.
4. Build an uplift model — T-Learner or S-Learner with CATE estimation.
5. Calibrate NBO — multiclass model considering CLV and budget.
6. A/B testing — compare campaigns with and without the model.

Model Evaluation Metrics

What the Work Includes

• Detailed analytical report on churn factors and segmentation
• ML model (churn + uplift + NBO) with API for integration
• Integration with CRM and campaign dispatch
• Documentation and team training for the operator
• Support for 3 months after launch

Timeline and Cost

Basic churn model on BSS data – 4–5 weeks. Full system with uplift, NBO, and CRM integration – 3–4 months. Cost is calculated individually, based on data volume and complexity. Payback comes from a 15–30% churn reduction: for an operator with a million-subscriber base, savings can be significant.

Why Work with Us

We are a team of senior ML engineers with 7+ years of telecom experience. Completed 15+ churn prediction projects for operators in Russia and the CIS. We guarantee quality: our models undergo A/B testing and deliver measurable business results. Certified in AWS SageMaker and PyTorch. Get a free pilot on your data within 2 weeks — contact us for a consultation.

Common Mistakes to Avoid

• Ignoring trend features: many rely only on point-in-time metrics like current ARPU, missing the early signs in decreasing usage.
• Using only a classification model without uplift: leads to wasting budget on sure things and lost causes.
• Not calibrating NBO to budget constraints: offering too much to high-CLV users without considering total campaign cost.
• Poor timing: sending offers too late or too early reduces effectiveness.

Request an analytical report on your data — we’ll show which segments are at risk. Contact us to discuss your specific case.

When does a time series forecasting model fail in production?

The CFO requests a quarterly sales forecast. An analyst builds SARIMA on three years of data, achieves MAPE 8.3% on the test set, and deploys. Two months later, the metric in production jumps to 23%. The root cause: the model was trained on pre‑COVID data, tested on a stable period, but production hit a promotion and supply chain disruption. Data leakage plus distribution shift—perfect notebook numbers, a broken forecast in reality. We have seen this pattern dozens of times across retail, fintech, and IoT. Our team has delivered more than 50 forecasting projects over 5+ years.

Incorrect cross-validation. Standard train_test_split for time series creates data leakage: the model sees future values during training. The correct approach is TimeSeriesSplit or walk‑forward validation with an expanding window.

Multiple seasonality. Hourly electricity consumption has three seasonalities: daily (24h), weekly (168h), yearly (8760h). SARIMA handles only one. Prophet can handle multiple but scales poorly to thousands of series.

Missing values and anomalies. A missing sensor reading is information (the sensor turned off), not NaN. Linear interpolation destroys this signal. Proper handling depends on the missingness mechanism.

Cold start. A new SKU in a 50,000‑item assortment has no history, yet a forecast is needed. Standard approaches fail; cross‑learning or feature‑based methods are required.

Why is model selection critical for your data?

Prophet (Meta) – a solid start for business data with clear seasonality and holidays. Fast setup, interpretable, built‑in outlier detection. Fails on irregular patterns and does not scale beyond ~10k series without parallelization.

Gradient boosting on features (LightGBM, XGBoost) – often underestimated. Engineer lags (t‑1, t‑7, t‑28), rolling means, day‑of‑week, holidays. The model trains on all series simultaneously, solving cold start via transfer learning. MAPE in retail often beats neural nets with proper feature engineering.

TFT (Temporal Fusion Transformer) – a transformer designed for interpretable forecasting with covariates. Built‑in variable selection, temporal attention, quantile outputs. Available in pytorch‑forecasting. Requires ~10,000+ records per series for stable training.

PatchTST – splits the series into patches (like ViT for images), capturing local patterns better than classic transformers. Excellent for long‑horizon forecasting (96–720 steps ahead).

N‑HiTS, N‑BEATS – attention‑free neural architectures, faster than TFT, competitive accuracy. N‑BEATS won the M4/M5 benchmarks for tasks without covariates.

How do we deploy TFT in production?

A typical pipeline via pytorch‑forecasting:

training = TimeSeriesDataSet(
    data,
    time_idx="time_idx",
    target="sales",
    group_ids=["store", "sku"],
    min_encoder_length=max_encoder_length // 2,
    max_encoder_length=max_encoder_length,  # 120 days
    min_prediction_length=1,
    max_prediction_length=max_prediction_length,  # 28 days
    static_categoricals=["store_type", "category"],
    time_varying_known_reals=["price", "promo_flag"],
    time_varying_unknown_reals=["sales"],
    target_normalizer=GroupNormalizer(groups=["store", "sku"], transformation="softplus"),
)

A common mistake: the default target_normalizer (StandardScaler) breaks predictions for series with zero values (no sales on weekends). GroupNormalizer with transformation="softplus" is the correct choice for count data.

Case study: retail demand forecasting

A chain of 120 stores, 8,000 SKUs, 28‑day forecast horizon. The original system: SARIMA per series, MAPE 18.4%, retraining cycle – 6 hours. We replaced it with TFT on PyTorch + pytorch‑forecasting: a single model for all series, MAPE 11.2%, retraining – 40 minutes on an A10G. Feature importance via variable selection revealed that day_before_holiday influences more than the holiday date itself. Annual savings on inference alone exceeded $50,000.

Step‑by‑step configuration

1. Data collection and preparation. Handle missing values (mark NaN, interpolate only for technical failures), aggregate to required frequency, engineer covariates (holidays, promotions, prices).
2. Create TimeSeriesDataSet. Set group_ids (store + SKU), time index, forecast horizon. Choose target_normalizer based on target distribution.
3. Train a baseline. Prophet or LightGBM first – to understand complexity.
4. Train TFT. Use TemporalFusionTransformer with loss=QuantileLoss(), tune learning rate and hidden layer sizes.
5. Validate and interpret. Walk‑forward test, analyze variable selection, build attention heatmaps.

How to properly evaluate forecast quality?

RMSE alone is misleading – it over‑penalizes large values. Our standard set:

• MAPE – interpretable, unstable near zero.
• sMAPE – symmetric, avoids division by small numbers.
• MASE (Mean Absolute Scaled Error) – normalized relative to a naive seasonal forecast, ideal for comparing series of different scales.
• Pinball loss – for probabilistic forecasting, inventory management.

We guarantee a model card with these metrics on the validation set and walk‑forward results on at least 6 months of history.

What deliverables do you receive?

• Documentation of chosen architecture and hyperparameter rationale.
• Reproducible training and inference pipeline (Docker + CI/CD + Airflow/Prefect).
• Committed code with unit tests for key components.
• Team training: retraining, output interpretation, deployment of new versions.
• 3 months of post‑delivery support (consultations, bug fixes, fine‑tuning).

The model is deployed via FastAPI or Triton Inference Server. Retraining is scheduled (e.g., weekly) via Airflow with drift validation and automatic rollback if metrics deteriorate.

Process and timeline

We start with EDA: visualization, ADF test, STL decomposition, analysis of missing values and outliers. This takes 2–3 days but often reveals systemic data issues that block forecasting. Then we build a baseline (naive seasonal, Prophet), engineer features for LightGBM, and select a neural architecture if needed. Walk‑forward validation with a realistic horizon. Deployment via API with automatic retraining scheduled via Airflow or Prefect.

Timeline: MVP forecast on one data type – 3–6 weeks. Hierarchical forecasting system with automation – 2–5 months. Cost is calculated individually based on data volume, number of series, and required accuracy.

Our team consists of certified ML engineers (AWS ML Specialty, GCP Professional ML Engineer) with 5+ years on the market and over 50 completed forecasting projects. Contact us for a free analysis of your data – we will assess the task and provide initial recommendations within 1–2 days. Request a consultation to ensure your forecasts work in production, not just in a notebook.