AI-Driven Anomaly Detection for Industrial Equipment

Imagine: a critical pump stops at a factory. One hour of downtime costs 500,000 rubles. We build predictive maintenance systems that predict failures 2-3 weeks before breakdown. The architecture revolves around a core priority: minimize missed failures while maintaining an acceptable false positive rate. The foundation is a combination of five methods—from instant thresholds to physics models.

Problems We Solve

Missed failures vs. false alarms. A single detector yields a high FPR—up to 30% on noisy data. Multi-level consensus reduces FPR to 5% while maintaining 95% sensitivity. The system is designed for industrial IoT and uses AI diagnostics at every level.

Noisy data and correlated incidents. One pump failure causes anomalies in pressure, temperature, and flow. Our system groups them into a single incident rather than generating multiple alerts.

How We Build the Anomaly Detection System

Multi-level architecture combines five approaches:

L1/L2 provide instant alerts, L3/L4 early diagnostics, L5 long-term trend. Consensus reduces the false positive rate by 40% compared to a single detector.

Feature Engineering for Equipment

We transform raw sensor signals (vibration, current, pressure) into multi-domain features:

import numpy as np
from scipy import stats, signal

def extract_equipment_features(raw_signal, sampling_rate=1000):
    # Time domain
    features = {
        'rms': np.sqrt(np.mean(raw_signal**2)),
        'peak': np.max(np.abs(raw_signal)),
        'crest_factor': np.max(np.abs(raw_signal)) / np.sqrt(np.mean(raw_signal**2)),
        'kurtosis': stats.kurtosis(raw_signal),
        'skewness': stats.skew(raw_signal),
        'peak_to_peak': np.ptp(raw_signal),
        'shape_factor': np.sqrt(np.mean(raw_signal**2)) / np.mean(np.abs(raw_signal))
    }
    # Frequency domain
    freqs, psd = signal.welch(raw_signal, fs=sampling_rate, nperseg=512)
    total_power = np.trapz(psd, freqs)
    bands = [(0, 100), (100, 500), (500, 2000), (2000, 5000)]
    for low, high in bands:
        mask = (freqs >= low) & (freqs < high)
        band_power = np.trapz(psd[mask], freqs[mask])
        features[f'band_power_{low}_{high}'] = band_power / total_power
    features['dominant_freq'] = freqs[np.argmax(psd)]
    features['spectral_centroid'] = np.sum(freqs * psd) / np.sum(psd)
    return features

What matters is not the absolute value but the deviation from the asset's normal baseline. Hence we compute delta features:

def compute_delta_features(current_features, baseline_features, trend_features_7d):
    deltas = {}
    for key in current_features:
        if key in baseline_features:
            deltas[f'{key}_delta_abs'] = current_features[key] - baseline_features[key]
            if baseline_features[key] != 0:
                deltas[f'{key}_delta_pct'] = ((current_features[key] - baseline_features[key]) / abs(baseline_features[key]) * 100)
        if key in trend_features_7d:
            deltas[f'{key}_trend_7d'] = trend_features_7d[key]
    return deltas

Unsupervised Detection

Isolation Forest with seasonality adaptation is trained on 30+ days of normal operation. Autoencoder (LSTM) captures reconstruction error for multi-sensor windows:

from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
import pandas as pd
import numpy as np

class EquipmentAnomalyDetector:
    def __init__(self, contamination=0.02):
        self.scaler = StandardScaler()
        self.model = IsolationForest(contamination=contamination, n_estimators=200, random_state=42)
        self.baseline_built = False

    def fit_baseline(self, normal_operation_features: pd.DataFrame):
        X = self.scaler.fit_transform(normal_operation_features)
        self.model.fit(X)
        self.baseline_built = True
        scores = self.model.score_samples(X)
        self.threshold = np.percentile(scores, 5)

    def detect(self, current_features: dict) -> dict:
        if not self.baseline_built:
            return {'status': 'no_baseline', 'anomaly': False}
        X = self.scaler.transform([list(current_features.values())])
        score = self.model.score_samples(X)[0]
        is_anomaly = score < self.threshold
        return {'anomaly_score': float(-score), 'anomaly': bool(is_anomaly), 'severity': self._classify_severity(-score)}

    def _classify_severity(self, anomaly_score):
        if anomaly_score > 0.8: return 'critical'
        if anomaly_score > 0.6: return 'high'
        if anomaly_score > 0.4: return 'medium'
        return 'low'

import torch
import torch.nn as nn

class SensorAutoencoder(nn.Module):
    def __init__(self, input_dim, hidden_dim=32, latent_dim=8, seq_len=60):
        super().__init__()
        self.encoder = nn.LSTM(input_dim, hidden_dim, batch_first=True)
        self.bottleneck = nn.Linear(hidden_dim, latent_dim)
        self.decoder = nn.LSTM(latent_dim, hidden_dim, batch_first=True)
        self.output_layer = nn.Linear(hidden_dim, input_dim)

    def forward(self, x):
        enc_out, _ = self.encoder(x)
        z = self.bottleneck(enc_out[:, -1, :])
        z_expanded = z.unsqueeze(1).repeat(1, x.shape[1], 1)
        dec_out, _ = self.decoder(z_expanded)
        reconstruction = self.output_layer(dec_out)
        return reconstruction

Alert Consensus

Different detectors catch different anomaly types. For standard equipment, agreement of 2 out of 3 detectors is required; for critical equipment, a single detector suffices. Alerts correlated in time and topology are grouped into one incident.

Why Multi-Level Architecture Is More Effective Than a Single Method?

Each method has blind spots. Isolation Forest poorly catches slow trend drift, while statistical methods miss multivariate patterns. The combination provides full coverage. In our projects, consensus reduced false alarms by 60%.

Model Drift Monitoring

Over time, equipment wear changes baseline features. Weekly KS-test checks for drift:

from scipy.stats import ks_2samp

def detect_model_drift(recent_features, baseline_features, p_threshold=0.01):
    drift_features = []
    for col in recent_features.columns:
        stat, p_value = ks_2samp(baseline_features[col], recent_features[col])
        if p_value < p_threshold:
            drift_features.append(col)
    drift_ratio = len(drift_features) / len(recent_features.columns)
    return {'drift_detected': drift_ratio > 0.3, 'drifted_features': drift_features, 'drift_ratio': drift_ratio}

When drift is detected, retraining is triggered on the last 60 days of data (if they are labeled as normal).

What's Included

1. Equipment audit and data collection — on-site engineer visit, sensor and historical log analysis.
2. Multi-level model development (L1-L5) — threshold setting, ML model training, physics model creation.
3. Integration into infrastructure — SCADA connection, data stream setup, dashboard and alerts.
4. Documentation and staff training — operation instructions, response procedures.
5. Support and retraining — drift monitoring, baseline updates, adaptation for new failures.

Process and Timelines

Total time: from 3 weeks to 3 months depending on complexity.

Typical Implementation Mistakes

• Training on dirty data (maintenance labels not removed)
• Ignoring seasonality (heating, night shifts)
• Lack of baseline for normal mode (need at least 30 days)
• Retraining models without drift check

Example from practice: at a fertilizer plant, the system detected an anomaly in a compressor bearing 18 days before failure. Repair costs were negligible compared to emergency shutdown expenses (about 500 thousand rubles per hour of downtime).

Want to assess potential for your equipment? Order a free audit—our engineers will analyze readiness for implementation. Get a consultation on architecture selection and data volume. Contact us to discuss your project.

Anomaly Detection: Autoencoders, Isolation Forest, PyOD

Server monitoring shows CPU 85%, memory 91% — is this the start of an attack or normal peak load? A classifier won’t help: anomalies are by definition rare, diverse, and not pre-labeled. Supervised learning requires examples of anomalies in the training set — so it fails on what you haven’t seen yet. Without an unsupervised approach, detection turns into guesswork.

Why Does Anomaly Detection Require an Unsupervised Approach?

The main problem: no labels and extreme class imbalance. Fraud transactions account for 0.01–0.1% of total volume, production defects 0.5–3%. With such ratios, a naive “all normal” classifier gives 99.9% accuracy but recall for the anomalous class near zero. Supervised models are powerless.

Second: “normality” is always contextual. A login at 3 AM may be normal for a night‑shift user but suspicious for a day‑worker. Bearing vibration at 2.3 mm/s depends on operating mode and machine age. So we embed context via feature engineering and time windows.

Third: quality assessment without ground truth. No standard test set — AUC‑ROC is possible only if a few labeled examples exist. For fully unlabeled data, only domain expert validation and indirect metrics work.

How to Distinguish an Anomaly from Noise in Real Time?

With adaptive thresholds and continuous monitoring of model statistics. In the case section we show how.

Isolation Forest is the standard baseline for tabular data. Idea: anomalies are isolated faster by random partitioning of the feature space. Works well at contamination=0.01–0.1, robust to feature scale, no normalization required. Implementation in sklearn.ensemble.IsolationForest.

Typical mistake: setting contamination='auto' without understanding the data. Auto mode assumes a threshold of -0.5, which may not match the actual anomaly proportion. Better: estimate expected anomaly percentage through domain knowledge and set it explicitly. We guarantee contamination tuning for your case.

PyOD (Python Outlier Detection) is a library with 40+ algorithms under a unified API — OCSVM, LOF, COPOD, ECOD, DeepSVDD, AutoEncoder. Useful for quickly comparing methods on the same data.

Autoencoders are the main method for unstructured data (time series, images, logs). Train the network to reconstruct normal data; anomalies yield high reconstruction error. Anomaly threshold is the 95th or 99th percentile of error on a validation set of normal data.

Practical problem with autoencoders: overfitting on “normal” patterns that are still rare. If the training set contains even a few anomalies, the model may learn to reconstruct them well. Solution: thorough cleaning of training data or using a Variational Autoencoder (VAE), which generalizes better.

LSTM‑AE for time series captures temporal dependencies better than a regular AE. Especially effective for multivariate time series (10+ sensors simultaneously). Implementation via PyTorch, training with MSELoss on sliding windows.

In Detail: Anomaly Detection in Industrial Time Series

Problem: vibration sensors on 12 pumps at a chemical plant, 6 sensors per pump, frequency 100 Hz. Need to warn of impending failure 4–24 hours in advance.

Solution architecture: raw data → feature extraction (RMS, kurtosis, peak factor, FFT amplitudes at resonant frequencies) → normalization by 24‑hour sliding window → LSTM‑AE → reconstruction error → threshold logic + alerting.

LSTM window size: 60 seconds (6000 points at 100 Hz). Too small a window misses slow patterns, too large loses sensitivity to rapid changes.

Anomaly threshold: not fixed, but adaptive. threshold = mean(errors_last_7d) + 3 * std(errors_last_7d). As normal state drifts (planned wear), the threshold adapts, avoiding false positives.

Result over a 6‑month pilot: detected 4 out of 5 real pre‑failure conditions (recall 0.8), with 2 false alarms over 6 months (precision 0.67). Before implementation: 3 unplanned shutdowns. After implementation: cost savings verified in the pilot report.

What Specifics Does Fraud Detection Face?

Financial transactions have several features that complicate detection:

• Concept drift: fraud patterns change faster than normal behavior. A model trained six months ago becomes obsolete.
• Adversarial adaptation: advanced fraudsters adapt — making transactions resemble normal ones.
• Temporal dependency: a series of normal transactions followed by one unusual transfer is a sequence anomaly, not a single point.

Practical stack for fraud detection: LightGBM with SMOTE oversampling for the supervised part (known fraud cases) + Isolation Forest for unsupervised (new patterns). Both signals combined in an ensemble; final decision via thresholds tuned for acceptable FPR (0.1–1% of transactions sent to manual review).

How to Evaluate Quality Without Labels?

When ground truth is absent, we evaluate using:

• Synthetic anomaly injection: add artificial anomalies (spike, level shift, point outlier) and check if the model detects them.
• Expert validation: random sample of top‑K anomalies → expert review → precision.
• Business metric: did the number of missed incidents / false alarms decrease after deployment?

Technical detail: the adaptive threshold is computed as mean(errors) + k * std(errors) on a 7‑day sliding window. Coefficient k is tuned on a validation set with synthetic anomalies to achieve FPR < 0.1%. When features drift, the window automatically shifts.

Process

1. Interview with domain experts — understand what “normality” means and what incidents have occurred.
2. EDA and data preparation — cleaning, feature creation, time windows.
3. Baseline (Isolation Forest) — fast validation on known incidents.
4. Model selection and customization — Autoencoder / LSTM‑AE / ensemble.
5. Training, validation with synthetic anomalies.
6. Deployment to production — pipeline on Kafka + Flink / Airflow, alerting to Telegram/Slack, drift monitoring.
7. Post‑deployment support — monitor model metrics, update thresholds.

What's Included

• Audit of current data and processes
• Development and training of models (Isolation Forest / Autoencoder / LSTM‑AE / ensemble)
• Configuration of adaptive thresholds and alerting
• Anomaly monitoring dashboard (Grafana / Streamlit)
• Model card and pipeline documentation
• Training for your team (2–3 sessions)
• 3‑month warranty support

Timeline: baseline system with one method — 2–4 weeks. Production system with adaptive thresholds, alerting, and monitoring — 2–5 months. Pricing is calculated individually for your case.

Our team has 8+ years of experience in industrial analytics and 15+ successful projects in anomaly detection for telemetry, finance, and IT monitoring. Get a consultation — we’ll tell you how to solve your problem. Contact us to discuss your data and receive a preliminary architecture proposal.