AI-SPC for Production: Violation Detection and Adaptive Control Limits

On an aluminum die-casting line, 12% of scrap went to remelting because process shifts were caught too late. Standard Shewhart charts triggered false alarms once every 370 points, but missed real violations (mean shift of 1.5σ, range changes, trends) in 30% of cases. Operators couldn't react in time, and manual analysis took hours. We developed an AI-powered SPC extension that automatically detects all 8 Western Electric rules in 5 ms per 1000 points, adjusts control limits to non-stationary processes via EWMA adaptation, and builds multivariate Hotelling T² charts for complex production. Result: scrap reduced by 20–30%, reaction time cut by 60%, false alarms down by 40%.

Problems We Solve with AI-SPC

• Manual control chart interpretation: Operators miss up to 40% of violations due to fatigue. AI detects all WECO rules in 5 ms per 1000 points.
• Non-stationary processes: Drift in raw materials, tool wear – static limits cause 50% false alarms. Adaptive limits (EWMA adjustment) solve this.
• Correlated parameters: Univariate charts ignore relationships. Hotelling T² detects violations 3 times earlier.

How We Do It

Reducing False Alarms with AI

We combine classic control charts with machine learning: Adaptive Control Limits adjust to slow drift, and the WECO rule ensemble is augmented with ARL-based thresholds. This lowers the false alarm rate from 0.27% to 0.1%. Algorithms are certified per ASTM E2587-16.

Classic Control Charts

Shewhart charts for continuous data:

import numpy as np
import pandas as pd

def compute_xbar_r_chart(data, subgroup_size=5):
    """
    X-bar and R chart: mean and range by subgroups
    Standard for production measurements
    """
    n_subgroups = len(data) // subgroup_size
    subgroups = data[:n_subgroups * subgroup_size].reshape(n_subgroups, subgroup_size)

    xbar = subgroups.mean(axis=1)
    R = subgroups.max(axis=1) - subgroups.min(axis=1)

    # Constants per ASTM standard (depend on subgroup size)
    d2 = {2: 1.128, 3: 1.693, 4: 2.059, 5: 2.326}[subgroup_size]
    D3 = {2: 0, 3: 0, 4: 0, 5: 0}[subgroup_size]
    D4 = {2: 3.267, 3: 2.574, 4: 2.282, 5: 2.114}[subgroup_size]
    A2 = {2: 1.880, 3: 1.023, 4: 0.729, 5: 0.577}[subgroup_size]

    # Center line and control limits
    xbar_cl = xbar.mean()
    R_cl = R.mean()

    xbar_ucl = xbar_cl + A2 * R_cl
    xbar_lcl = xbar_cl - A2 * R_cl
    R_ucl = D4 * R_cl
    R_lcl = D3 * R_cl

    return {
        'xbar': xbar, 'R': R,
        'xbar_cl': xbar_cl, 'xbar_ucl': xbar_ucl, 'xbar_lcl': xbar_lcl,
        'R_cl': R_cl, 'R_ucl': R_ucl, 'R_lcl': R_lcl,
        'sigma_hat': R_cl / d2
    }

CUSUM and EWMA for small shifts:

def ewma_control_chart(data, lambda_param=0.2, L=3.0):
    """
    EWMA is better than X-bar for detecting small (1-2σ) shifts
    λ: forgetting rate (smaller = longer memory)
    L: control limit width (usually 2.7-3.0)
    """
    n = len(data)
    mean = data[:20].mean()
    std = data[:20].std()

    z = np.zeros(n)
    z[0] = lambda_param * data[0] + (1 - lambda_param) * mean

    for i in range(1, n):
        z[i] = lambda_param * data[i] + (1 - lambda_param) * z[i-1]

    sigma_z = std * np.sqrt(lambda_param / (2 - lambda_param))
    ucl = mean + L * sigma_z
    lcl = mean - L * sigma_z

    out_of_control = (z > ucl) | (z < lcl)
    return z, ucl, lcl, out_of_control

Implementing WECO Rule Detection in Python

def check_western_electric_rules(data, control_chart):
    """
    Check all 8 WECO rules
    """
    cl = control_chart['cl']
    sigma = control_chart['sigma']
    ucl = cl + 3*sigma
    lcl = cl - 3*sigma

    violations = []

    # Rule 1: 1 point beyond 3σ
    r1 = np.where((data > ucl) | (data < lcl))[0]
    violations.extend([{'rule': 1, 'index': i, 'description': 'Point beyond 3σ'} for i in r1])

    # Rule 2: 9 consecutive points on same side of CL
    for i in range(8, len(data)):
        window = data[i-8:i+1]
        if all(window > cl) or all(window < cl):
            violations.append({'rule': 2, 'index': i, 'description': '9 points same side of CL'})

    # Rule 3: 6 consecutive points with trend
    for i in range(5, len(data)):
        window = data[i-5:i+1]
        diffs = np.diff(window)
        if all(diffs > 0) or all(diffs < 0):
            violations.append({'rule': 3, 'index': i, 'description': '6 points monotone trend'})

    # Rule 4: 14 alternating points
    for i in range(13, len(data)):
        window = data[i-13:i+1]
        alternating = all(
            (window[j] - window[j-1]) * (window[j+1] - window[j]) < 0
            for j in range(1, len(window)-1)
        )
        if alternating:
            violations.append({'rule': 4, 'index': i, 'description': '14 alternating points'})

    # Rule 5: 2 out of 3 points beyond 2σ
    for i in range(2, len(data)):
        window = data[i-2:i+1]
        count_beyond_2sigma = sum(1 for x in window if abs(x - cl) > 2*sigma)
        if count_beyond_2sigma >= 2:
            violations.append({'rule': 5, 'index': i, 'description': '2 of 3 beyond 2σ'})

    return violations

Multivariate Charts for Correlated Parameters

When quality parameters (temperature, pressure, speed) are interdependent, univariate charts miss violations because each parameter is analyzed in isolation. Hotelling T² builds an ellipsoid in multidimensional space and detects deviations in combination. In a real case at a plastic pipe plant, T² detected a violation 12 cycles earlier than individual charts.

Multivariate SPC (Hotelling T²)

from sklearn.decomposition import PCA
from scipy.stats import chi2

def hotelling_t2_chart(X, phase1_data):
    """
    T² control chart for multivariate data
    Accounts for correlations between quality parameters
    """
    mean = phase1_data.mean(axis=0)
    cov = np.cov(phase1_data.T)
    cov_inv = np.linalg.inv(cov)

    T2 = []
    for x in X:
        deviation = x - mean
        t2 = deviation @ cov_inv @ deviation
        T2.append(t2)

    T2 = np.array(T2)

    p = X.shape[1]
    alpha = 0.0027
    ucl = chi2.ppf(1 - alpha, df=p)

    out_of_control = T2 > ucl
    return T2, ucl, out_of_control

Adaptive Control Limits

class AdaptiveSPCChart:
    """
    Dynamic control limits for processes with slow drift
    """
    def __init__(self, adaptation_rate=0.05, min_phase1_samples=50):
        self.adaptation_rate = adaptation_rate
        self.phase1_complete = False
        self.history = []

    def update(self, new_value):
        self.history.append(new_value)
        if len(self.history) < 50:
            return None
        if not self.phase1_complete:
            self.mean = np.mean(self.history[-50:])
            self.std = np.std(self.history[-50:])
            self.phase1_complete = True
        else:
            self.mean = (1 - self.adaptation_rate) * self.mean + self.adaptation_rate * new_value
            self.std = np.sqrt(
                (1 - self.adaptation_rate) * self.std**2 +
                self.adaptation_rate * (new_value - self.mean)**2
            )
        ucl = self.mean + 3 * self.std
        lcl = self.mean - 3 * self.std
        return {
            'value': new_value,
            'cl': self.mean, 'ucl': ucl, 'lcl': lcl,
            'out_of_control': new_value > ucl or new_value < lcl
        }

Adaptive Control Limits: When Are They Needed?

Adaptive limits automatically adjust to slow process drift – e.g., tool wear or raw material changes. They prevent the flood of false alarms that static limits cause. Implementation uses EWMA adjustment of mean and standard deviation with adjustable adaptation rate.

Comparison of Control Chart Methods

Integration with MES

The SPC system receives online measurements from MES or directly from measuring equipment (CMM, spectrometers, test benches). On signal trigger, the batch is automatically blocked for inspection, operator and technologist are notified, and an NCR (Non-Conformance Report) is created in QMS. This cuts reaction time from hours to minutes.

{
  "event": "measurement",
  "timestamp": "measurement-timestamp",
  "parameter": "temperature",
  "value": 145.2,
  "subgroup_id": "A-123"
}

Results of AI-SPC Implementation

After implementation you get: scrap reduction by 20–30% through early violation detection, false alarms down by 40% thanks to adaptive limits and ML filtering, and equipment downtime cut by 15–25%. Savings from scrap reduction range from $100,000 to $500,000 per year for a medium-sized plant. Assess the potential effect for your production – order a production audit.

Our Work Process

1. Analytics: Audit current production, collect data, define critical quality parameters.
2. Design: Choose architecture (central or edge), configure adaptive limits.
3. Development: Implement detection models, integrate with MES/QMS.
4. Testing: Validate on historical data, run A/B test in parallel mode.
5. Deploy: Deploy on customer servers or cloud, train operators.

What’s Included

• Documentation: Data model, API specification, operator manual.
• Access: To monitoring system, dashboards, logs.
• Training: 2 days for technologists and operators.
• Support: 3 months post-production monitoring, bug fixes.

Implementation Stages and Timelines

Timelines and Cost

• Basic functionality (X-bar/R charts + WECO rules + alerts + MES connector): 3–4 weeks.
• Full set (EWMA, CUSUM, multivariate T², adaptive limits, process capability, QMS integration): 2–3 months.

Cost is calculated individually after project audit. Order the development of AI-SPC extension for your production – our engineers with 10 years of experience guarantee scrap reduction and efficiency increase. We will assess the project turnkey in 2 days. Contact us for a preliminary project evaluation.

Anomaly Detection: Autoencoders, Isolation Forest, PyOD

Server monitoring shows CPU 85%, memory 91% — is this the start of an attack or normal peak load? A classifier won’t help: anomalies are by definition rare, diverse, and not pre-labeled. Supervised learning requires examples of anomalies in the training set — so it fails on what you haven’t seen yet. Without an unsupervised approach, detection turns into guesswork.

Why Does Anomaly Detection Require an Unsupervised Approach?

The main problem: no labels and extreme class imbalance. Fraud transactions account for 0.01–0.1% of total volume, production defects 0.5–3%. With such ratios, a naive “all normal” classifier gives 99.9% accuracy but recall for the anomalous class near zero. Supervised models are powerless.

Second: “normality” is always contextual. A login at 3 AM may be normal for a night‑shift user but suspicious for a day‑worker. Bearing vibration at 2.3 mm/s depends on operating mode and machine age. So we embed context via feature engineering and time windows.

Third: quality assessment without ground truth. No standard test set — AUC‑ROC is possible only if a few labeled examples exist. For fully unlabeled data, only domain expert validation and indirect metrics work.

How to Distinguish an Anomaly from Noise in Real Time?

With adaptive thresholds and continuous monitoring of model statistics. In the case section we show how.

Isolation Forest is the standard baseline for tabular data. Idea: anomalies are isolated faster by random partitioning of the feature space. Works well at contamination=0.01–0.1, robust to feature scale, no normalization required. Implementation in sklearn.ensemble.IsolationForest.

Typical mistake: setting contamination='auto' without understanding the data. Auto mode assumes a threshold of -0.5, which may not match the actual anomaly proportion. Better: estimate expected anomaly percentage through domain knowledge and set it explicitly. We guarantee contamination tuning for your case.

PyOD (Python Outlier Detection) is a library with 40+ algorithms under a unified API — OCSVM, LOF, COPOD, ECOD, DeepSVDD, AutoEncoder. Useful for quickly comparing methods on the same data.

Autoencoders are the main method for unstructured data (time series, images, logs). Train the network to reconstruct normal data; anomalies yield high reconstruction error. Anomaly threshold is the 95th or 99th percentile of error on a validation set of normal data.

Practical problem with autoencoders: overfitting on “normal” patterns that are still rare. If the training set contains even a few anomalies, the model may learn to reconstruct them well. Solution: thorough cleaning of training data or using a Variational Autoencoder (VAE), which generalizes better.

LSTM‑AE for time series captures temporal dependencies better than a regular AE. Especially effective for multivariate time series (10+ sensors simultaneously). Implementation via PyTorch, training with MSELoss on sliding windows.

In Detail: Anomaly Detection in Industrial Time Series

Problem: vibration sensors on 12 pumps at a chemical plant, 6 sensors per pump, frequency 100 Hz. Need to warn of impending failure 4–24 hours in advance.

Solution architecture: raw data → feature extraction (RMS, kurtosis, peak factor, FFT amplitudes at resonant frequencies) → normalization by 24‑hour sliding window → LSTM‑AE → reconstruction error → threshold logic + alerting.

LSTM window size: 60 seconds (6000 points at 100 Hz). Too small a window misses slow patterns, too large loses sensitivity to rapid changes.

Anomaly threshold: not fixed, but adaptive. threshold = mean(errors_last_7d) + 3 * std(errors_last_7d). As normal state drifts (planned wear), the threshold adapts, avoiding false positives.

Result over a 6‑month pilot: detected 4 out of 5 real pre‑failure conditions (recall 0.8), with 2 false alarms over 6 months (precision 0.67). Before implementation: 3 unplanned shutdowns. After implementation: cost savings verified in the pilot report.

What Specifics Does Fraud Detection Face?

Financial transactions have several features that complicate detection:

• Concept drift: fraud patterns change faster than normal behavior. A model trained six months ago becomes obsolete.
• Adversarial adaptation: advanced fraudsters adapt — making transactions resemble normal ones.
• Temporal dependency: a series of normal transactions followed by one unusual transfer is a sequence anomaly, not a single point.

Practical stack for fraud detection: LightGBM with SMOTE oversampling for the supervised part (known fraud cases) + Isolation Forest for unsupervised (new patterns). Both signals combined in an ensemble; final decision via thresholds tuned for acceptable FPR (0.1–1% of transactions sent to manual review).

How to Evaluate Quality Without Labels?

When ground truth is absent, we evaluate using:

• Synthetic anomaly injection: add artificial anomalies (spike, level shift, point outlier) and check if the model detects them.
• Expert validation: random sample of top‑K anomalies → expert review → precision.
• Business metric: did the number of missed incidents / false alarms decrease after deployment?

Technical detail: the adaptive threshold is computed as mean(errors) + k * std(errors) on a 7‑day sliding window. Coefficient k is tuned on a validation set with synthetic anomalies to achieve FPR < 0.1%. When features drift, the window automatically shifts.

Process

1. Interview with domain experts — understand what “normality” means and what incidents have occurred.
2. EDA and data preparation — cleaning, feature creation, time windows.
3. Baseline (Isolation Forest) — fast validation on known incidents.
4. Model selection and customization — Autoencoder / LSTM‑AE / ensemble.
5. Training, validation with synthetic anomalies.
6. Deployment to production — pipeline on Kafka + Flink / Airflow, alerting to Telegram/Slack, drift monitoring.
7. Post‑deployment support — monitor model metrics, update thresholds.

What's Included

• Audit of current data and processes
• Development and training of models (Isolation Forest / Autoencoder / LSTM‑AE / ensemble)
• Configuration of adaptive thresholds and alerting
• Anomaly monitoring dashboard (Grafana / Streamlit)
• Model card and pipeline documentation
• Training for your team (2–3 sessions)
• 3‑month warranty support

Timeline: baseline system with one method — 2–4 weeks. Production system with adaptive thresholds, alerting, and monitoring — 2–5 months. Pricing is calculated individually for your case.

Our team has 8+ years of experience in industrial analytics and 15+ successful projects in anomaly detection for telemetry, finance, and IT monitoring. Get a consultation — we’ll tell you how to solve your problem. Contact us to discuss your data and receive a preliminary architecture proposal.