AI forecasting of IoT data: from preprocessing to edge

Why IoT time series forecasting breaks in practice

A model converges well on historical exports, but on live telemetry, IoT time series forecasting falls apart in the first weeks. Collectors at different sites write data in their own formats, at different frequencies, and with their own timestamps.

Joining such a stream without bringing it to a unified time grid produces shifts — and the model learns from events that did not occur simultaneously.

Gaps and noise in telemetry are not an anomaly but an operating mode. Communication with the sensor drops, the value sticks at the last reading, and spikes occur during equipment startup.

Interpolation smooths the gap, and the model confidently draws a flat line where there was a real jump.

Separately, model drift accumulates: after a change in regulations, replacement of a unit, or seasonal load growth, the relationship between inputs and target changes, while the weights remain the same.

On a holdout sample, the metrics look decent — the discrepancy appears only in operation, when the forecast stops matching the fact.

Operations pay for this with delayed reaction to deviation: until the discrepancy becomes obvious, equipment goes into emergency repair instead of planned, and downtime and lost output are counted after the fact.

Below we break down how this is addressed — from stream preprocessing to forecasting directly on the equipment.

What predictability delivers: measurable effects

Effect is measured on the same line before and after: hours of unplanned stops, crew visits, manual inspections per shift. We keep the series history in InfluxDB, train models on several seasons of operation — this is enough to separate normal wear from an anomaly.

Downtime: a third of stops move out of unplanned

A vibration peak and bearing temperature rise are visible several hours before failure. Intervention is moved into a planned window, and at the pilot site the share of unplanned downtime falls by about a third.

The class of failures addressed first

Most confidently, the model catches mechanical degradation, clogs, and overheating — what causes most incidents. Rare electronics failures with few examples are left to scheduled control: promising accuracy on them would be dishonest.

Repair planning horizon

For slow processes, the horizon is a day or more; for fast units, hours. This is enough to put repair into the schedule and prepare a spare part in advance, rather than after a failure.

Prediction accuracy and false alarms

We tune the threshold to the cost of a miss: for critical units we keep a margin, for secondary ones we disturb personnel less often. In a test run, the share of correct triggers stayed above 0.9 with a lead time of 4–6 hours.

Manual checks by fact, not by schedule

Scheduled rounds are replaced by alarm-driven inspection: the operator goes where the model sees a deviation. The number of manual checks per shift decreases, and attention goes to genuinely worn units.

Model computation modes: batch, streaming, edge

The computation mode is chosen not by model size, but by where the decision arises and how quickly it loses value.

A batch run over accumulated history gives the fullest picture and allows model versions to be compared with each other; streaming keeps a window in memory and responds as data arrives; edge computation runs where the data is born, without exporting raw measurements.

In practice, the modes do not compete but complement each other: batch retrains and calibrates the model, streaming serves operational decisions, and a fallback version remains on devices, running when the channel breaks. Below are the applicability boundaries of each mode and what you have to pay for.

The choice between them is determined by the cost of an error and the acceptable reaction time, not by implementation convenience: the combination of "batch retraining + streaming inference + edge fallback" covers most industrial tasks, whereas betting on a single mode almost always runs into its limitations.

Implementation stages: from data audit to model handover

Data from sensors arrives unevenly: communication gaps, calibration drift, different polling frequencies. A model on a "raw" series catches noise, so IoT time series forecasting follows a fixed sequence — each stage has a verifiable output.

1. Data audit. We break down the sources: history depth, share of gaps, spikes, delivery delays. The output is a list of series suitable for forecasting and a realistic prediction horizon.
2. Pipeline design. The path from the on-device buffer to storage: where rolling windows are computed, how gaps are filled, how features are versioned. We fix the design before code so that training and the real stream compute identically.
3. Feature engineering. Lags, rolling statistics, calendar and external factors — temperature, line load, shift. We check each feature for real-time reproducibility: one computed only offline breaks in operation.
4. Training and validation. We split history into three non-overlapping periods: training, tuning, control. We calculate metrics by horizons and equipment operating modes, not as a single average across the series.
5. Testing on historical slices. We run the model over past windows with the same feature scheme as in operation. We look at the error on failures, drift, and seasonal peaks — where the forecast is needed by production.
6. Handover to operation. The model, pipeline configurations, and features go into monitoring with alerts for drift and accuracy degradation. The customer's team receives documentation, a breakdown of metrics, and training for independent support.

What remains in your perimeter after launch

A project is delivered not when the model has shown the required metric on a holdout sample, but when your engineer can launch the pipeline, explain each of its links, and update the model without us.

Therefore, what is handed over to the perimeter is not a folder with scripts, but a working system with a description of all its parts.

• Pipeline diagram — the data path from telemetry ingestion to forecast output: which links exist, where intermediate marts are stored, at which points restart occurs, and what happens when communication with some sensors is lost.
• Configuration files — aggregation window, forecast horizon, trigger thresholds, and outlier rejection rules are moved out of code: the on-duty engineer changes them without rebuilding.
• Retraining procedure — by what sign the discrepancy between forecast and fact is recorded, who confirms the rollout of a new version, and how to roll back to the previous one with a single action.
• Data access policies — roles for reading raw telemetry and writing to the forecast storage, separation by site, and logging of access.
• Documentation — architecture diagram, description of input stream fields, feature registry, known model limitations, and typical causes of degradation.
• Instructions for the on-duty shift — scenarios for stream loss, growing processing delay, and discrepancy between forecast and fact, indicating when to call us.

All of this is versioned in your repository and edited during operation — the perimeter does not remain without a description after handover.

Case study: predicting pump failures from telemetry

The customer is a production site with a fleet of 60+ pump units in a continuous cycle. Telemetry was collected from them for years: vibration, motor current, bearing temperature, inlet and outlet pressure.

The data lay in an archive and was used for post-mortem analysis — no one counted on it for forecasting. We built failure prediction on these same series, without replacing sensors or stopping equipment.

What it was like before implementation

A failure was discovered after the fact: first pressure dropped or emergency protection tripped, then the repair crew went out. The dispatcher manually reviewed several hundred tags per shift, and attention went to values that had already deviated.

A bearing or seal was replaced in emergency mode — along with the unit, a batch of product and the service life of neighboring units were lost.

What we changed

We assembled features over a 30-day window: vibration trends, current growth at constant load, bearing temperature drift relative to the mode.

The model outputs a risk estimate for a horizon of up to a week and is recalculated as telemetry arrives from Grundfos units and the rest of the fleet. The alert goes to the repair service indicating the unit and time margin, not "check the pump."

We separately configured drift control: if the input data distribution shifts, the system reports it before accuracy drops. The share of emergency stops fell by a factor of three, and repair moved into a planned window — without night call-outs and unplanned line downtime.

How does edge forecasting differ from cloud forecasting in IoT?

A time series forecasting model can be computed in two places — on site and in the cloud. These are not mutually exclusive options: the choice is determined by where fast response is needed and where analytics over accumulated history is sufficient.

Edge computation keeps prediction close to the sensors: response fits within fractions of a second and does not depend on channel quality, while perimeter autonomy is preserved when communication breaks — the device accumulates readings and uploads them upon restoration.

Only aggregates and events go over the network, so connectivity at the site remains cheap to maintain.

Cloud computation enables heavy models and training on the full fleet history, but every prediction depends on connectivity: when the channel degrades, response delay grows, and when it breaks, the perimeter stops.

In practice, a hybrid is more often chosen: a lightweight model version lives on the device, the full one in the cloud, and the weights are synchronized on a schedule. Loss of communication then does not break the forecast, and the cloud refines it when data arrives.

Common technical doubts before starting

The questions that most often come up at the assessment stage: history depth, sensor behavior, model updates on a running perimeter, and how to output the result externally. Below are brief technical answers.

What to do with gaps in telemetry?

We label gaps by type: single sensor failures, planned stops, communication breaks. The first are restored by interpolation with a reliability flag, the second are marked as service intervals and excluded from training.

The gap mask goes as a separate feature — the model does not learn from holes as if they were signal.

How to update the model without stopping the perimeter?

Training runs on a copy of the data outside the perimeter, and the new version is run in shadow mode alongside the current one. We compare forecasts on the live stream and switch traffic only after the metrics match. Rollback is a return to the previous version, and the perimeter is not stopped.

How to know that the forecast is no longer accurate?

We monitor not only the error but also its drift: comparison with a naive forecast, the share of outputs outside the confidence interval, stability across horizons. Discrepancy on new modes is a signal for additional training, not a reason to wait for a scheduled review.

How to output the forecast outside the perimeter?

We deliver results through a one-way gateway: only forecast values and quality labels go outside, without raw setpoints and control commands. The perimeter remains closed, and analytics receives data for monitoring and reporting.

Let's discuss your task and come back with a plan

Describe the task in free form: what object, what sensors, and what parameter we are forecasting, and what event needs to be seen in advance.

As input, a telemetry export for several months, a failure log with timestamps, and a polling scheme are sufficient — we will clarify the rest in conversation. We will analyze the data and come back with an estimate and work plan.

• What to send: telemetry export in CSV or Parquet, polling frequency by channel, list of sensors with units of measurement, log of failures and downtime.
• Data review: gaps, outliers, sensor drift, and clock desynchronization; we will show which channels carry signal and which carry noise.
• Estimate and work plan: forecast horizon, acceptable error, where we compute — in the cloud or on edge controllers at the equipment.
• First site for launch: one line or unit, a minimal "sensor → feature → warning" perimeter so the effect is visible on a small volume.
• Acceptance metrics: share of caught failures, false alarms per shift, time margin from warning to event.
• Model support: drift monitoring, scheduled retraining, versioning, and rollback to the previous build.

We will adapt the sequence to your perimeter — we will start with the site where downtime costs the most.