Federated learning: where the model breaks on disparate data
Data resides across perimeters: at customers, partners, branches. Collecting it into a single storage is impossible — policy prohibits export, and some datasets never leave the perimeter at all.
Federated learning removes this contradiction: training on distributed data happens where the records already reside, and only model updates travel between participants, not the data itself.
Before that, the team hits a ceiling. On disparate datasets, each participant trains alone: there are few examples, rare classes are represented by single records, and model quality on a small sample does not improve from new features or a change of architecture. A shared model would provide a gain, but there is nothing to train it on.
Mechanics: the coordinator sends the current model version to participants, each fine-tunes it on its own perimeter and returns only weights, and the coordinator averages them taking into account the volume and quality of the local contribution. Records do not leave the perimeter at any step — this is the main argument for security and legal.
The scheme breaks predictably.
Participants bring different data schemas and different class balance: a rare class of one participant weighs as much as a frequent class of another in averaging, and a node that drops out for several iterations pulls the aggregate aside and nullifies the progress of others. We analyze how this is closed.
What model training without uploading data upward provides
Federated learning (Federated Learning) changes the assembly point: the model is trained where the data resides, and only updates go upward.
The first measurable consequence — the failure class "no data — no model" goes away: the project does not stall if one of the participants cannot export the corpus.
The share of sites with a working model grows — instead of isolated launches in a distributed perimeter, most connected participants reach the result.
The model learns on the entire aggregate without collecting it in one place
Each node computes an update on its records and transmits only parameters upward — raw data does not leave the local perimeter. The server aggregates updates and sends out a new version; the cycle repeats until the metric reaches a plateau.
The contribution of a participant with a small sample is not lost: weights are aggregated with a correction for volume, so rare segments enter the model just like mass ones.
Smaller leakage surface and fewer legal approvals
Model updates rather than records themselves go outside — the volume of transmitted artifacts is reduced by orders of magnitude, and personal data remains under local control.
Role separation and access logging are configured inside the participant's perimeter, so interception of the exchange channel does not expose records.
The legal perimeter narrows: instead of approving corpus export, it is enough to describe the exchange format and aggregation rules — approvals go faster because the subject of discussion is the model, not personal data.
Federated learning: modes of exchanging model updates
Participants in the perimeter do not transmit raw data to each other — only model updates travel over the network: weights after a local training round or their compressed representation.
The resistance of the perimeter to participant dropout and how predictably the model reaches the target metric depend on how this exchange is organized and who performs update aggregation.
Synchronous and asynchronous exchange
In synchronous mode, a round closes when all participants have sent updates: aggregation uses the full set, the metric grows evenly, but the pace is set by the slowest site.
Asynchronous accepts updates as they arrive — fast nodes do not wait for the rest, but version divergence control is needed, otherwise fresh weights are diluted by outdated ones.
A workable compromise is synchronous rounds with a threshold: a round closes based on the share of participants or a specified waiting time, and the perimeter does not stall because of one node.
Aggregation on a dedicated server or without it
The classic scheme is a dedicated aggregation server: it combines updates, maintains the global model version, and stores a round log, which provides simple audit and rollback.
When a single node cannot be trusted, aggregation is distributed among participants: the dropout of an individual site does not break the perimeter, but version coordination and conflict resolution fall on the participants themselves.
The common limitation of the approach is that exchanging updates by itself does not close leakage through weights: noise and limits on the contribution of a single participant are needed.
| Variant | When suitable | Limitations |
|---|---|---|
| Synchronous exchange | Sites are comparable in capacity and channel | Pace is set by the slowest node |
| Asynchronous exchange | Different load and connection quality | Version divergence, weight dilution |
| Dedicated aggregation server | Round audit and simple rollback are needed | Single point of failure and trust |
| Distributed aggregation | A single node cannot be trusted | Version coordination is more complex |
Stages of implementing a federated perimeter
A federated perimeter is implemented not by deploying a server, but by a sequence of steps: each next one relies on the result of the previous.
A skipped stage comes back during convergence checks, when the model has already diverged on someone else's slice, and the cause is sought across all nodes at once.
Audit and inventory of perimeters
At each participant, we record the composition of features and their types, identifier schema, labeling, and permitted access modes.
Discrepancies in field names, gaps, and different units of measurement break aggregation more than the choice of algorithm, so we assemble a common dictionary before the first training.
- Exchange schema — what is computed at the node and what goes outside: only model parameters, without raw records.
- Setup of local training and aggregation — number of rounds, participant quorum, rejection of anomalous updates.
- Pilot implementation on two nodes — short rounds on a limited slice to catch configuration errors.
- Convergence check — metrics on a holdout sample of each participant and control for absence of degradation.
- Handover to operation — rollback procedure, round monitoring, training of the customer's team.
Pilot on two participants before scaling
Two nodes are enough to test exchange, access, and convergence on live data, but coordination still remains simple. Then we connect participants one by one and compare metrics — this way the growth of the perimeter does not mask regression.
Schemes, access policies, and regulations at the output
The customer receives not a presentation, but a working federated learning perimeter (federated learning) with everything needed for independent operation. Below is a list of artifacts that are transferred with the solution and updated as the system evolves.
Access policies and participant roles
- Participant roles: round coordinator, local data owner, contribution auditor, rollout operator — with clearly described rights and prohibitions for each.
- Access policies: who initiates a round, who sees aggregated metrics and the training log, who has the right to stop the process.
- Perimeter scheme: composition of nodes, route of updates between them, contribution control points, and rules for cutting off anomalous models.
- Participant configurations: local training parameters, limits on the volume of transmitted updates, session schedule.
Regulations for connecting a new perimeter
- Participant connection template: checklist, baseline set of configurations, and a verification round without transmitting raw data.
- Operating regulations: round calendar, quality thresholds for the aggregated model, actions when a node drops out, and version rollback procedure.
- Documentation: architecture description, training log, instructions for the on-duty engineer, and incident escalation procedure.
Case: a single model without exporting data from clinics
The task came from an association of nine medical organizations: each has its own format of images and labeling, and exchanging the studies themselves is impossible — neither legally nor ethically.
Training on local samples produced weak models on rare classes: sensitivity for them stayed around 0.6, and the retraining cycle took six weeks.
In each organization's perimeter, a training node was deployed: data remains in place, and only the weight update vector goes outside.
Aggregation happens on our side — weights are averaged across rounds, with secure aggregation and differential privacy on top, so the original record cannot be reconstructed from the update.
We took the open-source Flower as a basis, supplementing it with a round log and rollback to the previous model version.
Metrics before and after
| Indicator | Before | After |
|---|---|---|
| Sensitivity on rare classes | 0.61 | 0.86 |
| Share of false-positive conclusions | 18% | 9% |
| Retraining cycle | 6 weeks | 5 days |
| Records that left the perimeter | entire samples | 0 |
What changed in operation
Connecting a tenth site is a new round, not an integration from scratch: the node is deployed in its perimeter and then participates in training on equal terms with the others.
Each round goes into the log — the site's contribution and quality dynamics are visible, and rollback to the previous version takes minutes.
Security and the regulator see the same artifact: the model is trained on the aggregate of nine samples, while not a single record left the perimeter.
How does a federated perimeter differ from centralized data collection?
Centralized data collection is the familiar path: export samples from sources into a single storage, clean, train the model, and roll out.
Quality control is higher: feature distributions are visible, outliers are easier to remove, and borderline cases are easier to label; training is reproducible on one test bench.
The price is approvals for each export, data copies with an unclear lifespan, and a separate perimeter for protecting them.
A federated perimeter (Federated Learning) flips the scheme: data remains at the source site, and only weight updates and aggregated statistics travel between nodes.
Accuracy is usually lower than with full collection: heterogeneity of samples at sites and losses during averaging have an effect. But approval requirements shrink to a regulation for exchanging updates instead of permission to transfer personal data.
When a federated perimeter is not needed: if there is a single data source, it is yours, and the samples are homogeneous — centralized training will produce a more accurate model and is cheaper to support.
Comparison of approaches should be based not on "security in general," but on the number of sites, export restrictions, and who is responsible for data quality at each node.
Supporting the perimeter is separate engineering work: orchestrator of training rounds, model version registry, monitoring of lagging nodes, rollback when metrics diverge. Therefore it is planned when restrictions on data transfer are stable, not one-off.
Frequently asked questions: privacy, model quality, failures
Doubts at the evaluation stage of a federated learning perimeter are usually the same: will the model converge on heterogeneous data, what to do with a participant that spoils updates, how to leave the perimeter without losing the model, and who owns the result. Below are short answers; details are fixed in project documentation before start.
Will the model converge if participant data differ greatly?
Yes, but convergence is set by perimeter parameters. We fix the number of local epochs, weight updates by data volume and quality, and monitor the metric on a common control set. If the spread between participants grows, we reduce local epochs, and training stabilizes.
What happens if a participant tries to poison the model?
The aggregator cuts off anomalous updates: we compare them with the general training direction and check on a control set, not on the participant's word. A suspicious contribution goes into quarantine, and the participant's weight in subsequent rounds is reduced.
How does a participant leave the perimeter and return back?
The perimeter is not reassembled: the aggregator stops taking the participant's updates into account; the weights already received remain in the model. Upon return, it trains locally from scratch or starts from the common base model; round history is preserved.
Who owns the trained model and how is contribution divided?
Rights are fixed in the contract before start: data remains with the owner, and the procedure for using the common model is the subject of a separate agreement. A participant's contribution is calculated by the volume and quality of updates, not by the size of its data.
Let's discuss your task: perimeter, data, constraints
An estimate for such a project is not built on general words — a conversation is needed about your perimeter, data, and what already works.
To make the first conversation substantive, it is enough to describe a few things: this will be sufficient to understand the scope and come back with a plan, not with clarifying questions.
- Data composition — which sources participate in training, which fields the model needs, where sensitive attributes reside, and who owns each dataset.
- Perimeter constraints — what is prohibited from being taken outside the perimeter, which information security and regulator requirements apply, and whether there are fully isolated segments.
- Training participants — how many nodes, how homogeneous they are by data volume and quality, how often they lose connection, who administers them.
- Current model and metric — what already works, by which metric you measure quality, and what level you consider sufficient for launch.
- Infrastructure at the nodes — what is deployed, whether accelerators and orchestration are available, who has rights to launch training.
- Timelines and acceptance — who accepts the result, which checks are mandatory, and whether rollback to the previous model version is needed.
In return, we will come back with an estimate and a plan: a perimeter scheme, sequence of stages, composition of artifacts, and a list of risks with points where work can be paused without losing the result.
Describe the task in a convenient form — we will come back with an estimate and a plan and propose the next step.
