Backend development: where the system breaks under load
A backend rarely fails all at once — more often it degrades under load in stages: first response time grows at peaks, then individual operations start timing out, and only then does the system become completely unavailable.
You can live between these states for months, attributing complaints to the client's slow connection.
The first failure point is heavy queries to storage.
A fetch by an unindexed field, an aggregate over the entire table, or fetching related entities one by one in a loop, when traffic grows tenfold, turns from tens of milliseconds into seconds, and the connection pool is consumed faster than it can be released.
Second — locks in the database. A long transaction that updates rows needed by other queries builds a queue of waiters; if a call to an external service gets inside the transaction, the queue grows in proportion to its response time.
Some requests fail, the client retries them, and the retries add load on top of an already clogged queue.
Third — queues and duplicated logic. A queue without a size limit and without retry control grows faster than the worker can process it, and each retry multiplies the load on the same nodes.
A scenario duplicated across a controller, a console command, and a handler diverges over time: a change in one place breaks behavior in the other two, and this surfaces only during an incident.
For the product, all this results in increased response time during peak hours, failures of some operations, and the inability to name the application's bottleneck without profiling. Let's look at how this is addressed.
Practical effect: response speed, failures, downtime
Architectural decisions are visible not in diagrams but in measurable values: how much load the system sustains at peak, what share of operations completes successfully, and how many minutes it takes to recover after a failure.
Profiling and queues on Symfony provide these numbers before release, not after an incident. Below are four effects that the customer verifies against their own metrics.
Response time under load
We separate reads and writes, move heavy operations to background processing, and cache what is read more often than it changes.
The response time of the main scenario stays within the working range when requests grow severalfold: degradation starts later and is visible in metrics earlier than in complaints.
System predictability and failure class
An external service responds with a delay or is silent — this should not bring down checkout. Retry limits, failure isolation, and idempotent operations move some errors into the self-recovering class: the queue finishes processing, and the user receives confirmation. The class of failures that previously required manual data fixes goes away.
Recovery time and cost of downtime
Deployment without stopping the service, migrations separate from code, rollback to the previous release with one command — recovery is measured in minutes, not hours of coordination. The shorter the downtime window, the fewer missed orders and support requests.
Backend development formats: monolith, modules, services
The architecture of the server side determines how much it costs to bring in a second team and how quickly one function can be rewritten without affecting the others.
All three options are implementable on Symfony; the difference is not in the tools but in where module boundaries run and how tightly the code is coupled between them.
Therefore, choosing an architecture is an answer to the questions "how many people work on the project at the same time" and "are its parts loaded differently."
| Option | When it fits | Limitations |
|---|---|---|
| Single monolith | One team, domain still being clarified, speed of the first version matters | Module boundaries blur, code coupling grows, one change requires a full test run and a release of the entire system |
| Modular monolith | 2–4 teams, domain is clear, parts of the system evolve at different speeds | Discipline is required: dependency rules between modules are checked automatically, otherwise boundaries disappear within a couple of months |
| Service architecture | Parts of the system are loaded unevenly or require separate scaling and their own release cycle | Operations are more expensive: data consistency between services, network failures, separate environments, and observability for each |
In most cases, we start with a modular monolith: it provides explicit module boundaries and low code coupling but remains a single deployable application — this is cheaper both in development and in maintenance.
Extracting services is a continuation of evolution, not a starting decision: when the load on a specific part grows severalfold, the module is extracted separately, and the rest of the code is not rewritten.
Implementation procedure: from code audit to handover to operations
We work in stages, and each ends with a verifiable artifact, not a report on work done.
We fix the order before the start: first we understand what already works, then we change the data schema and configuration, then we test behavior under load — and only after that do we hand over to operations with procedures.
- Code and infrastructure audit. We review the project structure, duplicated business logic, areas without tests, and implicit dependencies between modules. The output is a list of bottlenecks with risk assessment and the order of fixes.
- Data schema and responsibility boundaries. We determine where the source of truth is, which relationships are mandatory, and which indexes are needed for real queries. We describe boundaries between modules with a schema and exchange rules — this removes half of contentious changes in the future.
- Environment and configuration setup. We bring local, test, and production environments to a single form, move secrets out of code, and describe migrations so that they run without manual steps and do not block deployment.
- Load testing. We capture a profile of real requests, reproduce the peak scenario, and see where response time grows and how the system degrades — through queues, failures, or deferred operations.
- Fixes for bottlenecks. Heavy queries, intermediate storage of frequent responses, moving background tasks out — we change what is confirmed by measurements, not what is "usually slow."
- Handover to operations. We deliver the project together with an update procedure: deployment order, conditions for rollback to the previous version, access, control points, and incident response procedure.
What remains in the project after delivery: diagrams, policies, procedures
Delivering code without documentation means the system survives only until the first serious failure or contractor change. Therefore, handover is a set of artifacts by which an engineer who has not seen the project can deploy the system, restore it, and change access rights. Everything is stored next to the code and updated with it, not in a separate folder.
- Architecture description: service boundaries, data flows, queues, cache, external integrations, and failure points — with an interaction diagram.
- Data schemas: entity and relationship model, migration history, rules for storing and deleting personal data.
- Access policies and roles: a "role — action — resource" matrix, the procedure for granting and revoking access, and where secrets are stored.
- Deployment diagram: composition of environments, configuration and environment variables, deployment order, and rollback to the previous release.
- Backup procedure: what is backed up and how often, retention period, and a protocol for verifying restoration in a clean environment.
- Update procedure: procedure for installing platform and dependency updates, maintenance windows, compatibility checks.
- Knowledge transfer to the team: sessions with the customer's engineers, recordings of sessions, a list of typical operations and known limitations.
The result is that the system can be developed by your own team or handed over to another contractor without reverse-engineering the code.
On a Symfony backend, such a package is assembled as standard: configuration, migrations, and policies are already described in the project — they only need to be recorded in a document.
Case study: reduced response time and eliminated an entire class of failures
A service with personal accounts and peak traffic: during the minutes of the morning rush, the number of requests grew severalfold.
The server side on Symfony was running on a single application node, and at peak, response time stretched into seconds, while some requests timed out.
The project case is illustrative because we captured metrics before changes and reworked the server side based on bottlenecks, not gut feeling.
Where the losses were
Every request for a list or card performed heavy synchronous operations — recalculating aggregates and calling external services directly in the handler.
Added to this were multiple database queries to fetch related entities and a complete absence of caching for hot data: the database, not the CPU, became the bottleneck.
At peak, locks accumulated, queue workers could not process tasks fast enough, and the unavailability of one external service dragged down timeouts for the entire account area.
What changed
Recalculations and external calls were moved to a queue with concurrency limits and separate pools for task types. Hot queries were covered by a cache with invalidation by related entities, and heavy queries were rewritten and reinforced with composite indexes.
For external dependencies, strict timeouts and degradation were set: the account area returns the last saved data instead of an error.
Metrics were captured with the same load profile. Before: response time at the 95th percentile was seconds, the share of responses with a server error was a few percent, and the service degraded at peak.
After: response time decreased sevenfold, the share of server failures dropped to fractions of a percent, and throughput on the same fleet tripled.
The elimination of failures was confirmed by load tests: we sustain a fourfold peak versus a normal evening load, response time grows linearly, without collapse or manual intervention.
What to do if the system is already running and cannot be stopped?
There is no need to stop the service — rework proceeds alongside the running system. New code is brought up on the same data and gradually takes on load while the old contour continues serving requests.
This mode requires discipline: parallelizing logic, staged migration, and backward compatibility at every step.
A matching layer appears at the application boundary: it routes a request either to the current implementation or to the new one.
Both contours work with the same storage — writes first go to the old one, and changes are duplicated; discrepancies are logged and resolved before the new branch becomes primary.
Contracts are extended rather than broken: new fields are optional, and the shape of previous responses is preserved, so external consumers do not notice the migration.
Switching happens in parts — by individual operations, user groups, or traffic segments. The share can be raised gradually and, at any step, reverted with a control flag, without deployment or downtime.
This is the difference from a full rewrite: risk is localized to one area rather than spread across the entire release.
For applications on Symfony, such a layer fits on standard routing and the service container — no separate platform for parallel operation is needed.
Before the start, we fix contracts, the data mapping schema, the set of metrics, and the rollback procedure: then the migration proceeds in predictable steps, and the system remains running the entire time.
Technical concerns before starting work
Before starting work, the customer checks not the team's speed but the risks: who holds the code, what happens with access, and what will remain after the contractor leaves. These concerns are resolved not with promises but with a working scheme. Below are answers to the most frequently asked questions.
We have our own development staff — why an external team?
Usually this is a mixed model: our engineers cover the server side on Symfony and infrastructure, while your staff covers domain logic and interfaces. We fix boundaries by modules so that changes do not collide in the same code, and your developers see every change.
Who owns access to the code and environments?
The repository, servers, and domains are registered to you from day one. We work under your accounts with limited rights, and you can revoke them at any time. Project handover is the background of the work, not a separate stage at the end.
We have a legacy solution — will you take it on?
First we analyze the actual state: tests, migrations, dependencies, hot queries. Then either a staged replacement by modules without stopping the product, or isolation of problematic areas and gradual decommissioning. We provide an estimate after the analysis, not before it.
What about data volume and performance?
We look at the load profile: read-to-write ratio, table growth, and the heaviest queries. We tune indexes, cache, and queues for it, and move reporting out of the main contour. We repeat the load test before every major release.
How does project handover to another team work?
Documentation, data schemas, environment settings, and the deployment procedure live in the repository together with the code. Handover is supporting the new contractor during onboarding, not a folder of files on request.
Let's discuss the task: what is needed for an estimate and plan
An estimate for backend work relies on the project's input data, not on general wishes.
The more accurately scenarios and current metrics are described, the fewer assumptions in the work plan and the more realistic the stage timelines and the load headroom built into the architecture.
To discuss the task substantively, what already exists is enough: process descriptions, access to the repository and environments, and monitoring data.
We do not wait for perfect documents — if something is missing, we record an assumption in the work plan and clarify it at the first stage, without shifting the overall deadline.
- Scenarios and roles: what actions users, administrators, and external systems perform, and which operations are critical in terms of response time.
- Load profile: peak periods, read-to-write ratio, seasonality, and expected growth in the number of requests.
- Current code: access to the repository, the branch with the current state, migrations, dependencies, and runtime environment requirements.
- Integrations: list of external services, exchange protocol, and the party responsible for their availability and changes.
- Infrastructure and access: environment diagram, access to logs and monitoring systems, and rights to run load tests.
- Constraints: security requirements, personal data handling, action auditing, and the update release procedure.
- Current metrics: response time of critical operations, incident frequency, known bottlenecks, and workarounds.
- Acceptance criteria: what result is considered achieved and by which metrics we verify it together.
From this set, we assemble a work plan: breakdown into stages, their timelines, team composition, and intermediate review points where the customer sees results before final delivery.
