Automatic Failover: Server Crash Switchover Setup

When your production server goes down, every second of downtime costs revenue. Without automatic failover, recovery can take hours while someone manually intervenes. We configure automatic failover so that traffic switches to a standby server within 30–120 seconds, slashing RTO from hours to seconds. For a typical e-commerce site earning $10,000/hour, reducing RTO from 15 minutes to 45 seconds saves over $2,200 per outage. With over 7 years of experience in fault-tolerant architectures and 40+ failover projects, our engineers hold AWS and Linux certifications and deliver stable, battle-tested solutions.

How automatic failover works

Failover can be implemented at different stack levels: DNS, load balancer, virtual IP (VRRP), and database. Each has trade-offs in switchover speed, complexity, and cost.

DNS-level is the simplest but slowest. A health check polls the primary every 10–30 seconds. On failure, the DNS record is updated to point to the standby server. The total delay comprises the record TTL plus detection time: 60–300 seconds. Suitable for most web applications where a 5-minute pause is acceptable.

Load Balancer (e.g., AWS ALB/NLB, nginx upstream) switches in 5–30 seconds but requires both servers in the same cloud or region. Health checks operate at the balancer level.

VRRP / Keepalived uses a virtual IP that moves between servers when the master fails, achieving 2–5 second switchover. Classic for on-premise and dedicated setups. Keepalived is 5 times faster than DNS failover.

Database failover is a separate challenge. The application must know the new primary DB. Solutions like Patroni (PostgreSQL), MHA (MySQL), or AWS RDS Multi-AZ handle this automatically.

DNS vs Load Balancer: Comparison

DNS failover is 10× slower than a load balancer but simpler and provider-agnostic.

Example: AWS Route 53 configuration

Route 53 Failover Policy:
  Primary record → 1.2.3.4 (main server)
    Health check: HTTP GET /health, port 443
    Failure threshold: 3 consecutive failures
    Request interval: 10 seconds
  Secondary record → 5.6.7.8 (standby server)
    Evaluate target health: Yes

The /health endpoint must verify real readiness: database accessible, cache working, disk space sufficient. It should return 200 only when fully operational. Our engineers customize this check to your stack.

Keepalived for bare metal and VPS

# /etc/keepalived/keepalived.conf on PRIMARY
vrrp_script check_app {
    script "/usr/local/bin/check_app.sh"
    interval 5
    weight -20
    fall 2
    rise 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    virtual_ipaddress {
        192.168.1.100/24
    }
    track_script {
        check_app
    }
}

The check_app.sh script verifies application liveness locally. After two consecutive failures, the BACKUP server (priority 90) takes over the virtual IP.

Why failover testing is critical

Regular drills are mandatory. An untested failover will likely fail when you need it. Our standard verification protocol:

1. Confirm monitoring records the initial state.
2. Simulate failure: systemctl stop nginx or iptables -I INPUT -p tcp --dport 80 -j DROP on primary.
3. Record time until switchover.
4. Verify service via the standby server.
5. Restore primary, test automatic fallback.

Target metrics: detection time < 30 s, switch time < 60 s, total RTO < 120 s. Automatic failover is 20 times faster than manual recovery.

Case study: High-traffic e-commerce failover

A client with a peak-time load of 5000 requests per second experienced 15-minute downtimes when their single server failed. We implemented a two-tier failover: Keepalived for virtual IP switching and Patroni for PostgreSQL database failover. The result: automatic recovery in under 45 seconds, even during Black Friday traffic spikes. Revenue loss from unplanned downtime dropped by over 85%, saving approximately $2,000 per outage.

What's included in our turnkey failover setup

• Analysis of current architecture and availability requirements
• Designing the failover scheme (DNS, load balancer, VRRP, DB)
• Configuring health checks and monitoring
• Implementing data synchronization (database replication, file sync, sessions)
• Developing automatic switchover scripts
• Testing failure and recovery scenarios
• Documentation and team training
• Optional post-deployment support

Our team: 7+ years of experience, 40+ implemented failover projects, and certified engineers (AWS, Linux).

Setup duration estimates

Exact timeline depends on infrastructure complexity. Contact us for a free assessment.

The split-brain problem and its solution

Split-brain occurs when both servers claim to be primary. In Keepalived, it is solved by fencing (STONITH) — the weaker node is forcibly shut down upon conflict. In PostgreSQL/Patroni, an external DCS (etcd, Consul, ZooKeeper) acts as an arbiter. We guarantee your scheme prevents this scenario through split-brain prevention mechanisms.

Monitoring failover events

Every switchover is an incident that requires investigation. Alertmanager or PagerDuty capture the event. A ticket is automatically created in Jira/Linear. After remediation, a root cause analysis determines why the primary failed.

Contact us to discuss your project. Order failover configuration and ensure your service stays available.

We regularly encounter a situation: "The site is not opening" at 3 a.m. — and it turns out that the VPS disk is full because nginx logs haven't been rotated for six months. Or the server went down under load on the day of an advertising campaign launch because the shared hosting had a limit of 50 concurrent connections. Setting up hosting and deployment is not about "where it's cheaper" but about what happens when something goes wrong. Our team helps avoid such incidents by designing infrastructure that accounts for real load patterns.

When to choose Vercel and Netlify?

Vercel is built for Next.js — deploy in one push, preview deployments for every PR, automatic CDN, Edge Functions, ISR without configuration. For frontend projects and JAMstack, it's the optimal choice: no operational overhead, time-to-deploy measured in minutes.

Real limitations: Vercel Serverless Functions run in us-east-1 by default (latency for Europe +80–100ms), Function timeout 300 seconds on Pro, Bandwidth 1TB/month on Pro. For heavy backend, you need workers or a separate server.

Netlify is closer to static sites and Edge Functions based on Deno Deploy. Build minutes are the main limitation on the free tier.

Why is Docker the foundation of predictable deployment?

"It works on my machine" — classic. Docker solves this through environment containerization. But a bad Dockerfile creates new problems.

A typical mistake: copying everything into the image without .dockerignore, resulting in an 800MB image instead of 80MB. node_modules inside the image weighs as much. Correct approach: multi-stage build.

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM node:20-alpine AS runner
WORKDIR /app
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./package.json
EXPOSE 3000
CMD ["npm", "start"]

Final image: 180MB instead of 1.2GB. CI build time is reduced due to layer caching — if package.json hasn't changed, the layer with npm ci is taken from cache.

Docker Compose for local development and simple production scenarios: application + PostgreSQL + Redis in one configuration. For production on a single server, it's a perfectly viable option if there's no requirement for horizontal scaling.

More about containerization — Wikipedia: Docker.

How to set up Nginx as a reverse proxy?

Nginx in front of the application is standard for VPS and dedicated servers. Main functions: SSL termination, gzip, static files, rate limiting, upstream load balancing.

A configuration often done incorrectly: worker_processes auto — number of processes equals CPU count. worker_connections 1024 — that's 1024 per worker process. With 4 CPUs and 1024 connections = 4096 concurrent connections. For a high-traffic site, you need worker_connections 4096 and set keepalive_timeout 65.

For static assets with hash in the filename:

location ~* \.(js|css|woff2|png|webp)$ {
    expires 1y;
    add_header Cache-Control "public, immutable";
}

immutable tells the browser: don't revalidate this file even on hard refresh. This only works correctly with content-hashed filenames (which Vite/webpack do by default). Documentation — Wikipedia: Nginx.

AWS: flexibility and complexity

EC2 + Auto Scaling Group — classic for horizontal scaling. AMI with pre-installed application, Launch Template, ASG with min/desired/max instances, Application Load Balancer. When CPU > 70% for 3 minutes — scale out, when CPU < 30% for 15 minutes — scale in. Health check via ALB removes unhealthy instances from rotation.

ECS Fargate — containers without managing EC2. Deploy a Docker image, specify CPU/memory (512 CPU units = 0.5 vCPU, from 512MB memory), Fargate launches it. More expensive than Lambda, but no cold start and no timeout limitations. Suitable for long-running processes, WebSocket servers, heavy workers.

RDS for PostgreSQL with Multi-AZ: automatic failover in 1–2 minutes when primary fails. Read Replicas for scaling reads. RDS Proxy for connection pooling — Lambda functions cannot hold long-term connections, the proxy buffers this.

Kubernetes: when it is justified

K8s adds significant operational complexity. Justified when: multiple teams deploy independent services, fine-grained resource allocation per service is needed, canary deployments and blue/green without downtime are required.

AWS EKS, GKE, or managed k8s from Hetzner (cheaper). Helm charts for standard services. Horizontal Pod Autoscaler based on CPU and custom metrics (RPS via Prometheus).

For most startups and medium-sized projects, Kubernetes is overkill. ECS or Fly.io provide 80% of the capabilities with 20% of the operational complexity.

Monitoring and alerting

A server without monitoring is waiting for an incident. Minimal stack: Prometheus + Grafana (or Grafana Cloud for managed), alerting on disk > 80%, memory > 85%, CPU > 90% over 5 minutes, error rate > 1%. Uptime via Better Uptime or Upptime (self-hosted).

Logs: Loki + Grafana or CloudWatch Logs Insights. Structured JSON logs (winston, pino) are mandatory — otherwise, log searching becomes a pain.

What is included in hosting setup

• Audit of current infrastructure and load profiling
• Selection of target architecture (VPS, AWS, serverless, Kubernetes)
• Setting up CI/CD pipeline (GitHub Actions, GitLab CI) with automatic deployment
• IaC via Terraform or Pulumi (infrastructure as code)
• Configuration of Nginx, SSL certificates, HTTP/2, brotli
• Monitoring and alerting (Prometheus + Grafana, PagerDuty)
• Documentation of runbooks and team training

Additionally, contact us if you need migration from current hosting or integration with external services.

Work process

1. Audit of current infrastructure (2–5 days)
2. Selection of target architecture with load and budget justification (1–3 days)
3. Setting up CI/CD pipeline (GitHub Actions, GitLab CI) (2–5 days)
4. IaC via Terraform or Pulumi (3–10 days)
5. Setting up monitoring and alerting (2–5 days)
6. Documentation of runbooks and team training (1–3 days)

Our experience — 7 years on the market, over 50 projects, guarantee of operability after deployment.

Timeline

• Basic deployment on VPS with Docker + Nginx + CI/CD: 1–2 weeks.
• Setting up AWS infrastructure with Auto Scaling, RDS, CDN: 3–6 weeks.
• Migration to EKS from scratch: 6–12 weeks.
• Setting up Vercel/Netlify for JAMstack: 3–5 days.

The cost is calculated individually depending on complexity and scope of work. Get a consultation — we'll evaluate your architecture in one day.