Automatic Resource Scaling Based on Load

Imagine: your e-commerce site on a Friday evening under load — CPU at 90%, latency rising, and you can't add servers manually in time. Or the opposite: on weekdays, servers idle while you pay for 80% unused capacity. Autoscaling solves both problems. We've set up horizontal scaling for 20+ projects — from startups to enterprise. Below — real code, configs, and common mistakes.

Problems We Solve

Overspending on Resources in Quiet Times

Average web server CPU utilization is 15-20%. Holding capacity for peak means paying for 80% unused power. Autoscaling keeps minimum instances and adds new ones as load grows. For example, an e-commerce site with peak load 20k RPS normally runs 10 servers, though average load is 2k RPS. Autoscaling allows running 2 servers and adding as needed. For infrastructure cost optimization, savings amount to 30-40% of the budget. For one client (e-commerce with 20k RPS peak), implementation cut monthly expenses by $4,500. Cost savings: $4,500 per month.

Downtime During Sudden Spikes

A DDoS or viral post can multiply load tenfold in minutes. Without scaling, the site goes down. Our solution reacts within seconds using Target Tracking and a 60s scale-out cooldown.

Complexity of Manual Management

Even an experienced admin can't manually add servers in time. Automation eliminates human error.

How We Do It: Stack and Case Study

AWS Auto Scaling Group with Target Tracking

We use Terraform for infrastructure as code. Example configuration:

resource "aws_autoscaling_group" "app" {
  name                = "app-asg"
  min_size            = 2
  max_size            = 20
  desired_capacity    = 3
  vpc_zone_identifier = var.private_subnet_ids

  launch_template {
    id      = aws_launch_template.app.id
    version = "$Latest"
  }

  health_check_type         = "ELB"
  health_check_grace_period = 60

  target_group_arns = [aws_lb_target_group.app.arn]
}

# Target Tracking: keep CPU at 60%
resource "aws_autoscaling_policy" "cpu_tracking" {
  name                   = "cpu-tracking"
  autoscaling_group_name = aws_autoscaling_group.app.name
  policy_type            = "TargetTrackingScaling"

  target_tracking_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ASGAverageCPUUtilization"
    }
    target_value       = 60.0
    scale_in_cooldown  = 300
    scale_out_cooldown = 60
  }
}

Scale-out cooldown (60s) is shorter than scale-in (300s) — reacting quickly to growth, slowly removing resources.

Kubernetes HPA with Custom Metrics

Horizontal Pod Autoscaler combined with Prometheus Adapter allows scaling by custom metrics:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app
  minReplicas: 2
  maxReplicas: 50
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

The http_requests_per_second metric comes from Prometheus via kube-state-metrics and Prometheus Adapter.

KEDA: Scaling by External Sources

KEDA (Kubernetes Event-Driven Autoscaling) scales pods by queue length from RabbitMQ, Kafka, SQS:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: queue-processor
spec:
  scaleTargetRef:
    name: worker-deployment
  minReplicaCount: 1
  maxReplicaCount: 30
  triggers:
    - type: rabbitmq
      metadata:
        host: amqp://rabbitmq:5672/
        queueName: tasks
        queueLength: "50"

Scaling to zero when the queue is empty — saving resources.

Predictive Scaling

AWS Predictive Scaling forecasts load based on historical data (minimum 14 days) and adds resources proactively:

resource "aws_autoscaling_policy" "predictive" {
  name                   = "predictive"
  autoscaling_group_name = aws_autoscaling_group.app.name
  policy_type            = "PredictiveScaling"

  predictive_scaling_configuration {
    mode                         = "ForecastAndScale"
    scheduling_buffer_time       = 300
    max_capacity_breach_behavior = "IncreaseMaxCapacity"

    metric_specification {
      target_value = 60
      predefined_scaling_metric_specification {
        predefined_metric_type = "ASGAverageCPUUtilization"
      }
      predefined_load_metric_specification {
        predefined_metric_type = "ASGTotalNetworkIn"
      }
    }
  }
}

Scaling Method Comparison

How to Choose the Right Metric?

The metric is key to effective scaling. CPU lags, Request Rate needs a baseline, P95 is the most accurate but complex. In practice we use a combination of CPU + Request Rate. For async workers — Queue Depth. According to AWS Auto Scaling documentation (Amazon EC2 Auto Scaling) supports multiple metrics in one policy. Infrastructure costs drop by 35% with properly selected metrics.

What If Scaling Doesn't Trigger on Time?

Check cooldown (scale-out faster than scale-in), health check grace period. Sometimes the metric hasn't updated. We recommend a load test with k6: k6 run --vus 1000 --duration 10m script.js.

Process

1. Analytics: analyze load profile, historical data, select metrics.
2. Design: determine scaling type, tools (AWS ASG, K8s HPA, KEDA).
3. Implementation: write IaC (Terraform/Pulumi), set up monitoring (CloudWatch, Prometheus).
4. Testing: load testing, verify response time and downtime.
5. Deploy: production rollout, alerts (SNS, PagerDuty).
6. Support: monitor effectiveness, adjust thresholds.

What's Included

• Architectural documentation
• Infrastructure code (Terraform/Helm)
• Monitoring and alerts
• Load test case
• Team training (1 session)

Implementation Timelines

Cost is calculated individually after load audit. To get an accurate estimate and timeline, contact us.

Our Experience

Over 5 years in high-load infrastructure. We've implemented horizontal scaling for 20+ projects — from startups to enterprise (e-commerce, fintech). We use proven solutions with SLA guarantees. Want to set up autoscaling? Get a consultation — we'll evaluate your project and propose the optimal solution.

We regularly encounter a situation: "The site is not opening" at 3 a.m. — and it turns out that the VPS disk is full because nginx logs haven't been rotated for six months. Or the server went down under load on the day of an advertising campaign launch because the shared hosting had a limit of 50 concurrent connections. Setting up hosting and deployment is not about "where it's cheaper" but about what happens when something goes wrong. Our team helps avoid such incidents by designing infrastructure that accounts for real load patterns.

When to choose Vercel and Netlify?

Vercel is built for Next.js — deploy in one push, preview deployments for every PR, automatic CDN, Edge Functions, ISR without configuration. For frontend projects and JAMstack, it's the optimal choice: no operational overhead, time-to-deploy measured in minutes.

Real limitations: Vercel Serverless Functions run in us-east-1 by default (latency for Europe +80–100ms), Function timeout 300 seconds on Pro, Bandwidth 1TB/month on Pro. For heavy backend, you need workers or a separate server.

Netlify is closer to static sites and Edge Functions based on Deno Deploy. Build minutes are the main limitation on the free tier.

Why is Docker the foundation of predictable deployment?

"It works on my machine" — classic. Docker solves this through environment containerization. But a bad Dockerfile creates new problems.

A typical mistake: copying everything into the image without .dockerignore, resulting in an 800MB image instead of 80MB. node_modules inside the image weighs as much. Correct approach: multi-stage build.

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM node:20-alpine AS runner
WORKDIR /app
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./package.json
EXPOSE 3000
CMD ["npm", "start"]

Final image: 180MB instead of 1.2GB. CI build time is reduced due to layer caching — if package.json hasn't changed, the layer with npm ci is taken from cache.

Docker Compose for local development and simple production scenarios: application + PostgreSQL + Redis in one configuration. For production on a single server, it's a perfectly viable option if there's no requirement for horizontal scaling.

More about containerization — Wikipedia: Docker.

How to set up Nginx as a reverse proxy?

Nginx in front of the application is standard for VPS and dedicated servers. Main functions: SSL termination, gzip, static files, rate limiting, upstream load balancing.

A configuration often done incorrectly: worker_processes auto — number of processes equals CPU count. worker_connections 1024 — that's 1024 per worker process. With 4 CPUs and 1024 connections = 4096 concurrent connections. For a high-traffic site, you need worker_connections 4096 and set keepalive_timeout 65.

For static assets with hash in the filename:

location ~* \.(js|css|woff2|png|webp)$ {
    expires 1y;
    add_header Cache-Control "public, immutable";
}

immutable tells the browser: don't revalidate this file even on hard refresh. This only works correctly with content-hashed filenames (which Vite/webpack do by default). Documentation — Wikipedia: Nginx.

AWS: flexibility and complexity

EC2 + Auto Scaling Group — classic for horizontal scaling. AMI with pre-installed application, Launch Template, ASG with min/desired/max instances, Application Load Balancer. When CPU > 70% for 3 minutes — scale out, when CPU < 30% for 15 minutes — scale in. Health check via ALB removes unhealthy instances from rotation.

ECS Fargate — containers without managing EC2. Deploy a Docker image, specify CPU/memory (512 CPU units = 0.5 vCPU, from 512MB memory), Fargate launches it. More expensive than Lambda, but no cold start and no timeout limitations. Suitable for long-running processes, WebSocket servers, heavy workers.

RDS for PostgreSQL with Multi-AZ: automatic failover in 1–2 minutes when primary fails. Read Replicas for scaling reads. RDS Proxy for connection pooling — Lambda functions cannot hold long-term connections, the proxy buffers this.

Kubernetes: when it is justified

K8s adds significant operational complexity. Justified when: multiple teams deploy independent services, fine-grained resource allocation per service is needed, canary deployments and blue/green without downtime are required.

AWS EKS, GKE, or managed k8s from Hetzner (cheaper). Helm charts for standard services. Horizontal Pod Autoscaler based on CPU and custom metrics (RPS via Prometheus).

For most startups and medium-sized projects, Kubernetes is overkill. ECS or Fly.io provide 80% of the capabilities with 20% of the operational complexity.

Monitoring and alerting

A server without monitoring is waiting for an incident. Minimal stack: Prometheus + Grafana (or Grafana Cloud for managed), alerting on disk > 80%, memory > 85%, CPU > 90% over 5 minutes, error rate > 1%. Uptime via Better Uptime or Upptime (self-hosted).

Logs: Loki + Grafana or CloudWatch Logs Insights. Structured JSON logs (winston, pino) are mandatory — otherwise, log searching becomes a pain.

What is included in hosting setup

• Audit of current infrastructure and load profiling
• Selection of target architecture (VPS, AWS, serverless, Kubernetes)
• Setting up CI/CD pipeline (GitHub Actions, GitLab CI) with automatic deployment
• IaC via Terraform or Pulumi (infrastructure as code)
• Configuration of Nginx, SSL certificates, HTTP/2, brotli
• Monitoring and alerting (Prometheus + Grafana, PagerDuty)
• Documentation of runbooks and team training

Additionally, contact us if you need migration from current hosting or integration with external services.

Work process

1. Audit of current infrastructure (2–5 days)
2. Selection of target architecture with load and budget justification (1–3 days)
3. Setting up CI/CD pipeline (GitHub Actions, GitLab CI) (2–5 days)
4. IaC via Terraform or Pulumi (3–10 days)
5. Setting up monitoring and alerting (2–5 days)
6. Documentation of runbooks and team training (1–3 days)

Our experience — 7 years on the market, over 50 projects, guarantee of operability after deployment.

Timeline

• Basic deployment on VPS with Docker + Nginx + CI/CD: 1–2 weeks.
• Setting up AWS infrastructure with Auto Scaling, RDS, CDN: 3–6 weeks.
• Migration to EKS from scratch: 6–12 weeks.
• Setting up Vercel/Netlify for JAMstack: 3–5 days.

The cost is calculated individually depending on complexity and scope of work. Get a consultation — we'll evaluate your architecture in one day.