Setting up Spot/Preemptible Instances for Batch Jobs

Without Spot Instances, batch processing in the cloud costs 3–5 times more. Clients often overpay for on-demand resources that sit idle between tasks. As engineers, we have been using Spot and Preemptible Instances for over 5 years to reduce costs by 60–80% without compromising reliability. Below are specific methods, configurations, and proven solutions tested on dozens of projects.

Which tasks are most efficiently executed on Spot Instances?

Spot Instances are ideal for stateless batch tasks: CI/CD workers (each build isolated), image and video processing (transcoding, resizing), ML training with checkpoint-based approaches, parsing and ETL pipelines, rendering, antivirus scans, analytical queries. They are not suitable for stateful databases (risk of data loss), web servers without rapid replacement, or services with strict SLA lacking a DR strategy.

How does Spot Fleet reduce interruption probability?

The key to stability is a Spot Fleet with multiple instance types. If m5.xlarge is unavailable, the Fleet picks up m5a.xlarge or c4.xlarge. We use the capacityOptimized strategy — it selects pools with the greatest free capacity, lowering interruption probability. Example configuration:

{
  "SpotFleetRequestConfig": {
    "AllocationStrategy": "capacityOptimized",
    "TargetCapacity": 10,
    "LaunchTemplateConfigs": [
      {
        "LaunchTemplateSpecification": {"LaunchTemplateId": "lt-xxx", "Version": "1"},
        "Overrides": [
          {"InstanceType": "m5.xlarge", "WeightedCapacity": 1},
          {"InstanceType": "m5a.xlarge", "WeightedCapacity": 1},
          {"InstanceType": "m4.xlarge", "WeightedCapacity": 1},
          {"InstanceType": "c5.xlarge", "WeightedCapacity": 1}
        ]
      }
    ]
  }
}

Why is checkpointing critical?

Spot Instances can be terminated at any moment. Without checkpointing, lost progress makes their cost savings meaningless. Implementing a state-saving mechanism allows tasks to restart from the last saved point, minimizing losses. In practice, this yields up to 95% effective runtime even under frequent interruptions.

How to properly handle the Spot Interruption Notice?

AWS sends a metadata event 2 minutes in advance (more in AWS documentation). The application should poll the endpoint and upon an interruption signal, save a checkpoint. The code below shows a Python implementation with graceful exit:

import requests
import signal
import sys

def check_spot_interruption():
    """Call every 5 seconds from the worker"""
    try:
        response = requests.get(
            'http://169.254.169.254/latest/meta-data/spot/interruption-notice',
            timeout=1
        )
        if response.status_code == 200:
            return True  # Interruption imminent
    except requests.exceptions.RequestException:
        pass
    return False

class BatchWorker:
    def process_task(self, task):
        # Checkpoint every N items
        for i, item in enumerate(task.items):
            if i % 100 == 0 and check_spot_interruption():
                self.save_checkpoint(task.id, i)
                sys.exit(0)  # Graceful exit, task will be restarted

            self.process_item(item)

        task.mark_complete()

Interruption handling steps:

1. Poll the metadata endpoint every 5 seconds.
2. Upon receiving the signal, save a checkpoint (e.g., in S3 or Redis).
3. Graceful exit with code 0 so the queue (SQS) does not mark the task as failed.

You can also use AWS EventBridge for automation: event → Lambda → checkpoint saving, instance removal from pool, task return to queue.

What is Karpenter and how does it manage Spot nodes?

Karpenter (AWS) automatically selects the instance type (including Spot) and handles interruptions: upon receiving a notice, it cordons and drains the node, rescheduling pods. Example Provisioner:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: batch-workers
spec:
  requirements:
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
    - key: "node.kubernetes.io/instance-type"
      operator: In
      values: ["m5.xlarge", "m5a.xlarge", "m4.xlarge", "c5.xlarge"]
  taints:
    - key: batch
      effect: NoSchedule
  consolidation:
    enabled: true

Strategy comparison: Karpenter compared to manual Spot Fleet management reduces interruption reaction time by half due to automatic cordon and drain. Spot Fleet is simple but requires manual template management. Karpenter offers dynamic scaling and automatic recovery but is more complex to configure. Both provide 60–80% savings.

GCP Preemptible / Spot VMs: nuances

GCP Preemptible: max 24-hour lifetime, 30-second notice. Spot VMs — no 24-hour limit, only based on availability. Creation via gcloud:

gcloud compute instances create batch-worker \
  --machine-type=n2-standard-4 \
  --provisioning-model=SPOT \
  --instance-termination-action=STOP \
  --zone=us-central1-a

Checkpointing is also applied here, but with a shorter notice (30 seconds). Implementation is similar to AWS, but polling the GCP metadata server.

What's included in turnkey setup?

• Audit of current workloads and selection of suitable ones
• Design of Spot Fleet / Provisioner with multiple instance types
• Implementation of checkpointing and interruption handling (Python/Go/Java code)
• Integration with EventBridge, Lambda, queues (SQS, RabbitMQ)
• Monitoring setup (CloudWatch, Prometheus) and alerts
• Documentation and team training
• Testing with interruption simulation
• Post-launch support for 1 month

Estimated timelines

Cost is calculated individually based on your volume and complexity. Request an estimate — simply write to us. We guarantee transparent pricing and scope fixation.

Real savings in numbers

Typical savings are 60–80% compared to on-demand. Overhead for interruptions and restarts is 5–15% of time. On one ML training project using p3.2xlarge, savings reached 70%. Our clients confirm: proper checkpointing implementation pays for itself in 1–2 months. Contact us to calculate the savings for your workload. Order an audit of your batch jobs — our engineers with 5+ years of experience and AWS/GCP certifications will help reduce your cloud bill without performance loss.

We regularly encounter a situation: "The site is not opening" at 3 a.m. — and it turns out that the VPS disk is full because nginx logs haven't been rotated for six months. Or the server went down under load on the day of an advertising campaign launch because the shared hosting had a limit of 50 concurrent connections. Setting up hosting and deployment is not about "where it's cheaper" but about what happens when something goes wrong. Our team helps avoid such incidents by designing infrastructure that accounts for real load patterns.

When to choose Vercel and Netlify?

Vercel is built for Next.js — deploy in one push, preview deployments for every PR, automatic CDN, Edge Functions, ISR without configuration. For frontend projects and JAMstack, it's the optimal choice: no operational overhead, time-to-deploy measured in minutes.

Real limitations: Vercel Serverless Functions run in us-east-1 by default (latency for Europe +80–100ms), Function timeout 300 seconds on Pro, Bandwidth 1TB/month on Pro. For heavy backend, you need workers or a separate server.

Netlify is closer to static sites and Edge Functions based on Deno Deploy. Build minutes are the main limitation on the free tier.

Why is Docker the foundation of predictable deployment?

"It works on my machine" — classic. Docker solves this through environment containerization. But a bad Dockerfile creates new problems.

A typical mistake: copying everything into the image without .dockerignore, resulting in an 800MB image instead of 80MB. node_modules inside the image weighs as much. Correct approach: multi-stage build.

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM node:20-alpine AS runner
WORKDIR /app
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./package.json
EXPOSE 3000
CMD ["npm", "start"]

Final image: 180MB instead of 1.2GB. CI build time is reduced due to layer caching — if package.json hasn't changed, the layer with npm ci is taken from cache.

Docker Compose for local development and simple production scenarios: application + PostgreSQL + Redis in one configuration. For production on a single server, it's a perfectly viable option if there's no requirement for horizontal scaling.

More about containerization — Wikipedia: Docker.

How to set up Nginx as a reverse proxy?

Nginx in front of the application is standard for VPS and dedicated servers. Main functions: SSL termination, gzip, static files, rate limiting, upstream load balancing.

A configuration often done incorrectly: worker_processes auto — number of processes equals CPU count. worker_connections 1024 — that's 1024 per worker process. With 4 CPUs and 1024 connections = 4096 concurrent connections. For a high-traffic site, you need worker_connections 4096 and set keepalive_timeout 65.

For static assets with hash in the filename:

location ~* \.(js|css|woff2|png|webp)$ {
    expires 1y;
    add_header Cache-Control "public, immutable";
}

immutable tells the browser: don't revalidate this file even on hard refresh. This only works correctly with content-hashed filenames (which Vite/webpack do by default). Documentation — Wikipedia: Nginx.

AWS: flexibility and complexity

EC2 + Auto Scaling Group — classic for horizontal scaling. AMI with pre-installed application, Launch Template, ASG with min/desired/max instances, Application Load Balancer. When CPU > 70% for 3 minutes — scale out, when CPU < 30% for 15 minutes — scale in. Health check via ALB removes unhealthy instances from rotation.

ECS Fargate — containers without managing EC2. Deploy a Docker image, specify CPU/memory (512 CPU units = 0.5 vCPU, from 512MB memory), Fargate launches it. More expensive than Lambda, but no cold start and no timeout limitations. Suitable for long-running processes, WebSocket servers, heavy workers.

RDS for PostgreSQL with Multi-AZ: automatic failover in 1–2 minutes when primary fails. Read Replicas for scaling reads. RDS Proxy for connection pooling — Lambda functions cannot hold long-term connections, the proxy buffers this.

Kubernetes: when it is justified

K8s adds significant operational complexity. Justified when: multiple teams deploy independent services, fine-grained resource allocation per service is needed, canary deployments and blue/green without downtime are required.

AWS EKS, GKE, or managed k8s from Hetzner (cheaper). Helm charts for standard services. Horizontal Pod Autoscaler based on CPU and custom metrics (RPS via Prometheus).

For most startups and medium-sized projects, Kubernetes is overkill. ECS or Fly.io provide 80% of the capabilities with 20% of the operational complexity.

Monitoring and alerting

A server without monitoring is waiting for an incident. Minimal stack: Prometheus + Grafana (or Grafana Cloud for managed), alerting on disk > 80%, memory > 85%, CPU > 90% over 5 minutes, error rate > 1%. Uptime via Better Uptime or Upptime (self-hosted).

Logs: Loki + Grafana or CloudWatch Logs Insights. Structured JSON logs (winston, pino) are mandatory — otherwise, log searching becomes a pain.

What is included in hosting setup

• Audit of current infrastructure and load profiling
• Selection of target architecture (VPS, AWS, serverless, Kubernetes)
• Setting up CI/CD pipeline (GitHub Actions, GitLab CI) with automatic deployment
• IaC via Terraform or Pulumi (infrastructure as code)
• Configuration of Nginx, SSL certificates, HTTP/2, brotli
• Monitoring and alerting (Prometheus + Grafana, PagerDuty)
• Documentation of runbooks and team training

Additionally, contact us if you need migration from current hosting or integration with external services.

Work process

1. Audit of current infrastructure (2–5 days)
2. Selection of target architecture with load and budget justification (1–3 days)
3. Setting up CI/CD pipeline (GitHub Actions, GitLab CI) (2–5 days)
4. IaC via Terraform or Pulumi (3–10 days)
5. Setting up monitoring and alerting (2–5 days)
6. Documentation of runbooks and team training (1–3 days)

Our experience — 7 years on the market, over 50 projects, guarantee of operability after deployment.

Timeline

• Basic deployment on VPS with Docker + Nginx + CI/CD: 1–2 weeks.
• Setting up AWS infrastructure with Auto Scaling, RDS, CDN: 3–6 weeks.
• Migration to EKS from scratch: 6–12 weeks.
• Setting up Vercel/Netlify for JAMstack: 3–5 days.

The cost is calculated individually depending on complexity and scope of work. Get a consultation — we'll evaluate your architecture in one day.