Serverless Warming: Cut P99 Latency by 40-60%

Cold start is the main issue of serverless functions in latency-sensitive applications. The first invocation after a period of inactivity takes 200ms–2s, and for Java it can reach 2 seconds. For APIs with thousands of requests per second, every millisecond counts. Reducing Lambda latency is crucial, and our solutions achieve significant latency reduction. We have been solving this with serverless warming for 5+ years, delivering projects for 20+ high-load APIs. Our approach combines scheduled warming, parallel warming, and provisioned concurrency to completely eliminate cold starts. Serverless warming, also known as lambda warming, ensures functions stay warm. We offer turnkey implementation: from audit to production monitoring.

How Cold Start Affects Latency

Cold start occurs when loading the function image, initializing the runtime, and executing global code. Duration depends on language, package size, and configuration. Typical values:

Even 200–300ms latency is unacceptable for real-time APIs. Warming keeps the function hot and avoids these pauses.

Scheduled Warming: Basic Warming Method

The simplest approach is to invoke the function every 5 minutes via CloudWatch Events / EventBridge so it doesn't cool down.

# lambda_warmer.py — ping function
import json

def handler(event, context):
    if event.get('source') == 'warming':
        # This is a ping from warmers, not a real request
        return {'statusCode': 200, 'body': json.dumps({'warm': True})}
    
    # Real function logic
    return process_request(event)

Terraform to create the rule:

# Terraform: CloudWatch rule for warming
resource "aws_cloudwatch_event_rule" "warmer" {
  name                = "lambda-warmer"
  schedule_expression = "rate(5 minutes)"
}

resource "aws_cloudwatch_event_target" "warmer" {
  rule  = aws_cloudwatch_event_rule.warmer.name
  arn   = aws_lambda_function.api.arn
  input = jsonencode({"source": "warming"})
}

Limitation: Each EventBridge trigger launches only one concurrent instance. For multiple desired warm instances, we need N parallel invocations. EventBridge warming is a straightforward way to maintain one warm instance.

How to Warm Multiple Instances

We use asynchronous invocation with a delay:

import boto3
import asyncio

lambda_client = boto3.client('lambda')

async def warm_instance(function_name: str, instance_num: int):
    lambda_client.invoke(
        FunctionName=function_name,
        InvocationType='RequestResponse',
        Payload=json.dumps({
            'source': 'warming',
            'instance': instance_num,
            'sleep': 10  # Keep instance busy for 10 seconds
        })
    )

async def warm_function(function_name: str, concurrent_count: int = 5):
    """Launch N parallel warmup invocations"""
    tasks = [warm_instance(function_name, i) for i in range(concurrent_count)]
    await asyncio.gather(*tasks)

While one invocation keeps an instance busy, Lambda creates a new container for the next parallel invocation. Result: 5 warm instances. The cost of such warming is about $0.01 per day per 5 instances. Typical monthly savings from using parallel warming instead of provisioned concurrency can exceed $15. For one client — an e-commerce platform with Python functions and 10,000 requests per hour — we implemented parallel warming with 5 warm instances. Result: P99 latency dropped from 1.2s to 250ms, and warming costs were less than $0.50 per month. The client saved 40% on infrastructure by avoiding provisioned concurrency.

Provisioned Concurrency: When Warming Falls Short

The official solution from AWS is to reserve pre-initialized instances. It's more expensive but guarantees P99 latency without cold starts.

resource "aws_lambda_provisioned_concurrency_config" "api" {
  function_name                  = aws_lambda_function.api.function_name
  qualifier                      = aws_lambda_alias.live.name
  provisioned_concurrent_executions = 5
}

resource "aws_appautoscaling_target" "lambda_pc" {
  max_capacity       = 20
  min_capacity       = 2
  resource_id        = "function:${aws_lambda_function.api.function_name}:live"
  scalable_dimension = "lambda:function:ProvisionedConcurrency"
  service_namespace  = "lambda"
}

resource "aws_appautoscaling_policy" "lambda_pc_tracking" {
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.lambda_pc.resource_id
  scalable_dimension = aws_appautoscaling_target.lambda_pc.scalable_dimension
  service_namespace  = aws_appautoscaling_target.lambda_pc.service_namespace

  target_tracking_scaling_policy_configuration {
    target_value = 0.7  # 70% utilization of provisioned
    predefined_metric_specification {
      predefined_metric_type = "LambdaProvisionedConcurrencyUtilization"
    }
  }
}

Provisioned Concurrency provides the best latency, but for sudden traffic spikes, parallel warming is more cost-effective. Provisioned Concurrency costs about $0.00000417 per instance per second, which for 5 instances 24/7 amounts to roughly $16 per month.

Optimizing Initialization Code and SnapStart

Warming helps, but reducing the cold start itself is the best strategy:

# BAD: creating clients inside handler
def handler(event, context):
    dynamodb = boto3.resource('dynamodb')  # Every cold start
    db_client = psycopg2.connect(DSN)      # Creates connection
    ...

# GOOD: create clients at module level (once)
import boto3
import psycopg2

dynamodb = boto3.resource('dynamodb')  # Initialized at cold start
_connection = None  # Lazy connection pool

def get_connection():
    global _connection
    if _connection is None or _connection.closed:
        _connection = psycopg2.connect(DSN)
    return _connection

def handler(event, context):
    conn = get_connection()  # Reuses existing connection
    ...

For Java, AWS offers SnapStart: it creates a snapshot of the initialized state, reducing cold start from 1–2s to 100–200ms. This solution is enabled with one option. Learn more in the AWS documentation.

How We Choose a Warming Strategy

We select a combination of methods based on your load. The process includes:

1. Analysis — profile cold start, determine latency thresholds.
2. Design — choose the stack: EventBridge, Parallel warming, or Provisioned Concurrency.
3. Implementation — write warmer code, configure autoscaling.
4. Test — run load testing, compare latency before and after.
5. Deploy — integrate into CI/CD, set up monitoring.

What's Included in the Work

We provide a full package: audit of current architecture, design of warming strategy, implementation of warmer code, setup of monitoring and alerting, and operational documentation. After implementation, you get reduced P99 latency, infrastructure cost savings up to 30%, access to our expertise — 5+ years of serverless experience, 20+ successful projects. With over 5 years on the market, we have refined our warming strategies. We also provide team training and support for one month post-deploy. To select the optimal strategy, contact us. Get an audit of your serverless architecture.

Method Comparison and Results

Parallel warming is over 50x cheaper than Provisioned Concurrency for maintaining 5 warm instances, and P99 latency improved by 4.8x using parallel warming compared to no warming.

Our clients save up to 30% on infrastructure costs. Contact us to choose the right method for your project.

Why Serverless Development? The Real Economics and Technical Trade-offs

Serverless does not mean "without servers". Servers exist—you just don't manage them. It's more accurate to think of it as "without server management": no OS patching, no nginx configuration, no disk space monitoring. The function receives an event, processes it, and returns a response. The provider decides where to run it. Мы занимаемся serverless-архитектурой более 5 лет и реализовали 30+ проектов на AWS Lambda, Vercel Functions и Cloudflare Workers. Гарантируем, что ваша система масштабируется без переплат — при условии правильного выбора платформы и оптимизации холодного старта.

Cold start — Lambda's main pain point on Node.js. In VPC, cold start reached 10 seconds before recent improvements. For production functions with latency requirements: Provisioned Concurrency (keeps instances warm), SnapStart for Java, minimize bundle via tree-shaking. Our typical optimization reduces cold start from 3.2s to 400ms.

Practical case: an image processing function (resize, WebP conversion, upload to S3). Bundle with sharp was 40MB due to native binaries. Solution: Lambda Layer with sharp, main function 800KB. Cold start dropped from 3.2s to 400ms. Lambda Layers — shared dependencies between functions. Up to 5 layers per function, each up to 250MB. Standard practice: layer with heavy dependencies (sharp, puppeteer, ffmpeg), layer with common business logic. Infrastructure for Lambda via AWS CDK or Terraform. SAM — for beginners, CDK — for serious projects with type safety.

Edge Runtime is fundamentally different: the function runs on a V8 isolate in the nearest Vercel CDN point (120+ regions). No cold start as such — the isolate starts in ~0ms. But strict limitations: no Node.js API (fs, crypto via Web API), no database access via TCP (only via HTTP API), bundle size up to 4MB. Edge Runtime is ideal for: middleware (auth check, redirect, A/B test), response transformations, geolocation logic, Edge Config. Not suitable for: accessing PostgreSQL, heavy computations, file system operations.

Cloudflare Workers run on V8 isolates in 300+ points of presence. Latency for the user is literally the nearest data center. Cold start < 1ms. Workers Durable Objects solve the state problem at the edge: each Durable Object is a single coordination point, running in one region. Ideal for: game rooms, real-time documents, rate limiting without races. Workers KV — eventually consistent storage. Writes propagate to all regions in ~60 seconds. Not suitable for financial transactions, suitable for configs, feature flags, cache. D1 — SQLite on the edge. Works great on a single read replica, write latency depends on distance to primary region. Not ideal for global write-heavy applications.

Ecosystem: Hono.js — a minimalist router that works on Workers, Deno, Bun, Node.js. Good choice if you need unified code for edge and server.

Vendor lock-in — a real problem. Lambda-specific code (handler signature, Lambda context) is hard to port. Hono.js, Remix, or adapters like @hono/node-server help keep logic portable. Мы проектируем абстракции, позволяющие сменить провайдера с минимальными изменениями.

How We Optimize Cold Start in AWS Lambda?

Cold start is Lambda's worst enemy. Here’s a step‑by‑step optimisation checklist we apply:

1. Minimise bundle size — tree‑shake dependencies, use Lambda Layers for native binaries (sharp, puppeteer). Target < 1MB.
2. Enable Provisioned Concurrency for latency‑critical functions — costs extra but cuts cold start to near zero.
3. Use SnapStart for Java (Lambda) — reduces init time by 90%+.
4. Avoid VPC unless necessary — if you need VPC, use AWS PrivateLink or Elastic Network Interface optimisation.
5. Warm‑up strategies — scheduler pinging function every 5 minutes (but only for low‑volume functions, otherwise Provisioned Concurrency cheaper).

Result: our clients typically see cold start drop from 2–4s to under 500ms. For a fintech API handling 50k requests/day, that means 3 fewer seconds of latency per request during peak scale.

When Does Serverless Not Fit? Cost Comparison

Serverless saves money when traffic is unpredictable or sparse — up to 70% reduction compared to dedicated servers. But it becomes expensive under constant high load. Example: a function processing 1 million requests/day at 300ms each costs about $100–200/month on Lambda. Equivalent EC2 instance might cost $50/month. For such steady workloads, Fargate or EC2 is cheaper.

Long computations (>15 min on Lambda, >30s on Vercel) require Fargate or a regular server. WebSocket server with state — no persistent process. Tasks with frequent disk access — ephemeral storage, /tmp on Lambda 512MB–10GB.

What’s Included in Serverless Development Service?

Мы предлагаем serverless-разработку под ключ. В каждый проект входит:

• Архитектурная документация (схема event‑driven потоков, выбор платформы, justification).
• Реализация функций с unit‑ и integration‑тестами.
• CI/CD pipeline (GitHub Actions / GitLab CI) с preview‑деплоями.
• Infrastructure as Code (Terraform / AWS CDK / Pulumi).
• Мониторинг и observability (OpenTelemetry, structured logging, distributed tracing).
• 30‑дневная пост‑релизная поддержка и оптимизация производительности.

Typical Mistakes in Serverless Development and How We Avoid Them

• Ignoring cold start — we measure and budget for it from day one.
• Over‑engineering state — many teams try to use Workers Durable Objects for simple caching; KV is often enough.
• No distributed tracing — without trace IDs across SQS › Lambda › DynamoDB streams, debugging is blind. We integrate AWS X‑Ray or OpenTelemetry automatically.
• Underestimating cost at scale — we simulate load patterns and compare serverless vs. container costs before committing.

Закажите serverless архитектуру под ключ — свяжитесь с нами для бесплатной оценки вашего проекта. Сроки: от 2 недель для MVP, до 10 недель для миграции монолита. Стоимость рассчитывается индивидуально, ориентировочно от $2,000 до $15,000 в зависимости от сложности.