Canary deployment setup for web application

Configuring Canary Deployment for Web Applications

Canary deployment is gradually switching traffic to a new version: first 1–5% of users, then 10%, 25%, 50%, and finally 100%. It allows you to detect problems on real traffic before full migration.

Principle

v1.0 (95%) ←── 95% of requests
                              ← Load Balancer
v1.1 (5%)  ←──  5% of requests

If metrics (error rate, latency, conversion) are normal → increase percentage. If degradation → rollback to 0%.

Nginx split_clients

# /etc/nginx/nginx.conf
split_clients "${remote_addr}${http_user_agent}" $upstream_pool {
    5%   canary;     # 5% → new version
    *    stable;     # 95% → old version
}

upstream stable {
    server 10.0.0.10:8080;
}

upstream canary {
    server 10.0.0.11:8080;  # new version
}

server {
    location / {
        proxy_pass http://$upstream_pool;
    }
}

To change percentage — edit config and reload Nginx.

Canary via Cookie (sticky routing)

# User always goes to the same version
map $cookie_canary $upstream_canary {
    "1"  canary;
    default stable;
}

# Or force enable for testers
map $http_x_canary_override $upstream_override {
    "true" canary;
    default $upstream_canary;
}

Kubernetes + NGINX Ingress

# stable-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-stable
---
# canary-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-canary
---
# canary-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "5"   # 5% traffic
spec:
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            backend:
              service:
                name: myapp-canary-svc
                port: { number: 80 }

# Increase to 25%
kubectl annotate ingress myapp-canary \
    nginx.ingress.kubernetes.io/canary-weight=25 --overwrite

# Full migration → update stable deployment
kubectl set image deployment/myapp-stable myapp=registry/myapp:v1.1.0

# Remove canary
kubectl delete ingress myapp-canary
kubectl delete deployment myapp-canary

AWS: Weighted Target Groups

import boto3

elbv2 = boto3.client('elbv2')

def set_canary_weight(listener_arn: str, stable_tg: str, canary_tg: str, canary_weight: int):
    """stable_weight + canary_weight should equal 100"""
    stable_weight = 100 - canary_weight

    elbv2.modify_listener(
        ListenerArn=listener_arn,
        DefaultActions=[{
            'Type': 'forward',
            'ForwardConfig': {
                'TargetGroups': [
                    {'TargetGroupArn': stable_tg, 'Weight': stable_weight},
                    {'TargetGroupArn': canary_tg,  'Weight': canary_weight},
                ],
                'TargetGroupStickinessConfig': {
                    'Enabled': True,
                    'DurationSeconds': 3600,  # stickiness 1 hour
                }
            }
        }]
    )

Automatic Canary with Metrics Analysis

# canary-rollout.py
import time
import boto3
import requests

PROMETHEUS_URL = "http://prometheus:9090"

def get_error_rate(version: str, duration: str = "5m") -> float:
    query = f'rate(http_requests_total{{version="{version}",status=~"5.."}}[{duration}]) / rate(http_requests_total{{version="{version}"}}[{duration}])'
    r = requests.get(f"{PROMETHEUS_URL}/api/v1/query", params={"query": query})
    result = r.json()["data"]["result"]
    return float(result[0]["value"][1]) if result else 0.0

def progressive_rollout():
    steps = [5, 10, 25, 50, 75, 100]
    canary_weight = 0

    for target_weight in steps:
        print(f"Setting canary weight to {target_weight}%")
        set_canary_weight(LISTENER_ARN, STABLE_TG, CANARY_TG, target_weight)

        # Wait and check metrics
        time.sleep(300)  # 5 minutes per step

        error_rate = get_error_rate("canary")
        print(f"Canary error rate: {error_rate:.2%}")

        if error_rate > 0.01:  # >1% errors
            print(f"Error rate too high ({error_rate:.2%}), rolling back!")
            set_canary_weight(LISTENER_ARN, STABLE_TG, CANARY_TG, 0)
            return False

    print("Canary rollout complete!")
    return True

Implementation Timeline

• Nginx canary on VPS: 1–2 days
• Kubernetes NGINX Ingress canary: 2–3 days
• Automatic rollout with metrics: 3–5 days