Continuous Load Testing in CI/CD Pipeline

Continuous Load Testing in CI/CD Pipeline

With every deploy to production, the risk of introducing performance regression is high. One suboptimal SQL query or a forgotten N+1 problem can increase response time by 50%, and users will switch to competitors. We integrate automated load tests directly into your CI/CD pipeline to catch such regressions before they reach master. The result: you are confident that every commit does not break SLA for latency and throughput.

We configure smoke tests with threshold values that turn the pipeline into a gatekeeper: if p95 latency exceeds 500ms or error rate exceeds 1%, the pipeline fails and the developer receives a notification. This is not a replacement for full-scale load testing, but a quick safeguard.

Problems We Solve

• Performance regression after deploy — even a micro-change in code can slow down a critical endpoint. Without automated tests, you learn about the problem only after user complaints or monitoring alerts. We set up baseline comparison: each test runs against the stable branch, and if deviation >20%, the pipeline is blocked.
• Inefficient queries and bottlenecks — our scripts emulate typical user scenarios (listing, creating a post, searching). If a query starts to take longer, we see it on metric charts.
• Lack of performance-first culture — developers often don't think about performance during code review. Continuous Load Testing makes performance visible: each PR is accompanied by a comment with test results (p95, error rate).

Tools and Their Place in CI

k6 is the best choice for CI: JS scripts, built-in statistics, threshold-based pass/fail, native integration with GitHub Actions and GitLab CI. More at k6 documentation.

Artillery — YAML configuration, convenient for describing scenarios without code.

Gatling — Scala/Java, detailed HTML reports, convenient for Java teams.

Basic k6 Script

// tests/performance/api-smoke.js
import http from 'k6/http'
import { check, sleep } from 'k6'
import { Rate, Trend } from 'k6/metrics'

// Custom metrics
const errorRate = new Rate('errors')
const postCreateDuration = new Trend('post_create_duration')

export const options = {
  // Load profile for CI: fast, non-destructive
  stages: [
    { duration: '30s', target: 10 },  // warm-up
    { duration: '1m',  target: 10 },  // sustained load
    { duration: '10s', target: 0 },   // cool-down
  ],

  // Pipeline will break if thresholds are not met
  thresholds: {
    http_req_duration: [
      'p(95)<500',   // p95 < 500ms
      'p(99)<1000',  // p99 < 1000ms
    ],
    errors: ['rate<0.01'],            // errors < 1%
    http_req_failed: ['rate<0.01'],   // HTTP errors < 1%
    post_create_duration: ['p(95)<800'],
  }
}

const BASE_URL = __ENV.BASE_URL || 'http://localhost:3000'
const AUTH_TOKEN = __ENV.AUTH_TOKEN

export function setup() {
  // Once: get token or prepare data
  const res = http.post(`${BASE_URL}/api/auth/login`, JSON.stringify({
    email: '[email protected]',
    password: 'testpassword'
  }), { headers: { 'Content-Type': 'application/json' } })

  return { token: res.json('token') }
}

export default function(data) {
  const headers = {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${data.token || AUTH_TOKEN}`
  }

  // Scenario 1: list posts (70% traffic)
  const postsList = http.get(`${BASE_URL}/api/posts?limit=20`, { headers })
  check(postsList, {
    'posts list: status 200': (r) => r.status === 200,
    'posts list: has items': (r) => r.json('data').length > 0
  })
  errorRate.add(postsList.status !== 200)

  sleep(Math.random() * 0.5)  // random pause 0-500ms

  // Scenario 2: create post (20% traffic)
  if (Math.random() < 0.2) {
    const start = Date.now()
    const createPost = http.post(`${BASE_URL}/api/posts`, JSON.stringify({
      title: `Test post ${Date.now()}`,
      content: 'Load test content'
    }), { headers })

    postCreateDuration.add(Date.now() - start)
    check(createPost, {
      'create post: status 201': (r) => r.status === 201,
    })
    errorRate.add(createPost.status !== 201)
  }

  sleep(0.3)
}

How to Set Performance Thresholds in k6?

Thresholds are the key mechanism for automatic pipeline failure. In the script options, we define acceptable boundaries: for example, http_req_duration: ['p(95)<500', 'p(99)<1000']. This means 95% of requests must be faster than 500ms, and 99% faster than 1 second. If a threshold is violated, the test finishes with an error, and CI stops deployment.

We also use custom metrics for specific business operations (e.g., post creation time) and set separate thresholds on them. That way, you know for sure that critical functionality is not degrading.

GitHub Actions Integration

# .github/workflows/performance.yml
name: Performance Tests

on:
  push:
    branches: [main, staging]
  pull_request:
    branches: [main]

jobs:
  performance:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_DB: testdb
          POSTGRES_PASSWORD: testpass
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s

    steps:
      - uses: actions/checkout@v4

      - name: Start application
        run: |
          docker compose -f docker-compose.test.yml up -d api
          npx wait-on http://localhost:3000/health --timeout 60000

      - name: Run k6 smoke test
        uses: grafana/[email protected]
        with:
          filename: tests/performance/api-smoke.js
          flags: --out json=results.json
        env:
          BASE_URL: http://localhost:3000
          K6_PROMETHEUS_RW_SERVER_URL: ${{ secrets.PROMETHEUS_URL }}

      - name: Parse results
        if: always()
        run: |
          # Show summary in PR comment
          jq -r '.metrics | {
            p95: .http_req_duration["p(95)"],
            p99: .http_req_duration["p(99)"],
            errors: .http_req_failed.rate
          }' results.json

      - name: Comment PR with results
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs')
            const results = JSON.parse(fs.readFileSync('results.json'))
            const p95 = results.metrics.http_req_duration['p(95)'].toFixed(0)
            const errorRate = (results.metrics.http_req_failed.rate * 100).toFixed(2)

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: `## Performance Test Results\n\n| Metric | Value | Threshold |\n|--------|-------|-----------|\n| p95 latency | ${p95}ms | <500ms |\n| Error rate | ${errorRate}% | <1% |`
            })

Baseline Comparison Between Deployments

#!/bin/bash
# scripts/compare-performance.sh

CURRENT_BRANCH=$(git branch --show-current)
BASELINE_BRANCH="main"

# Test current code
k6 run --out json=current.json tests/performance/api-smoke.js

# Switch to baseline
git stash
git checkout $BASELINE_BRANCH
docker compose up -d --build api
sleep 10
k6 run --out json=baseline.json tests/performance/api-smoke.js

# Comparison
node - <<'EOF'
const current = require('./current.json')
const baseline = require('./baseline.json')

const metrics = ['http_req_duration']
for (const m of metrics) {
  const cp95 = current.metrics[m]['p(95)']
  const bp95 = baseline.metrics[m]['p(95)']
  const delta = ((cp95 - bp95) / bp95 * 100).toFixed(1)

  if (cp95 > bp95 * 1.2) {  // regression > 20%
    console.error(`REGRESSION: ${m} p95 degraded by ${delta}%`)
    process.exit(1)
  }
  console.log(`${m} p95: ${cp95}ms vs ${bp95}ms baseline (${delta}%)`)
}
EOF

# Return to current branch
git checkout $CURRENT_BRANCH
git stash pop

Artillery for Scenario Description

# tests/performance/user-journey.yml
config:
  target: "{{ $processEnvironment.BASE_URL }}"
  phases:
    - duration: 60
      arrivalRate: 5
      rampTo: 20
      name: "Ramp up"
    - duration: 120
      arrivalRate: 20
      name: "Sustained load"

  ensure:
    thresholds:
      - http.response_time.p95: 500
      - http.request_rate: 15

scenarios:
  - name: "Browse and purchase"
    weight: 70
    flow:
      - get:
          url: "/api/products"
          expect:
            - statusCode: 200
      - post:
          url: "/api/cart"
          json:
            productId: "{{ $randomInt(1, 100) }}"
            quantity: 1

  - name: "Search only"
    weight: 30
    flow:
      - get:
          url: "/api/search?q={{ $randomString(5) }}"

Load Testing Tool Comparison

Why Use k6 for CI?

k6 has built-in CI/CD support: it works as a CLI tool, requires no GUI, and results can be output to JSON for further processing. Unlike Gatling (requires Scala/Java and HTML report generation) or Artillery (YAML configs, but fewer metrics), k6 provides flexible metrics and simple integration with GitHub Actions via ready-made actions. For teams already using JavaScript, the entry threshold is minimal.

Process of Work

1. Analysis — identify critical endpoints and user scenarios.
2. Design — develop load scenarios, configure load profiles (ramp-up, sustained).
3. Implementation — write scripts (k6, Artillery, or Gatling), integrate into CI.
4. Testing — run smoke tests on staging, adjust thresholds.
5. Deployment — include tests in pipeline, configure automatic PR comments.

Timelines and What's Included

Setting up a basic set of k6 smoke tests with thresholds and CI integration takes from 1 to 2 business days. If baseline comparison and custom metrics are needed, we extend to 3-5 days. The scope includes:

• Load test scripts (2-3 scenarios)
• Thresholds configuration
• Integration with GitHub Actions or GitLab CI (YAML pipeline)
• PR comment with test results
• Documentation for running and maintenance
• Team consultation on interpreting results

We guarantee that tests will not interfere with the main development process: they run in parallel and take no more than 5 minutes.

Why Choose Us

• Over 5 years of experience in load testing and CI/CD
• 50+ successful projects — from startups to enterprise
• We use only proven tools (k6, Grafana, Prometheus)
• We provide a guarantee on correct test operation for a month after implementation

Contact us to discuss your project: get a consultation on integrating Continuous Load Testing into your pipeline.

Why are unit tests important but not a panacea?

A bug found by a unit test costs minutes to fix. The same bug in production costs hours of incident response, compensations, and lost trust. In an online store project, a discount calculation error passed manual testing, went to production, and processed 37 orders at zero price in 4 hours. An automated test for edge cases would have caught it on the first push. With 7+ years in web application testing and over 200 projects delivered, we’ve seen this pattern repeat across industries.

Jest is the standard for JavaScript/TypeScript, but unit tests are justified only where there is isolated logic: transformation functions, validators, business rules, utilities. Testing React components with Jest + Testing Library is correct for behavioral tests: "button appears after loading", "form shows error on empty email". Snapshot tests (toMatchSnapshot) are a trap: they break on any layout change and become noise that developers update without looking. Code coverage is a poor quality metric: 80% coverage can be achieved with tests that check nothing. Coverage shows that code executed, not that it works correctly.

Vitest as an alternative to Jest for Vite projects: 10–20x faster due to native ES modules without Babel transformation. For monorepos with thousands of tests, the speed difference is noticeable. Wikipedia on unit testing describes the theoretical foundation — we apply it with real CI pipelines.

How to set up E2E tests that are not flaky?

Playwright outperforms Cypress on key parameters: native multi-tab, multi-origin, iframe support; parallel execution at test level; WebKit, Firefox, Chromium out of the box; no iframe for the app — tests run in a real browser.

Playwright codegen records actions and generates a test — a good starting point, but generated code needs refactoring. Locators by text content are fragile: getByRole('button', { name: 'Place order' }) is more robust than locator('.btn-primary').

Page Object Model is the standard for organizing E2E tests. Each page is a separate class with methods instead of direct locators. When a button moves from header to sidebar — change in one place, not across all tests.

Flaky tests typically arise from race conditions between request and render, animations without wait, and dependency on external APIs. Solution: page.waitForResponse() instead of page.waitForTimeout(), mocking external APIs via page.route().

// Bad
await page.click('#submit');
await page.waitForTimeout(2000);
await expect(page.locator('.success')).toBeVisible();

// Good
await page.click('#submit');
await page.waitForResponse(resp =>
  resp.url().includes('/api/orders') && resp.status() === 201
);
await expect(page.getByRole('alert', { name: /order created/i })).toBeVisible();

Our engineers guarantee test stability in CI. Playwright’s official documentation covers all API details — we use it daily on projects with millions of users.

How do Core Web Vitals affect ranking?

Google uses Core Web Vitals in ranking. Lighthouse CLI in CI pipeline: on every deploy we check that LCP < 2.5s, CLS < 0.1, INP < 200ms. Google Chrome study: 53% of users leave a site if it takes longer than 3 seconds to load — our tests prevent such losses.

Real problems that Lighthouse finds:

• Hero image without width/height attributes: CLS 0.35 on load.
• JavaScript bundle 2.1MB synchronously blocking parsing: INP 450ms.
• Fonts without font-display: swap: invisible text until font loads (FOIT).
• Unoptimized hero image 4MB: LCP 8.2s.

Lighthouse CI (lhci) saves metric history and posts a comment to PR with degradation. For one e‑commerce client, optimizing these metrics improved conversion by 18% and reduced server costs by $12k annually.

What does load testing solve?

k6 is a load testing tool with a JavaScript API. Scenarios are written as code, versioned in git, run in CI. Three main scenarios:

• Spike test — sharp load increase: 0 → 1000 users in 30 seconds. Simulates a campaign launch. Shows system's ability to handle spikes.
• Soak test — stable load for 2–4 hours. Detects memory leaks, connection pool exhaustion, performance degradation.
• Stress test — load above expected (150–200% of peak). Shows breaking point and graceful degradation.

Thresholds:

thresholds: {
  http_req_duration: ['p95<500', 'p99<1000'],
  http_req_failed: ['rate<0.01'],
}

p95 < 500ms means 95% of requests respond faster than half a second. If threshold is not met, k6 exits with error code, CI pipeline fails.

In one online store project, we detected API degradation at the 4th hour of the test: p95 increased from 200ms to 2s due to connection leaks. After optimization, the client saved $15k per year on incident response and extra infrastructure.

Testing pyramid in a project

What does the work include?

• Audit of current coverage and identification of critical user flows.
• Writing unit tests for key business logic, integration tests for API, E2E for user scenarios.
• Setting up parallel execution in CI (sharded workers for Playwright).
• Load testing with report and recommendations.
• Test case documentation, training your team on test practices.
• 1-month warranty support after implementation.
• Delivery of all test artefacts (code, CI configs, run histories).

How do we work?

1. Analysis — audit of current testing, identification of weak spots, priority setting.
2. Design — tool selection, test plan writing, approval.
3. Implementation — writing tests, CI integration.
4. Testing — running all levels, result analysis, bug fixing.
5. Deployment — going live, metric monitoring, team training.

Timeline

Setting up a full test pipeline (Jest + Playwright + k6 + Lighthouse CI) from scratch: 2–4 weeks. E2E test coverage of an existing project (20–30 scenarios): 3–6 weeks. Load testing with report and recommendations: 1–2 weeks. Cost calculated individually after audit.

Ready to discuss your project? Leave a request — we will audit your current web application testing for free and propose a plan that can save up to 60% on incident costs. Get a consultation on web application testing — contact us today.