Improve API Performance: Gateway Caching with Kong, Nginx, AWS

Imagine: your API handles 10,000 requests per second, the backend barely copes, and half the responses return the same data. Caching at the gateway level solves this in a couple of hours without changing service code. Recently, on a project with 50 microservices, we implemented caching on Kong, and the database load dropped by 80%, while the average response time decreased from 500 ms to 50 ms. We integrate caching on the gateway (Kong, Nginx, AWS) and reduce backend load by 30–70%. After configuration, hit rate reaches 95%. This article is based on practice backed by 10+ years of experience on 40+ projects.

Why caching on gateway is better than on the backend?

Cache at the gateway level intercepts the request before it reaches the service. The backend receives no load for cached requests. On the backend, caching often complicates logic and requires modifications to each service. The gateway, on the other hand, is a single configuration point: add a plugin or nginx directive, and it's done. In Kong, the proxy-cache plugin is enabled globally or for specific routes with a single line of configuration. Infrastructure savings can reach 40% — clients typically save $5,000–$15,000 per month on infrastructure costs.

Which endpoints to cache and which not?

Safe to cache:

• GET requests with public data (catalog, directories, articles)
• Responses with Cache-Control: public, max-age=N headers from upstream
• Endpoints that do not depend on session or authorization

Do not cache:

• Any POST, PUT, PATCH, DELETE
• Requests with Authorization: Bearer ... (except user-based isolation)
• Responses with status 4xx/5xx (except deliberate 404 caching)
• Personal data

Setting up caching in Kong, Nginx, and AWS

Below are working configs for three popular gateways. All are battle-tested.

Kong Gateway: proxy-cache plugin

plugins:
  - name: proxy-cache
    config:
      response_code: [200, 301, 404]
      request_method: [GET, HEAD]
      content_type:
        - application/json
        - application/json; charset=utf-8
      cache_ttl: 300
      strategy: memory
      memory:
        dictionary_name: kong_db_cache
# For production replace strategy with redis:
#      strategy: redis
#      redis:
#        host: redis.internal
#        port: 6379
#        timeout: 2000
#        database: 0
#        password: ${REDIS_PASSWORD}

Kong adds the X-Cache-Status header with values Hit, Miss, Bypass, Refresh. This helps with debugging.

Cache invalidation via Admin API:

# Remove a specific key
curl -X DELETE http://kong-admin:8001/proxy-cache/caches/{cache_key}

# Clear entire cache
curl -X DELETE http://kong-admin:8001/proxy-cache/

AWS API Gateway + ElastiCache

For REST API, native caching is enabled like this:

{
  "cacheClusterEnabled": true,
  "cacheClusterSize": "0.5",
  "methodSettings": {
    "GET /products": {
      "cachingEnabled": true,
      "cacheTtlInSeconds": 300,
      "cacheDataEncrypted": false,
      "requireAuthorizationForCacheControl": false
    }
  }
}

Terraform code:

resource "aws_api_gateway_stage" "main" {
  deployment_id        = aws_api_gateway_deployment.main.id
  rest_api_id          = aws_api_gateway_rest_api.main.id
  stage_name           = "v1"
  cache_cluster_enabled = true
  cache_cluster_size    = "0.5"

  method_settings {
    resource_path = "/products"
    http_method   = "GET"

    settings {
      caching_enabled      = true
      cache_ttl_in_seconds = 300
    }
  }
}

Invalidation via request with Cache-Control: max-age=0 header (requires execute-api:InvalidateCache permission).

Nginx: proxy_cache

proxy_cache_path /var/cache/nginx/api
  levels=1:2
  keys_zone=api_cache:10m
  max_size=1g
  inactive=10m
  use_temp_path=off;

server {
  location /api/v1/catalog/ {
    proxy_cache api_cache;
    proxy_cache_valid 200 5m;
    proxy_cache_valid 404 1m;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503;
    proxy_cache_background_update on;
    proxy_cache_lock on;

    # Cache key — without authorization parameters
    proxy_cache_key "$scheme$request_method$host$uri$is_args$args";

    # Do not cache if client sends session cookie
    proxy_cache_bypass $cookie_session_id;
    proxy_no_cache $cookie_session_id;

    add_header X-Cache-Status $upstream_cache_status;
    proxy_pass http://backend;
  }
}

proxy_cache_use_stale updating + proxy_cache_background_update on implements the stale-while-revalidate pattern: the user instantly receives the old response while the update happens in the background. Critical for high-load endpoints.

Performance comparison: Nginx on a gateway handles up to 100,000 RPS, which is 2x more than Kong in memory mode (50,000 RPS). AWS REST API handles up to 10,000 RPS depending on cluster size.

Avoiding cache stampede

When a popular key's TTL expires, dozens of workers simultaneously hit the upstream — this is a cache stampede. Three mitigation approaches:

• Mutex/lock: only one request updates the cache, others wait (e.g., proxy_cache_lock on in Nginx).
• Probabilistic early expiration: update the cache slightly before TTL with probability increasing toward expiration time.
• Stale-while-revalidate: serve the old response while the update proceeds (implemented above).

Invalidation is the headache of caching. The most reliable way is to use resource versioning (e.g., /api/v2/products instead of ?version=2). If versioning is not possible, use:

• Purge requests (Nginx purge, Kong Admin API).
• Short TTL with parallel updates (stale-while-revalidate).
• Tag-based invalidation (e.g., via Surrogate-Key in Kong).

Never clear the entire cache unless absolutely necessary — it will cause a cache stampede.

Process of work

1. Analysis: audit endpoints, identify candidates for caching.
2. Design: choose strategy, cache key, TTL, invalidation method.
3. Implementation: configure gateway, write Terraform/Ansible if needed.
4. Testing: measure hit ratio, verify correctness for authorized requests.
5. Monitoring: dashboards with hit rate, cache size, miss count.

What's included in the work

• Caching configuration for selected routes.
• Documentation on invalidation and monitoring.
• Team training (1–2 hours).
• Guarantee of correct operation for one month.

Contact us for a consultation — we'll assess your project within one day. Experience of over 10 years and 40+ successful implementations guarantee results. Order caching configuration and get results in 1–5 days.

API Development with REST, GraphQL, WebSocket, and tRPC

A client comes to us with a Postman collection of 200 endpoints and says: 'Everything works, but the frontend is slow.' We open the Network tab — 47 sequential requests to load one dashboard page. Each one waits for the previous. This is not a server speed issue — it's an API architecture problem. With 10 years on the market, we've redesigned dozens of such integrations, and we guarantee: the right protocol and contract solve the problem at its root.

When REST stops being enough

REST works well for simple CRUD operations. But as soon as a mobile app appears alongside the web interface, over-fetching begins: the mobile app requests /api/users/123 and gets a 4KB object, but only needs name and avatar. Multiply that by a list of 50 users — 200KB traffic instead of 8KB.

GraphQL solves this with selection sets. The client describes exactly the fields it needs, and the server returns only those. On a project with React Native + Next.js, we migrated from REST to Apollo Server: payload size on the main screen dropped from 340KB to 28KB — a 92% traffic savings. Our certified engineers confirm: the typical pain when adopting GraphQL is N+1 query. A resolver for the author field on a post calls SELECT * FROM users WHERE id = ? for each post in the list. On a page with 20 posts — 21 database queries. Solved with DataLoader — it batches queries and turns them into one SELECT * FROM users WHERE id IN (...).

What is tRPC and how is it better than REST/GraphQL?

If the entire stack is TypeScript (Next.js + Node/Bun), tRPC removes a whole layer of problems. You define a procedure on the server — the client gets full type-safety automatically, without code generation and without Swagger. Renamed a field in the Zod schema — TypeScript highlights all places on the frontend where it's used. tRPC reduces code by 2 times compared to REST + Swagger + openapi-typescript: no need to maintain a separate specification and generate types — everything is inferred from runtime validators. However, tRPC is not suitable if the API is consumed by third-party clients or mobile apps in other languages — in such cases we use GraphQL or REST with OpenAPI specification.

WebSocket and real-time: when SSE, when WS?

HTTP polling every 5 seconds is an illusion of real-time with up to 5 seconds delay and useless server load. For chats, live notifications, collaborative editing — WebSocket or Server-Sent Events. SSE is a one-way stream from server to client, works over ordinary HTTP, automatically reconnects. Suitable for notifications, data streaming, progress bars. WebSocket is bidirectional, needed for chats and collaborative features. Experience shows: 80% of 'real-time' tasks are solved with SSE, not WebSocket — fewer infrastructure complexities.

A typical mistake: opening a WebSocket connection for each page component. On one project, the dashboard opened 12 parallel WS connections. The correct approach is one connection manager at the application level, subscriptions through it. In our work results, we always transfer the connection scheme and a ready solution.

Swagger / OpenAPI as a contract

Documentation written after the fact becomes outdated the day after release. We write the OpenAPI 3.1 specification before development starts; it becomes the contract between frontend and backend. The frontend generates types via openapi-typescript, the backend validates incoming data using generated schemas. Contract deviation from implementation is caught on CI, not during review. For Laravel — l5-swagger or dedoc/scramble. For Node.js — @fastify/swagger or Zod + zod-to-openapi.

How to properly authenticate an API?

JWT with long-lived access tokens without rotation is a source of problems when compromised. The correct scheme: access token for 15 minutes, refresh token for 30 days with rotation on each use. Refresh token stored in an httpOnly cookie, access token in memory (not in localStorage). For inter-service communication — API Keys with scope limitations or mTLS. OAuth 2.0 with PKCE for public clients (SPA, mobile).

How to handle versioning and backward compatibility?

Breaking changes in an API without versioning break clients. Three approaches we use in projects:

We guarantee backward compatibility through automated checks (oasdiff) on CI.

How we develop APIs: step-by-step plan

1. Analysis — audit of current integrations, data schema compilation, protocol selection (REST/GraphQL/tRPC/WebSocket).
2. Contract design — OpenAPI or SDL (GraphQL) before the first line of code.
3. Development — implementation per contract, unit tests for each endpoint.
4. Load testing — k6: 500 virtual users, 10 minutes, p95 latency ≤ 200ms.
5. Deployment — CI/CD with backward compatibility check, automatic documentation publication.
6. Team training — handover of Postman collection or Playground, connection instructions.

What is included (deliverables)

• OpenAPI 3.1 specification (or SDL for GraphQL).
• Generated client types for TypeScript / Dart / Kotlin.
• Set of automated tests covering all endpoints (unit + integration).
• Load tests (k6) and report (p50/p95/p99 latency, RPS).
• Documentation in Swagger UI / Redoc / GraphiQL.
• Team training (2–4 hour workshop).
• Support for 30 days after delivery (per contract).

Our experience

• 10+ years in the API development market.
• 200+ completed projects (REST, GraphQL, WebSocket, tRPC).
• 50+ certified engineers (AWS, Kubernetes, API Design).
• Traffic savings averaging 85% when migrating from REST to GraphQL for mobile apps.
• 100% backward compatibility — not a single broken client in the last 3 years.

Timeline

API development for a typical SaaS project with 30–50 endpoints: from 3 to 8 weeks depending on business logic complexity and number of external integrations. Migration of an existing REST API to GraphQL: from 2 to 6 weeks. Adding a WebSocket layer to an existing backend: from 1 to 3 weeks. Cost is calculated individually after an audit. Get a consultation — contact us to discuss your project.