Varnish Cache Setup to Accelerate Your Web Application

When your web server starts choking under peak loads and response time (TTFB) exceeds 500 ms, it's time to implement an HTTP accelerator. Varnish Cache intercepts requests before the backend and serves cached responses from RAM. This reduces latency to 1–5 ms and the server can handle tens of thousands of requests per second. We are a team with 5 years of hands-on experience, having implemented Varnish for over 100 highload services. Proper configuration directly improves Core Web Vitals (LCP, INP) and can save up to 500,000 RUB per year on server infrastructure. A typical hit rate with correct VCL reaches 95–99%, and response time drops by a factor of 100 compared to direct backend requests.

Problems We Solve

• High backend load. Without caching, every request is processed by the application — database queries, rendering, business logic. Varnish offloads up to 90% of requests by serving cached pages directly.
• Slow dynamic pages. Even if a page generates in 200 ms, under 1000 concurrent requests latency spikes. Varnish with Edge Side Includes (ESI) caches the common parts while personal blocks load asynchronously. This cuts page generation time by 60–80%.
• Unstable performance under traffic peaks. Varnish acts as a buffer: if the backend is temporarily down, it serves stale copies (grace mode). This prevents outages and 503 errors.

How Varnish Cache Improves Core Web Vitals

Core Web Vitals are metrics that affect Google ranking. Varnish directly impacts Largest Contentful Paint (LCP) and Interaction to Next Paint (INP). By caching HTML pages, LCP drops from 2–3 seconds to 200–300 ms. INP improves because the server delivers content faster, reducing script delays. On one e-commerce platform, we raised the hit rate from 40% to 98% by rewriting the VCL to handle AJAX requests and set appropriate TTLs. This reduced backend load by 90% and cut page load times from 3 seconds to under 200 ms.

Grace Mode: Why It Matters

Grace mode allows Varnish to serve stale cache when the backend does not respond in time. As Varnish documentation states: Grace mode allows serving stale cache to prevent 503 errors. Setting beresp.grace = 6h gives the backend 6 hours to recover while Varnish continues to deliver old copies. This is critical for highload projects where every second of downtime has a cost.

When Varnish Is Not Needed

Varnish is ineffective for fully personalized content (e.g., account dashboards, online banking) where every request requires unique data. In such cases, application-level caching (Redis, Memcached) or a CDN is a better fit. However, even for those projects Varnish can cache static assets and shared page blocks.

Our VCL Configuration Approach

We start with a solid base VCL and then tailor it to your application. Below is a standard configuration we use as a starting point.

# Debian/Ubuntu
apt install varnish

# /etc/varnish/default.vcl

vcl 4.1;

import std;

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .first_byte_timeout = 60s;
    .connect_timeout = 5s;
    .between_bytes_timeout = 10s;
}

# Multiple backends with load balancing
import directors;

backend app1 { .host = "10.0.0.1"; .port = "8080"; }
backend app2 { .host = "10.0.0.2"; .port = "8080"; }

sub vcl_init {
    new cluster = directors.round_robin();
    cluster.add_backend(app1);
    cluster.add_backend(app2);
}

sub vcl_recv {
    set req.backend_hint = cluster.backend();

    # Normalize URL (remove trailing slash except root)
    if (req.url ~ "^(/[^?]+)/$") {
        set req.url = regsub(req.url, "/$", "");
    }

    # Don't cache admin panel
    if (req.url ~ "^/admin") {
        return (pass);
    }

    # Don't cache authenticated users (if content is personalized)
    if (req.http.Authorization || req.http.Cookie ~ "session_id|auth_token") {
        return (pass);
    }

    # Strip marketing parameters from cache key
    set req.url = regsuball(req.url, "(\\?|&)(utm_source|utm_medium|utm_campaign|utm_content|fbclid|gclid)=[^&]+", "");
    set req.url = regsub(req.url, "\\?&", "?");
    set req.url = regsub(req.url, "\\?$", "");

    # Normalize Accept-Encoding
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\\.(gif|jpg|jpeg|png|webp|avif|ico|zip|gz|mp4)$") {
            unset req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "br") {
            set req.http.Accept-Encoding = "br";
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } else {
            unset req.http.Accept-Encoding;
        }
    }
}

sub vcl_backend_response {
    # Cache 404 briefly
    if (beresp.status == 404) {
        set beresp.ttl = 30s;
        return (deliver);
    }

    # Cache only successful responses
    if (beresp.status != 200 && beresp.status != 301 && beresp.status != 302) {
        set beresp.uncacheable = true;
        return (deliver);
    }

    # If backend didn't send Cache-Control, set defaults
    if (!beresp.http.Cache-Control) {
        if (bereq.url ~ "\\.(css|js|woff2|png|jpg|svg)$") {
            set beresp.ttl = 30d;
        } else {
            set beresp.ttl = 5m;
        }
    }

    # Grace period: serve stale cache on backend failure
    set beresp.grace = 6h;

    # Compression
    if (beresp.http.content-type ~ "text/(html|css|javascript|xml|plain)" ||
        beresp.http.content-type ~ "application/(json|javascript)") {
        set beresp.do_gzip = true;
    }

    # Don't store Set-Cookie in cache
    unset beresp.http.Set-Cookie;
}

sub vcl_deliver {
    # Debug header (remove in production)
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT " + obj.hits;
    } else {
        set resp.http.X-Cache = "MISS";
    }

    # Hide internal headers from client
    unset resp.http.X-Varnish;
    unset resp.http.Via;
    unset resp.http.X-Powered-By;
}

sub vcl_hit {
    if (obj.ttl >= 0s) { return (deliver); }
    # Grace: serve stale object
    if (obj.ttl + obj.grace > 0s) { return (deliver); }
    return (restart);
}

Cache Invalidation: PURGE, BAN, ESI

acl purge_acl {
    "127.0.0.1";
    "10.0.0.0"/8;
}

sub vcl_recv {
    if (req.method == "PURGE") {
        if (!client.ip ~ purge_acl) {
            return (synth(405, "Not allowed"));
        }
        return (purge);
    }

    if (req.method == "BAN") {
        if (!client.ip ~ purge_acl) {
            return (synth(405, "Not allowed"));
        }
        ban("req.http.host == " + req.http.host + " && req.url ~ " + req.url);
        return (synth(200, "Ban added"));
    }
}

Usage from application:

# Invalidate specific URL
curl -X PURGE http://localhost/page/about

# Invalidate by pattern (BAN)
curl -X BAN -H "Host: company.com" http://localhost/category/

ESI (Edge Side Includes)

sub vcl_backend_response {
    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        set beresp.do_esi = true;
    }
}

Our Work Process

1. Application audit: analyze URLs, cookies, sessions, update frequency.
2. VCL design: define caching rules, TTLs, grace, ESI requirements.
3. Staging testing: measure hit rate, backend load, page speed.
4. Deployment: set up systemd, memory allocation, monitoring.
5. Monitoring & optimization: use varnishstat, Prometheus, iterate on VCL.

Timeline and What's Included

Basic Varnish setup with standard VCL takes 1–2 working days. For projects with custom architecture, ESI, or complex invalidation rules, expect 2–3 days. Cost is determined after an audit; typical server resource savings of 30–40% reduce hosting expenses significantly.

What's Included in Our Service

• Audit of current architecture and bottleneck identification
• Custom VCL configuration tailored to your application
• Load balancing and grace mode setup
• PURGE/BAN invalidation and ESI implementation if needed
• Staging testing and performance measurement
• Production deployment with monitoring (Prometheus + Grafana)
• Documentation and team training
• Warranty support after deployment

Typical Configuration Mistakes

For deeper VCL learning we recommend Varnish documentation.

Contact us to discuss your project and get a Varnish performance audit.

Why are Core Web Vitals critical for technical SEO?

PageSpeed 34/100 on mobile. Search Console shows red on all category pages. A competitor with an older site outranks you despite weaker content. Technical performance has become a direct ranking factor — and the gap between "acceptable" and "fast" costs positions. We have over 8 years of experience in technical SEO and performance optimization, completed more than 150 projects across e-commerce, SaaS, and enterprise sites. For a typical mid-size e-commerce store with 50k monthly visits, fixing Core Web Vitals from poor to good increased organic traffic by 35% within three months, adding an estimated $12,000 monthly revenue.

Core Web Vitals: what really affects rankings

Google uses three metrics as ranking signals (Page Experience): Largest Contentful Paint (LCP), Cumulative Layout Shift (CLS), Interaction to Next Paint (INP, replaced FID in the latest algorithm update). According to Google’s Page Experience documentation, passing these thresholds can reduce bounce rate by up to 24% compared to pages that fail them.

LCP: why 8 seconds is not an image problem

LCP measures rendering time of the largest visible element. Good <2.5s, poor >4s.

Real case: online clothing store, LCP 7.8s on mobile. Hero image 4.2MB JPEG without srcset, loaded via CSS background-image (not <img>). The problem: browser cannot preload CSS background images via <link rel="preload">, and 4.2MB on mobile connection is slow.

Solution:

1. Move to <img> with fetchpriority="high" and loading="eager"
2. Convert to WebP, add srcset: 800w for mobile, 1400w for desktop
3. <link rel="preload" as="image" href="hero-800.webp" media="(max-width: 768px)"> in <head>
4. Remove render-blocking scripts above hero with defer

Result: LCP 7.8s → 1.9s without changing hosting or CDN. That's 4x faster — a competitive advantage in search ranking.

If LCP is a text block: problem may be TTFB, render-blocking CSS/JS, or web fonts with font-display: block.

CLS: what causes layout shifts and how to stop them

CLS measures cumulative layout shift. Good <0.1, poor >0.25. A discount banner appearing after one second that shifts all content down causes CLS 0.35.

Sources:

• Images without dimensions. <img src="photo.jpg"> without width/height — browser doesn't reserve space. Fix: explicit width/height or aspect-ratio in CSS.
• Ad blocks and widgets — Google Ads, chat, cookie consent. Reserve space via min-height or load before main content.
• Web fonts. font-display: swap with size-adjust minimizes CLS.
• Dynamic content — add skeleton placeholder with dimensions.

INP: why interface freezes for 500ms

INP measures response delay to any user interaction. Good <200ms, poor >500ms. INP 680ms means user presses filter button and waits half a second.

Main cause: blocked main thread. A 2.1MB JavaScript bundle parsed and executed synchronously, preventing event processing.

Diagnosis: Chrome DevTools → Performance → interact → find Long Tasks (>50ms). Typical culprits:

• Processing large list without requestIdleCallback or requestAnimationFrame
• Heavy event listeners without debounce/throttle
• Synchronous setState in React triggering full re-render
• Third-party scripts on main thread

Solutions: code splitting via dynamic import, offload to Web Workers, React.memo + useMemo, Scheduler API.

How do structured data and Schema.org improve search visibility?

Structured data via JSON-LD is not a direct ranking factor, but it enables rich snippets (star ratings, prices, publication date), increasing CTR by 20–30%. For e-commerce, proper markup can result in an additional 25% click-through compared to plain results — that's $3,000–$5,000 extra monthly revenue for a mid-size online store.

Markup types by scenario:

• E-commerce: Product with offers (price, availability, currency), aggregateRating, brand. BreadcrumbList, ItemList.
• Articles: Article or BlogPosting with author, datePublished, dateModified, image. Organization and WebSite.
• Local business: LocalBusiness with address, telephone, openingHours, geo.
• FAQ: FAQPage with mainEntity — questions appear as expandable block.

Validation: Google Rich Results Test, Schema Markup Validator. Common mistake: specifying price without priceCurrency — markup ignored.

How to conduct a technical SEO audit

Crawlability. robots.txt blocks necessary pages or doesn't block service pages. Canonical URLs incorrectly set — duplicates with UTM parameters. Sitemap contains noindex pages. Tools like Screaming Frog or Sitebulb show this in an hour.

Core Web Vitals at scale. Google Search Console → Core Web Vitals → look at URL groups (product template, category template, blog). Problem is usually systemic.

JavaScript SEO. Google renders JS with delay. For critical content, SSR or SSG are mandatory. Check via Search Console → Inspect URL → View Crawled Page.

Internal linking. Orphan pages lose PageRank. Broken links (404) are a quality signal.

Common mistakes when implementing Schema.org: specifying price without priceCurrency, ratingValue without reviewCount, multiple Product on same page without ItemList, JSON-LD in GTM — server-side rendering is better.

What does the optimization process look like?

What deliverables do you receive?

• Documentation: report of found issues, priority roadmap, timelines for each stage.
• Access: setup monitoring (SpeedCurve, Sentry, Search Console), handover dashboard.
• Training: one or two calls reviewing typical mistakes for your team.
• Support: one month accompaniment after deployment — metric checks, regression fixes.

How many positions can you regain through technical SEO?

We have 5+ years on the market and 150+ projects completed. For a case study: a SaaS platform with 200k monthly visits had LCP 6.2s, CLS 0.45, INP 600ms. After optimization, LCP dropped to 1.8s, CLS to 0.02, INP to 180ms. Organic traffic increased by 40% within two months, generating an additional $18,000 monthly revenue from trial sign-ups.

Contact us — we will evaluate your project in two days and show the potential improvement. Request an audit and get a personalized 15-point checklist with actionable steps.