Varnish Integration — VCL Configuration and Cache Invalidation

How Varnish Works

Client -> Varnish (Port 80) -> Magento (Port 8080)
         Cache hit -> Response from Varnish
         Cache miss -> Fetch from Magento, cache, respond

Magento and Varnish

• Magento generates FPC HTML
• Varnish caches the HTML at the edge
• Subsequent requests served from Varnish cache
• Magento sends invalidation headers to Varnish

Setup Overview

# 1. Export VCL from Magento
php bin/magento varnish:vcl:generate \
    --host=127.0.0.1 \
    --port=6081 \
    --name=varnish.vcl

# 2. Install Varnish
apt install varnish

# 3. Deploy VCL
cp varnish.vcl /etc/varnish/default.vcl

# 4. Start Varnish
systemctl start varnish

Generated VCL Structure

vcl 4.1;

import std;

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}

sub vcl_recv {
    if (req.method == "PURGE") {
        return (purge);
    }
    
    # Remove tracking params
    set req.url = regsuball(req.url, "\?.*$", "");
    
    # Static files - long cache
    if (req.url ~ "\.(css|js|jpg|jpeg|png|gif|ico|svg)$") {
        return (hash);
    }
}

sub vcl_backend_response {
    # Set TTL for different content types
    if (beresp.http.X-Magento-TTL) {
        set beresp.ttl = std.duration(beresp.http.X-Magento-TTL, 3600s);
    }
    
    # Grace period for stale content
    set beresp.grace = 1d;
}

Custom VCL Modifications

# Custom grace handling
sub vcl_backend_response {
    set beresp.grace = 24h;
    set beresp.keep = 24h;
}

# Handle cookies for static files
sub vcl_recv {
    if (req.url ~ "\.(css|js|png|jpg|gif|ico)$") {
        unset req.http.Cookie;
    }
}

Backend Configuration

backend magento {
    .host = "magento-server";
    .port = "8080";
    .connect_timeout = 5s;
    .first_byte_timeout = 300s;
    .between_bytes_timeout = 60s;
    .probe = {
        .url = "/health_check";
        .interval = 5s;
        .timeout = 3s;
        .window = 5;
        .threshold = 3;
    }
}

Magento Cache Tags

Magento sends X-Magento-Tags header with cache tags:

X-Magento-Tags: catalog_product_123, catalog_category_45

PURGE Configuration

sub vcl_recv {
    if (req.method == "PURGE") {
        if (!client.ip ~ purge_acl) {
            return (synth(403, "Forbidden"));
        }
        return (purge);
    }
}

# Purge by tag
sub vcl_recv {
    if (req.method == "XCG" && req.http.X-Magento-Tags) {
        # Purge cache entries matching tags
        ban("req.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags);
        return (synth(200, "Banned"));
    }
}

Purge ACL

acl purge {
    "localhost";
    "127.0.0.1";
    "192.168.0.0"/16;
}

Soft Purge vs Hard Purge

# Hard purge - removes from cache immediately
curl -X PURGE http://varnish/magento-url

# Soft purge - marks for lazy invalidation
# (handled by VCL grace logic)

TTL Configuration

sub vcl_backend_response {
    # Default TTL
    set beresp.ttl = 1h;
    
    # Static files - long TTL
    if (beresp.url ~ "\.(css|js|jpg|png)$") {
        set beresp.ttl = 7d;
    }
    
    # HTML pages - short TTL
    if (beresp.http.Content-Type ~ "text/html") {
        set beresp.ttl = 15m;
    }
}

Grace Period

Grace allows serving stale content while backend is fetching fresh content:

sub vcl_backend_response {
    # Serve stale for up to 24h if backend is down
    set beresp.grace = 24h;
}

sub vcl_recv {
    # Use grace if backend is unhealthy
    if (!std.healthy(req.backend_hint)) {
        set req.grace = 24h;
    }
}

Grace Handling Logic

sub vcl_backend_response {
    # If backend response is an error, extend grace
    if (beresp.status >= 500) {
        set beresp.grace = 1h;
    }
}

sub vcl_synth {
    # Custom error pages
    if (resp.status == 503) {
        synthetic("Backend unavailable");
    }
}

Production VCL Settings

# Check Varnish status
varnishstat

# Check cache hit rate
varnishstat -f MAIN.cache_hit
varnishstat -f MAIN.cache_miss

# Purge all cache
varnishadm "ban.url .*"

# Check VCL syntax
varnishd -C -f /etc/varnish/default.vcl