Varnish Scaling

Multi-Varnish Topology

                    ┌─────────────┐
                    │ Load Balancer│
                    └──────┬──────┘
              ┌────────────┼────────────┐
              â–¼            â–¼            â–¼
        ┌──────────┐ ┌──────────┐ ┌──────────┐
        │ Varnish 1│ │ Varnish 2│ │ Varnish 3│
        └────┬─────┘ └────┬─────┘ └────┬─────┘
             │             │             │
             └──────┬──────┘──────┬──────┘
                    â–¼             â–¼
              ┌──────────┐ ┌──────────┐
              │ Web Node1│ │ Web Node2│
              └──────────┘ └──────────┘

Varnish Backend Config

# default.vcl
backend web1 {
    .host = "web1.example.com";
    .port = "8080";
    .connect_timeout = 5s;
    .first_byte_timeout = 60s;
    .between_bytes_timeout = 5s;
    .max_connections = 300;
    .probe = {
        .url = "/health_check.php";
        .timeout = 3s;
        .interval = 5s;
        .window = 5;
        .threshold = 3;
    }
}

backend web2 {
    .host = "web2.example.com";
    .port = "8080";
    .connect_timeout = 5s;
    .first_byte_timeout = 60s;
    .between_bytes_timeout = 5s;
    .max_connections = 300;
}

sub vcl_init {
    new web_servers = directors.round_robin();
    web_servers.add_backend(web1);
    web_servers.add_backend(web2);
}

sub vcl_recv {
    set req.backend_hint = web_servers.backend();
}

Cache Hierarchy

Level 1: Browser Cache (no request)
Level 2: CDN Edge Cache (regional)
Level 3: Varnish Cache (per-node)
Level 4: Magento FPC (application)
Level 5: Redis/Memcached (data)
Level 6: MySQL (source)

Varnish Cache Configuration

# Cache static assets aggressively
sub vcl_recv {
    if (req.url ~ "\.(css|js|jpg|jpeg|png|gif|ico|svg|woff2)$") {
        unset req.http.Cookie;
        return (hash);
    }
}

# Cache HTML pages with variation support
sub vcl_recv {
    # Support device/user-agent variations
    if (req.http.X-Device == "mobile") {
        set req.http.X-Varnish-Key = "mobile";
    } else {
        set req.http.X-Varnish-Key = "desktop";
    }
}

sub vcl_hash {
    hash_data(req.http.X-Varnish-Key);
}

# Cache TTL rules
sub vcl_backend_response {
    # Static assets: 30 days
    if (bereq.url ~ "\.(css|js|jpg|jpeg|png|gif|ico|svg)$") {
        set beresp.ttl = 30d;
        set beresp.http.Cache-Control = "max-age=2592000";
    }
    # HTML pages: varies by type
    elsif (beresp.http.Content-Type ~ "text/html") {
        set beresp.ttl = 24h;
        set beresp.http.Cache-Control = "max-age=86400";
    }
}

Invalidation Strategies

Strategy          | Mechanism           | Use Case
──────────────────|─────────────────────|──────────────
TTL Expiration    | Time-based          | Default
Purge by URL      | API call            | Content update
BAN (Purge All)   | Pattern matching    | Mass invalidation
Soft Purge        | Mark stale          | Background refresh

Magento Varnish Integration

# Purge single URL
curl -X PURGE http://varnish.example.com/catalog/product/view/id/123

# Ban pattern
curl -X BAN http://varnish.example.com -H "X-Purge-Regex: /catalog/.*"

# Flush all
curl -X BAN http://varnish.example.com -H "X-Purge-Regex: .*"

Magento Cache Invalidation Config

// app/etc/env.php
'cache' => [
    'frontend' => [
        'default' => [
            'backend' => 'Magento\Framework\Cache\Backend\Redis'
        ]
    ],
    'page_cache' => [
        'id_prefix' => 'magento_'
    ]
],
// Varnish invalidation handled by Magento FPC module
// Automatically invalidates on product/category save

Key Metrics

# Hit rate
curl http://varnish.example.com:6081/stats
# Hit rate should be > 80%

# Real-time stats
varnishstat -1

# Key metrics:
# MAIN.cache_hit: Cache hits
# MAIN.cache_miss: Cache misses  
# MAIN.backend_req: Backend requests
# MAIN.n_object: Cached objects
# MAIN.s_bodybytes: Bytes served

Health Check Monitoring

# Check backend health
varnishlog -g raw -q 'ReqMethod eq "GET" and BerespStatus eq 503'

# Monitor failed health checks
varnishlog -g raw -q 'FetchError ~ "no backend"'

# Alert if hit rate drops below 70%
# Hit rate = cache_hit / (cache_hit + cache_miss)

Performance Tuning

# Increase cache size
varnishd -s malloc,16G

# Optimize thread settings
varnishd -p thread_pool_min=200 -p thread_pool_max=4000

# Enable HTTP/2
varnishd -p feature=+http2