ElastiCache & Redis

ElastiCache Overview

Amazon ElastiCache is a managed in-memory data store supporting Redis and Memcached.

Redis vs Memcached

Create Redis Cluster

# Create a Redis cluster
aws elasticache create-cache-cluster \
  --cache-cluster-id my-redis \
  --engine redis \
  --engine-version 7.0 \
  --cache-node-type cache.r6g.large \
  --num-cache-nodes 1 \
  --vpc-security-group-ids sg-xxx \
  --cache-subnet-group-name my-subnet-group

# Create Redis replication group (multi-node)
aws elasticache create-replication-group \
  --replication-group-id my-redis-cluster \
  --description "Redis cluster" \
  --num-cache-clusters 3 \
  --cache-node-type cache.r6g.large \
  --engine redis \
  --engine-version 7.0 \
  --multi-az enabled \
  --automatic-failover enabled \
  --cache-subnet-group-name my-subnet-group \
  --security-group-ids sg-xxx

Create Memcached Cluster

aws elasticache create-cache-cluster \
  --cache-cluster-id my-memcached \
  --engine memcached \
  --engine-version 1.6 \
  --cache-node-type cache.r6g.large \
  --num-cache-nodes 3 \
  --vpc-security-group-ids sg-xxx \
  --cache-subnet-group-name my-subnet-group

Caching Strategies

Cache-Aside Pattern

Application → Check Cache → [Cache Miss] → Read DB → Write to Cache → Return
                          → [Cache Hit] → Return from Cache

import redis
import json

def get_user(user_id):
    # Check cache first
    cached = redis_client.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)
    
    # Cache miss - read from DB
    user = db.query("SELECT * FROM users WHERE id = %s", user_id)
    
    # Write to cache with TTL
    redis_client.setex(
        f"user:{user_id}",
        3600,  # 1 hour TTL
        json.dumps(user)
    )
    return user

Write-Through Pattern

Application → Write to Cache → Write to DB → Return

Write-Behind (Write-Back) Pattern

Application → Write to Cache → Return immediately
            (Async) → Write to DB in background

Redis Commands

# String operations
redis-cli SET user:123 '{"name":"John"}'
redis-cli GET user:123
redis-cli SETEX session:abc 3600 'data'  # Set with TTL

# Hash operations
redis-cli HSET user:123 name "John" age 30
redis-cli HGET user:123 name
redis-cli HGETALL user:123

# List operations
redis-cli LPUSH queue:tasks 'task1' 'task2' 'task3'
redis-cli RPOP queue:tasks
redis-cli LLEN queue:tasks

# Set operations
redis-cli SADD tags:post:1 "aws" "cloud" "devops"
redis-cli SMEMBERS tags:post:1
redis-cli SINTER tags:post:1 tags:post:2  # Common tags

# Sorted Set (leaderboard)
redis-cli ZADD leaderboard 100 "player1"
redis-cli ZADD leaderboard 200 "player2"
redis-cli ZREVRANGE leaderboard 0 9 WITHSCORES  # Top 10

# Pub/Sub
redis-cli PUBLISH channel1 "message"
redis-cli SUBSCRIBE channel1

ElastiCache Security and Replication

Security

# Enable encryption at rest
aws elasticache create-cache-cluster \
  --cache-cluster-id my-redis \
  --at-rest-encryption-enabled \
  --transit-encryption-enabled

# Enable encryption in transit
aws elasticache create-cache-cluster \
  --cache-cluster-id my-redis \
  --transit-encryption-enabled

# Configure AUTH token
aws elasticache modify-cache-cluster \
  --cache-cluster-id my-redis \
  --auth-token "LongPassword123!" \
  --auth-token-update-mode SET

# IAM authentication
aws elasticache create-cache-cluster \
  --cache-cluster-id my-redis \
  --auth-token-update-mode SET

Replication and Failover

# Multi-AZ replication group
aws elasticache create-replication-group \
  --replication-group-id my-redis-ha \
  --num-cache-clusters 3 \
  --multi-az enabled \
  --automatic-failover enabled \
  --snapshot-retention-limit 7 \
  --snapshot-window 03:00-05:00 \
  --preferred-maintenance-window sun:05:00-sun:06:00

# Check replication status
aws elasticache describe-replication-groups \
  --replication-group-id my-redis-ha \
  --query 'ReplicationGroups[0].NodeGroups[*].NodeGroupMembers[*].[CacheClusterId,CurrentRole]'

# Manual failover
aws elasticache test-failover --replication-group-id my-redis-ha --cache-cluster-id my-redis-ha-003

Replication Architecture

┌─────────────────────────────────────────────────────┐
│              Redis Replication Group                 │
├─────────────────────────────────────────────────────┤
│                                                     │
│  Primary (Write)                                    │
│  ┌──────────────────┐                              │
│  │ my-redis-001     │ ←── Writes go here           │
│  │ (AZ-1a)          │                              │
│  └────────┬─────────┘                              │
│           │                                        │
│     ┌─────┼─────────────┐                          │
│     │     │             │                          │
│  ┌──▼───────┐ ┌────────▼──┐                       │
│  │Replica-1 │ │Replica-2  │                       │
│  │(AZ-1b)   │ │(AZ-1c)    │                       │
│  │ Reads    │ │ Reads     │                       │
│  └──────────┘ └───────────┘                       │
│                                                     │
│  Automatic Failover: If primary fails,             │
│  a replica is promoted to primary                   │
└─────────────────────────────────────────────────────┘

ElastiCache Best Practices and Cost

Memory Optimization

# Monitor memory usage
redis-cli INFO memory

# Configure maxmemory policy
redis-cli CONFIG SET maxmemory-policy allkeys-lru

# Eviction policies
# noeviction: Return error when memory full
# allkeys-lru: Evict least recently used keys
# volatile-lru: Evict LRU keys with TTL
# allkeys-random: Evict random keys
# volatile-ttl: Evict keys with shortest TTL

Connection Pooling

import redis
from redis.connection import ConnectionPool

# Create connection pool
pool = ConnectionPool(
    host='my-redis.xxx.cache.amazonaws.com',
    port=6379,
    db=0,
    max_connections=20,
    decode_responses=True
)

# Use pool
redis_client = redis.Redis(connection_pool=pool)

Cost Optimization

Monitoring

# Key metrics to watch
aws cloudwatch get-metric-statistics \
  --namespace AWS/ElastiCache \
  --metric-name CacheHits \
  --dimensions Name=CacheClusterId,Value=my-redis \
  --start-time $(date -u -d '1 hour ago') \
  --end-time $(date -u) \
  --period 300 \
  --statistics Sum

# Important metrics:
# - CacheHitRate: Should be > 80%
# - Evictions: High = memory pressure
# - CurrentConnections: Monitor for spikes
# - CurrConnections: Active connections