Memory Estimation for Magento 2

PHP Memory in Magento

Memory Usage Patterns

Operation                    | Typical Memory
-----------------------------|---------------
Simple page load             | 64-128 MB
Category page (large catalog)| 128-256 MB
Product page                 | 96-192 MB
Cart/Checkout                | 128-256 MB
Admin order processing       | 128-256 MB
Import/Export                | 256-512 MB
Compilation (di:compile)     | 512 MB-1 GB
Static content deploy        | 512 MB-2 GB

Measuring Memory Usage

// Get current memory usage
$currentMemory = memory_get_usage(true);
$peakMemory = memory_get_peak_usage(true);

// Convert to MB
$currentMB = $currentMemory / 1024 / 1024;
$peakMB = $peakMemory / 1024 / 1024;

echo "Current: {$currentMB} MB\n";
echo "Peak: {$peakMB} MB\n";

// Per-request memory tracking
register_shutdown_function(function() {
    $usage = memory_get_peak_usage(true) / 1024 / 1024;
    error_log("Memory peak: {$usage} MB");
});

PHP Configuration

; php.ini settings for Magento
memory_limit = 512M          ; Per-request limit
max_execution_time = 300     ; For long operations
realpath_cache_size = 4096K  ; File path caching

Memory per Worker

PHP-FPM Worker Memory:
- Idle worker: 20-30 MB
- Active worker: 64-256 MB
- Peak worker: 512 MB

For 50 concurrent requests:
- 50 workers × 128 MB average = 6.4 GB
- With overhead: ~8 GB

Memory Optimization

// 1. Use generators for large datasets
public function processProducts(): Generator
{
    $products = $this->productCollection->load();
    foreach ($products as $product) {
        yield $product;
    }
    // Memory freed after each iteration
}

// 2. Unset variables after use
$data = $this->loadLargeDataset();
// Process data...
unset($data);

// 3. Use streaming for exports
public function exportProducts(): void
{
    $handle = fopen('php://output', 'w');
    foreach ($this->getProducts() as $product) {
        fputcsv($handle, $product->toArray());
    }
    fclose($handle);
}

Redis Usage in Magento

Cache Types

Cache Type           | Purpose               | Size Estimate
---------------------|-----------------------|---------------
config               | Configuration cache   | 1-5 MB
full_page_cache      | Full page cache       | 50-200 MB
block_html           | Block HTML cache      | 20-100 MB
collections          | Collection cache      | 10-50 MB
eav                  | EAV metadata          | 5-20 MB
reflection           | Class metadata        | 5-10 MB
translate            | Translation strings   | 2-10 MB

Session Storage

Session Size:
- Average session: 2-5 KB
- Cart session: 5-10 KB
- With items: 10-20 KB

For 1000 concurrent users:
- 1000 × 5 KB = 5 MB sessions
- With headroom: 20 MB

Redis Memory Calculation

// Calculate Redis memory needs
function calculateRedisMemory(
    $dailyPageViews,
    $cachedPages,
    $concurrentUsers,
    $cacheEntries
): array {
    $estimates = [
        'full_page_cache' => [
            'pages' => $cachedPages,
            'avg_size' => 50, // KB per cached page
            'total_mb' => ($cachedPages * 50) / 1024
        ],
        'session' => [
            'users' => $concurrentUsers,
            'avg_size' => 5, // KB per session
            'total_mb' => ($concurrentUsers * 5) / 1024
        ],
        'block_cache' => [
            'entries' => $cacheEntries,
            'avg_size' => 2, // KB per entry
            'total_mb' => ($cacheEntries * 2) / 1024
        ],
        'config_cache' => [
            'total_mb' => 5
        ]
    ];
    
    $totalMB = array_sum(array_column($estimates, 'total_mb'));
    
    // Add 50% headroom
    $recommendedMB = $totalMB * 1.5;
    
    return [
        'breakdown' => $estimates,
        'total_mb' => round($totalMB, 2),
        'recommended_mb' => round($recommendedMB, 2),
        'recommended_gb' => round($recommendedMB / 1024, 2)
    ];
}

// Example: 100K daily PV, 10K cached pages, 200 concurrent
$redis = calculateRedisMemory(100000, 10000, 200, 5000);
// total_mb: 54.88
// recommended_mb: 82.32
// recommended_gb: 0.08

Redis Configuration

# redis.conf for Magento
maxmemory 4gb                    # Max memory limit
maxmemory-policy allkeys-lru     # Eviction policy

# For sessions only
maxmemory 2gb
maxmemory-policy volatile-lru

Monitor Redis Memory

# Check Redis memory
redis-cli info memory

# used_memory: 1073741824 (1 GB)
# used_memory_rss: 1157627904 (1.1 GB)
# mem_fragmentation_ratio: 1.08

# Check cache hit rate
redis-cli info stats
# keyspace_hits: 1000000
# keyspace_misses: 50000
# hit_rate: 95%

InnoDB Buffer Pool

Buffer Pool Sizing

Rule of thumb: 70-80% of available RAM

Server RAM: 16 GB
Buffer Pool: 16 × 0.75 = 12 GB

Server RAM: 32 GB
Buffer Pool: 32 × 0.75 = 24 GB

For Magento:
- Small store (< 50K products): 4 GB
- Medium store (50K-200K products): 8 GB
- Large store (200K-1M products): 16 GB
- Enterprise (> 1M products): 32 GB+

Buffer Pool Configuration

# my.cnf for Magento
innodb_buffer_pool_size = 12G
innodb_buffer_pool_instances = 12  # 1 per GB
innodb_log_file_size = 1G
innodb_log_buffer_size = 64M
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT

Calculating Buffer Pool Hit Rate

-- Buffer pool hit rate
SELECT 
    (1 - (Innodb_buffer_pool_reads / Innodb_buffer_pool_read_requests)) * 100 AS hit_rate
FROM (
    SELECT 
        VARIABLE_VALUE AS Innodb_buffer_pool_reads
    FROM performance_schema.global_status
    WHERE VARIABLE_NAME = 'Innodb_buffer_pool_reads'
) a,
(
    SELECT 
        VARIABLE_VALUE AS Innodb_buffer_pool_read_requests
    FROM performance_schema.global_status
    WHERE VARIABLE_NAME = 'Innodb_buffer_pool_read_requests'
) b;

-- Target: > 99% hit rate

Monitor Buffer Pool

-- Current buffer pool status
SHOW STATUS LIKE 'Innodb_buffer_pool%';

-- Key metrics:
-- Innodb_buffer_pool_read_requests: Total reads
-- Innodb_buffer_pool_reads: Disk reads (bad)
-- Hit rate = 1 - (reads / requests)

MySQL Memory Usage

MySQL Memory Components:

Component                    | Calculation
-----------------------------|--------------------------------
Buffer Pool                  | innodb_buffer_pool_size
Per-connection buffers       | max_connections × (sort_buffer + read_buffer + ...)
Query Cache (if enabled)     | query_cache_size
OS Buffers                   | 10-20% of total

Example (32GB server):
- Buffer Pool: 24 GB
- Connections (200 × 10MB): 2 GB
- OS Buffers: 6 GB
- Total: 32 GB

Connection Pool Sizing

# Optimal connection settings
max_connections = 200         # Max simultaneous connections
thread_cache_size = 32        # Cache threads for reuse
table_open_cache = 4000       # Cache table handles

Total System Memory

Complete Calculation

function calculateTotalMemory(
    $serverType, // 'web', 'db', 'cache'
    $concurrentUsers,
    $databaseSizeGB
): array {
    $memory = [];
    
    if ($serverType === 'web') {
        $memory['php_workers'] = $concurrentUsers * 0.128; // 128MB per worker
        $memory['os_overhead'] = 2; // GB
        $memory['total_gb'] = array_sum($memory) + 2; // buffer
    }
    
    if ($serverType === 'db') {
        $memory['buffer_pool'] = $databaseSizeGB * 0.75;
        $memory['connections'] = 200 * 10 / 1024; // 200 connections × 10MB
        $memory['os_overhead'] = 4;
        $memory['total_gb'] = array_sum($memory) + 4;
    }
    
    if ($serverType === 'cache') {
        $memory['redis'] = 4; // GB
        $memory['os_overhead'] = 2;
        $memory['total_gb'] = array_sum($memory) + 2;
    }
    
    return $memory;
}

// Example: Web server, 200 concurrent users
$webMemory = calculateTotalMemory('web', 200, 0);
// php_workers: 25.6 GB
// os_overhead: 2 GB
// total_gb: 29.6 GB → recommend 32 GB

PHP Memory Optimization

Code-Level Optimization

// 1. Use lazy loading
// BAD: Eager loading all products
$products = $this->productCollection->load();
foreach ($products as $product) {
    // 50K products loaded at once
}

// GOOD: Lazy loading with iterator
$products = $this->productCollection->setPageSize(100);
$page = 1;
while ($products->getLastPageNumber() >= $page) {
    $products->setCurPage($page);
    foreach ($products as $product) {
        // Process one product at a time
    }
    $page++;
}

// 2. Use generators for large datasets
public function getProducts(): Generator
{
    $select = $this->connection->select()
        ->from('catalog_product_entity');
    
    $stmt = $this->connection->query($select);
    while ($row = $stmt->fetch()) {
        yield $row;
    }
}

// 3. Stream large exports
public function exportLargeCsv(): void
{
    $handle = fopen('php://output', 'w');
    foreach ($this->getProducts() as $product) {
        fputcsv($handle, $product);
    }
    fclose($handle);
}

Redis Optimization

# Redis memory optimization
maxmemory-policy allkeys-lru  # Evict least recently used

# Compress small values
rdbcompression yes

# Use hashes for small objects
hash-max-ziplist-entries 128
hash-max-ziplist-value 64

MySQL Optimization

# Buffer pool tuning
innodb_buffer_pool_size = 12G
innodb_buffer_pool_instances = 12

# Reduce memory per connection
sort_buffer_size = 256K
read_buffer_size = 128K
read_rnd_buffer_size = 256K
join_buffer_size = 128K

# Limit connections
max_connections = 200

Monitoring Dashboard

$memoryMetrics = [
    'php' => [
        'current' => memory_get_usage(true) / 1024 / 1024,
        'peak' => memory_get_peak_usage(true) / 1024 / 1024,
        'limit' => ini_get('memory_limit')
    ],
    'redis' => $this->getRedisMemory(),
    'mysql' => $this->getMysqlMemory(),
    'system' => $this->getSystemMemory()
];

Memory Sizing Guidelines

Component      | Small Store | Medium Store | Large Store
---------------|-------------|--------------|-------------
PHP per worker | 128 MB      | 256 MB       | 256 MB
Redis          | 2 GB        | 4 GB         | 8 GB
MySQL Buffer   | 4 GB        | 8 GB         | 16 GB
OS Overhead    | 2 GB        | 4 GB         | 8 GB

Total per server:
- Web: 8-32 GB
- Database: 16-32 GB
- Cache: 4-8 GB