Defining Service Levels
SLI/SLO/SLA Relationship
SLI (Service Level Indicator):
What you measure: latency, error rate, throughput
SLO (Service Level Objective):
Target: P99 < 500ms, error rate < 1%
SLA (Service Level Agreement):
Contract: breach = compensation
Magento SLIs
SLI Type | What to Measure | Target
─────────────|──────────────────────────────|─────────
Availability | Successful requests / total | 99.9%
Latency | Request duration (P99) | < 500ms
Error Rate | 5xx responses / total | < 0.1%
Throughput | Requests per second | > 1000
SLI Calculation
// Availability SLI
$successfulRequests = $metrics->getCounter('http_requests_total{status!~"5.."}');
$totalRequests = $metrics->getCounter('http_requests_total');
$availability = $successfulRequests / $totalRequests * 100;
// Target: 99.9%
// Latency SLI
$p99Latency = $metrics->getHistogramQuantile('http_request_duration_seconds', 0.99);
// Target: < 500ms
// Error Rate SLI
$errorRequests = $metrics->getCounter('http_requests_total{status=~"5.."}');
$errorRate = $errorRequests / $totalRequests * 100;
// Target: < 0.1%
SLO Targets
SLO Setting Guidelines
Function | Availability | Latency P99 | Error Rate
─────────────────|─────────────|─────────────|─────────────
Checkout | 99.99% | 500ms | 0.01%
Product Catalog | 99.9% | 200ms | 0.1%
Search | 99.9% | 300ms | 0.5%
Admin Panel | 99.9% | 1000ms | 1%
API | 99.95% | 300ms | 0.5%
SLO Configuration
# slo-config.yml
slos:
- name: checkout-availability
sli:
type: availability
query: 'sum(rate(http_requests_total{service="checkout",status!~"5.."}[5m])) / sum(rate(http_requests_total{service="checkout"}[5m]))'
target: 0.9999
window: 30d
- name: checkout-latency
sli:
type: latency
query: 'histogram_quantile(0.99, rate(http_request_duration_seconds_bucket{service="checkout"}[5m]))'
target: 0.5
window: 30d
SLO Monitoring
// Check SLO compliance
function checkSLO($sli, $target, $window) {
$actual = $this->metrics->query($sli, $window);
$compliance = $actual >= $target;
$errorBudgetRemaining = (1 - $actual) / (1 - $target) * 100;
return [
'slo_met' => $compliance,
'actual' => $actual,
'target' => $target,
'budget_remaining' => $errorBudgetRemaining
];
}
Error Budgets
Error Budget Concept
SLO: 99.9% availability over 30 days
Total minutes: 43,200
Error budget: 43,200 × 0.001 = 43.2 minutes
If 20 minutes downtime occurred:
Budget consumed: 20 / 43.2 = 46.3%
Budget remaining: 53.7%
Error Budget Policy
Budget Remaining | Policy
─────────────────|───────────────────────────
> 50% | Normal development
25-50% | Increased testing
10-25% | Feature freeze
< 10% | Reliability sprint
0% | Hard freeze
Error Budget Tracking
// Monthly error budget calculation
$sloTarget = 0.999;
$monthMinutes = 43200;
$totalBudget = $monthMinutes * (1 - $sloTarget); // 43.2 minutes
// Query actual downtime
$downtimeMinutes = $this->getDowntimeMinutes();
$remainingBudget = $totalBudget - $downtimeMinutes;
$consumedPercent = ($downtimeMinutes / $totalBudget) * 100;
// Alert based on consumption
if ($consumedPercent > 80) {
$this->alertService->critical(
"Error budget {$consumedPercent}% consumed"
);
}
Reliability Engineering
Reliability Practices
1. SLO-driven development
2. Error budget informed decisions
3. Post-incident reviews
4. Chaos engineering
5. Capacity planning
Post-Incident Review
# Incident Review Template
## Summary
- When: [timestamp]
- Duration: [minutes]
- Impact: [users affected]
- SLO impact: [budget consumed]
## Timeline
- [time] Alert fired
- [time] On-call acknowledged
- [time] Root cause identified
- [time] Fix deployed
- [time] Service restored
## Root Cause
[What went wrong]
## What Went Well
[Positive aspects]
## What Went Wrong
[Issues to address]
## Action Items
- [ ] [action] - [owner] - [due date]
Capacity Planning
Metrics to Track:
- Traffic growth rate
- Resource utilization trends
- SLO margin
- Error budget consumption rate
Planning:
- Project traffic 3 months out
- Ensure SLO targets achievable
- Scale before budget exhausted
- Review monthly
Quiz
1. What is the difference between SLI and SLO?
2. What happens when error budget reaches 0%?
3. What is the purpose of post-incident reviews?
Flashcards
Question
SLI vs SLO vs SLA?
Click to reveal answer
Answer
SLI: measure, "SLO": target, "SLA": contract
Question
Error budget at 0%?
Click to reveal answer
Answer
Hard freeze on features, reliability sprint only
Question
Post-incident review purpose?
Click to reveal answer
Answer
Learn from incidents, create action items to improve
Question
SLO window?
Click to reveal answer
Answer
Rolling period (30 days) for measuring compliance
Revision Notes
Key Takeaways
- 1. SLI measures actual performance, SLO sets target, SLA is contract
- 2. Error budgets guide feature vs reliability trade-offs
- 3. 0% budget triggers hard freeze on new features
- 4. Post-incident reviews drive continuous improvement
- 5. Capacity planning ensures SLO targets remain achievable
Interview Tips
- • Explain SLI/SLO/SLA with concrete Magento examples
- • Discuss error budget policies and their business impact
- • Describe reliability engineering practices and culture
Cheat Sheet
SLI/SLO/SLA:
SLI: Indicator (measure actual)
SLO: Objective (set target)
SLA: Agreement (contract)
Magento SLIs:
Availability: successful/total requests
Latency: P99 request duration
Error Rate: 5xx/total requests
Error Budget:
Budget = (1 - SLO) × time window
0% = Hard freeze, reliability sprint
Reliability:
SLO-driven development
Post-incident reviews
Chaos engineering
Capacity planning