CloudWatch Metrics and Alarms

CloudWatch Metrics is a time-series database for operational data. Every AWS service publishes standard metrics automatically—EC2 publishes CPU utilization, network bytes in/out, and disk read/write ops at 1-minute intervals. These metrics are free and available immediately.

Dimensions are key-value pairs that filter and group metrics. An EC2 instance metric might have dimensions InstanceId=i-0abc123 and AutoScalingGroupName=web-server-group. Dimensions let you view metrics per instance, per group, or across all instances. You can filter a dashboard to show only production instances or compare dev vs prod performance.

Custom metrics extend CloudWatch beyond AWS-provided data. Use the PutMetricData API to publish application-specific measurements. For example, a payment processing service might publish metrics for successful transactions per second, average payment amount, and retry count. Custom metrics cost $0.30 per metric per month for high-resolution metrics or $0.10 for standard.

Metric math enables derived calculations. Instead of alerting on raw CPU utilization, you might calculate the average CPU across an Auto Scaling group: AVG(m1) where m1 is the CPU metric filtered by group. Metric math supports expressions like SEARCH(), DIFF(), and FORECAST() for trend analysis.

High-resolution metrics sub-minute intervals (1-second to 1-second) are available for applications requiring sub-minute granularity. A real-time gaming service might publish player latency at 1-second resolution to detect micro-bursts that 1-minute averages would miss.

Consider a scenario: an API gateway publishes a custom metric called RequestLatency. You add dimensions for Environment (prod, staging) and Region (us-east-1, eu-west-1). Now you can view latency per environment per region, set different thresholds per environment, and build a dashboard comparing cross-region performance.

CloudWatch Alarms transition metrics from passive monitoring to active response. When a metric breaches a threshold, the alarm state changes and triggers actions—SNS notifications, Auto Scaling policies, or Lambda functions.

Threshold alarms are the simplest form. An alarm on EC2 CPUUtilization > 80% for 3 out of 5 data points triggers a notification. The evaluation period and datapoints-to-alarm settings prevent flapping—a single spike doesn't trigger an alert, but sustained high utilization does. Configure the treat_missing_data parameter carefully: missing treats missing data as breaching (aggressive), notBreaching treats it as healthy (conservative), and breaching is the default.

Anomaly detection uses machine learning to model expected metric behavior. Enable anomaly detection on a metric and CloudWatch learns daily and weekly patterns. An alarm triggers when the actual value deviates from the predicted band by a configurable threshold (e.g., 2 standard deviations). This is powerful for metrics with natural variation—CPU usage spikes during business hours and drops overnight. A static threshold of 70% would false-alarm every morning; anomaly detection adapts to the pattern.

Composite alarms combine multiple alarms using AND, OR, and NOT logic. Create individual alarms for high CPU, high memory, and high network utilization. A composite alarm fires only when 2 out of 3 are in ALARM state, reducing noise from isolated spikes. Composite alarms can also suppress actions—when the composite alarm is in ALARM state, suppress all child alarm notifications to prevent alert storms.

Alarm actions extend beyond notifications. An alarm can trigger an Auto Scaling scale-out policy, execute a Systems Manager automation to restart a service, or invoke a Lambda function that creates a Jira ticket. For auto-recovery, EC2 instance status check alarms can automatically recover failed instances by migrating them to new hardware while preserving the instance ID and private IP.

CloudWatch Dashboards aggregate metrics, alarms, and logs into a single operational view. A well-designed dashboard follows the hierarchy: summary at the top (service health, error budget status), detailed metrics in the middle (per-service latency, throughput, error rates), and infrastructure at the bottom (CPU, memory, disk).

Dashboard design principles include: color-coding (green/yellow/red for health), using widgets effectively (line charts for trends, number widgets for current values, alarm widgets for status), and organizing by audience. An executive dashboard shows high-level SLOs; an engineering dashboard shows per-service metrics; an on-call dashboard shows the most actionable information.

CloudWatch Synthetics creates canaries—configurable scripts that run on a schedule to monitor endpoints and APIs. A canary can simulate a user login flow, verify a checkout process, or check API response codes. Canaries run from multiple AWS regions and report availability and latency metrics to CloudWatch.

A practical canary for an e-commerce site might: load the homepage, search for a product, add it to cart, proceed to checkout (stopping before payment), and verify each step returns 200 and loads within 3 seconds. If any step fails or is slow, the canary creates a CloudWatch alarm and sends a notification.

Synthetics canary features include: screenshots on failure (visual regression detection), HAR files for network analysis, Lambda layers for custom dependencies, and VPC support for private endpoints. Canaries can run from 1-minute to 15-minute intervals, with costs of approximately $0.0012 per canary run.

Integrate canaries with Route 53 health checks for automated DNS failover. If the canary in us-east-1 detects the service is down, Route 53 can automatically route traffic to us-west-2. This creates end-to-end monitoring: synthetic checks verify external availability while internal metrics verify component health.