CloudWatch Logs Overview
Amazon CloudWatch Logs centralizes, stores, and analyzes log data from AWS services and applications. Use it to collect logs, create metrics, search and visualize records, and trigger alarms or automated actions.
Key concepts
- Log group: A container for log streams with shared retention, encryption, and access settings.
- Log stream: Ordered sequence of log events from a single source (EC2 instance, Lambda function, container task).
- Log event: A timestamped record (message) within a log stream.
- Retention: How long CloudWatch keeps log data (1 day — indefinite). Set per log group.
- Subscription filter: Real-time stream of log events forwarded to destinations (Lambda, Kinesis Data Streams, Kinesis Data Firehose, or partner systems).
- Metric filter: Extracts numeric data from log events and emits CloudWatch metrics for alarms and dashboards.
- CloudWatch Logs Insights: Interactive, serverless log analytics with a query language for ad hoc queries and dashboards.
Sources & ingestion
- AWS services: Lambda, API Gateway, ELB, CloudTrail, RDS, VPC Flow Logs, Route 53, etc., can send logs natively.
- Agents: Unified CloudWatch Agent, CloudWatch Logs agent, or Fluent Bit (recommended for containers) collect host and application logs.
- SDK / PutLogEvents: Applications can publish logs directly via the PutLogEvents API or Fluent/Logback appenders.
- Firehose / Kinesis: Use subscription filters to route logs to S3, OpenSearch, or third-party tools.
Core features
- Real-time subscriptions: Send live log events to Lambda/Kinesis/Firehose for processing, transformation, or archival.
- Logs Insights: Fast queries for pattern search, aggregation, and visualization. Supports queries, saved queries, and dashboards.
- Metric extraction: Create metric filters that turn log patterns into CloudWatch metrics. Useful for SLOs and alarms.
- Encryption: Server-side encryption with AWS KMS (SSE-KMS) for log groups.
- Access control: IAM policies and resource-based policies for log group access.
- Cross-account & cross-region: Subscription filter destinations support cross-account delivery.
- Archive to S3: Export or use Firehose to persist logs to S3 for long-term retention and compliance.
Logs Insights — sample queries
- Count errors per service in last 1 hour:
fields @timestamp, @message
| filter @message like /ERROR|Exception/
| stats count() by bin(1m) - Top 10 slowest endpoints:
fields @timestamp, @message
| parse @message /(?GET|POST) (?[^ ]+) .* latency=(?\d+)/
| stats avg(lat) as avgLatency by path
| sort avgLatency desc
| limit 10
(Use the console or CLI to run Insights queries and save them for dashboards.)
Alarms & metrics from logs
- Create metric filters on a log group to emit metrics (e.g., ErrorCount).
- Use CloudWatch Alarms on those metrics to notify via SNS or trigger automated remediation.
- Composite alarms can combine metric alarms from logs and other metrics.
Best practices
- Name log groups consistently (service-environment-component).
- Set appropriate retention for each log group to control cost.
- Use log levels and structured logging (JSON) for easier parsing.
- Extract key fields with metric filters or use JSON parsing in Logs Insights.
- Use subscription filters to send logs to S3 for long-term archival or to analytics systems.
- Use VPC endpoints (AWS PrivateLink) to keep log traffic inside your network when required.
- Limit high-cardinality fields when creating metrics to avoid metric explosion and cost spikes.
- Monitor ApproximateAgeOfOldestLogEntry to detect consumer or ingestion delays.
Cost considerations
- Costs include ingestion (per GB ingested), storage (per GB-month retained), and Logs Insights query costs.
- Reduce costs by adjusting retention, batching logs in agents, and archiving infrequently accessed logs to S3.
Limits & operational notes
- Default limits (per account/region) apply for log events, log group count, and PutLogEvents requests. Request quota increases if necessary.
- PutLogEvents requires ordering and sequence tokens when used directly. Agents handle this for you.
- Use batching and the agent to improve throughput and reduce API call rates.
