Watchtower is a log handler for Amazon Web Services (AWS) CloudWatch Logs. It acts as a lightweight adapter between the Python `logging` system and CloudWatch Logs, using the `boto3` AWS SDK to aggregate logs into batches and send them to AWS. It is currently at version 3.4.0 and sees regular, although not strictly scheduled, releases with bug fixes and new features.
pip install watchtowerVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to integrate Watchtower with the Python `logging` module to send logs to AWS CloudWatch. It sets up a basic logger and a `CloudWatchLogHandler`, then sends a few example log messages. Ensure your AWS credentials and default region are configured either via environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`) or through the AWS CLI (`aws configure`) for `boto3` to automatically pick them up.
Review existing code that logs complex objects and adjust downstream log parsing or processing logic to account for `repr()` string representations. If the old behavior is desired, custom JSON serialization should be implemented before passing data to the logger.
Ensure your Python environment uses a version of `watchtower` that includes the fix. For your own code, replace `datetime.utcnow()` with `datetime.now(timezone.utc)`.
Be aware of the byte-length limit for log messages. If logs frequently contain long Unicode strings, consider pre-truncating or encoding them to ensure meaningful parts are preserved within the byte limit before they reach Watchtower.
If customizing log stream names, ensure they comply with CloudWatch naming rules and the expected `strftime` format. For high-volume applications or those using process pools, ensure the log stream name is unique per source using template variables like `{machine_name}/{program_name}/{logger_name}/{process_id}`.To reduce noise, set the logging level for `boto3`, `botocore`, and `urllib3` to `WARNING` or higher. For example: `logging.getLogger('boto3').setLevel(logging.WARNING)`, `logging.getLogger('botocore').setLevel(logging.WARNING)`, `logging.getLogger('urllib3').setLevel(logging.WARNING)`.Attach an AWS managed IAM policy (e.g., `CloudWatchLogsFullAccess` for testing, or a more restrictive custom policy with `logs:CreateLogGroup`, `logs:CreateLogStream`, `logs:PutLogEvents`) to the IAM role or user credentials used by your application. Refer to `boto3` credentials documentation for how credentials are loaded.
Upgrade `watchtower` to version 3.0.0 or higher to use the `region_name` argument directly. If upgrading is not an option, configure the AWS region via a `boto3.session.Session` object and pass it using the `session` argument to `CloudWatchLogHandler` (e.g., `CloudWatchLogHandler(session=boto3.Session(region_name='your-region'))`), or ensure the AWS region is configured through environment variables or AWS config files which `boto3` will pick up automatically.
Remove `region_name` from the `CloudWatchLogHandler` constructor. Ensure the AWS region is set via `AWS_REGION` environment variable or provide a `boto3.Session` configured with the desired region using the `boto3_session` argument. For example, `handler = CloudWatchLogHandler(log_group_name='my-group', boto3_session=boto3.Session(region_name='us-east-1'))` or simply rely on `AWS_REGION` environment variable if it's set.
pip install watchtower
from watchtower import WatchtowerLogHandler
Configure AWS credentials using environment variables (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), the shared credentials file (~/.aws/credentials), or an attached IAM role for EC2/ECS/EKS/Lambda.
Grant the IAM entity (user/role) permissions for `logs:CreateLogGroup`, `logs:CreateLogStream`, and `logs:PutLogEvents` on the relevant CloudWatch Logs resources.