Install & Compatibility
Where this runs
tested against v0.19.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 0.591s · 63.1MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 4.2s · import 0.517s · 64MB
62MB installed
● package 62MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
TransferManager
✓ from s3transfer.manager import TransferManager
Low-level public API. Prefer boto3 client injection (s3_client.upload_file / download_file) for the most stable surface.
TransferConfig
✓ from boto3.s3.transfer import TransferConfig
✗ from s3transfer.manager import TransferConfig
boto3's TransferConfig adds aliases (max_concurrency, max_io_queue) and preferred_transfer_client on top of s3transfer's internal TransferConfig. Using the raw s3transfer one skips those aliases and the CRT dispatch logic.
S3Transfer
✓ from boto3.s3.transfer import S3Transfer
✗ import s3transfer; s3transfer.S3Transfer(...)
S3Transfer lives in boto3.s3.transfer, not directly on the s3transfer top-level namespace. Only classes explicitly documented in the boto3 S3 customization reference are considered public stable API.
BaseSubscriber
✓ from s3transfer.subscribers import BaseSubscriber
Subclass this to implement on_queued, on_progress, and on_done progress callbacks.
RetriesExceededError
✓ from boto3.exceptions import RetriesExceededError
✗ from s3transfer.exceptions import RetriesExceededError
boto3 wraps s3transfer's RetriesExceededError in its own exception for backwards compatibility. Catching the s3transfer version will miss errors raised through the boto3 S3Transfer shim.
Upload and download a file using TransferManager via the stable boto3.s3.transfer interface, with a custom TransferConfig.
import os
import boto3
from boto3.s3.transfer import TransferConfig, S3Transfer
# Credentials resolved from env, ~/.aws/credentials, or IAM role
aws_access_key = os.environ.get('AWS_ACCESS_KEY_ID', '')
aws_secret_key = os.environ.get('AWS_SECRET_ACCESS_KEY', '')
region = os.environ.get('AWS_DEFAULT_REGION', 'us-east-1')
bucket = os.environ.get('S3_BUCKET', 'my-bucket')
client = boto3.client(
's3',
region_name=region,
aws_access_key_id=aws_access_key or None,
aws_secret_access_key=aws_secret_key or None,
)
config = TransferConfig(
multipart_threshold=8 * 1024 * 1024, # 8 MB
max_concurrency=10,
num_download_attempts=5,
use_threads=True,
# Force classic manager; use 'auto' to allow CRT when awscrt is installed
preferred_transfer_client='classic',
)
transfer = S3Transfer(client, config)
# Upload
transfer.upload_file(
'/tmp/example.txt',
bucket,
'uploads/example.txt',
extra_args={'ContentType': 'text/plain'},
)
# Download
transfer.download_file(
bucket,
'uploads/example.txt',
'/tmp/example_downloaded.txt',
)
print('Transfer complete')
Debug
Known issues
breakings3transfer is explicitly NOT GA. Interfaces can break between minor versions (e.g. 0.x.0 → 0.y.0). Always pin to a minor version in production requirements (e.g. s3transfer>=0.16.0,<0.17.0).fixPin: s3transfer>=0.16.0,<0.17.0. For a more stable API surface, use the methods injected into the boto3 S3 client (s3_client.upload_file, s3_client.download_file) which are documented as stable.
affects: <1.0.0 (all current releases)
breakingMismatched awscrt version causes ImportError at import time. If awscrt is installed (e.g. pulled in by boto3[crt]) but is not at the version expected by the installed s3transfer/boto3, 'from boto3.s3.transfer import TransferConfig' raises ImportError: cannot import name 'S3ResponseError' from 'awscrt.s3'.fixUpgrade all three in lockstep: pip install --upgrade boto3 botocore s3transfer. If awscrt must stay pinned, set preferred_transfer_client='classic' or uninstall awscrt entirely.
affects: Observed around boto3 1.33.x / s3transfer 0.8.x; can recur on any awscrt mismatch
breakingTransferConfig parameter names differ between s3transfer.manager.TransferConfig and boto3.s3.transfer.TransferConfig. The boto3 version maps max_concurrency→max_request_concurrency and max_io_queue→max_io_queue_size. Passing the raw s3transfer config to boto3 code (or vice versa) silently ignores aliased parameters.fixAlways import TransferConfig from boto3.s3.transfer when working with the boto3 S3Transfer class.
affects: All versions
gotchaS3Transfer requires either a boto3 client OR a TransferManager instance — not both, not neither. Passing manager together with client, config, or osutil raises ValueError. Passing nothing also raises ValueError.fixUse S3Transfer(client=my_boto3_client) or S3Transfer(manager=my_manager), never mixing both argument groups.
affects: All versions
gotchaCRC32 is now the default checksum algorithm for uploads. If the receiving side or downstream tooling does not handle the x-amz-checksum-crc32 header (e.g., some S3-compatible stores), uploads may fail or produce unexpected validation errors.fixTo disable automatic checksums, ensure botocore config request_checksum_calculation is set to 'when_required', or explicitly pass ChecksumAlgorithm in extra_args.
affects: >=0.10.0 (CRC32 default introduced)
gotchaThread-local context (AWS X-Ray segments, OpenTelemetry spans, contextvars) is lost inside transfer worker threads. The transfer manager uses a ThreadPoolExecutor internally; tracing SDKs that rely on thread-local or context-variable storage will not see the parent trace context in worker threads.fixUse TransferConfig(use_threads=False) for single-threaded execution (loses concurrency) or propagate context manually via subscriber callbacks.
affects: All versions
gotchadownload_file and upload_file require filename to be a str or os.PathLike. Passing a raw bytes path or any other object raises ValueError immediately without touching S3.fixConvert pathlib.Path objects (accepted via os.fspath internally) or ensure the filename argument is a str. Example: transfer.download_file(bucket, key, str(path_obj)).
affects: All versions
breakingWhen using `upload_file` or `download_file`, a `FileNotFoundError` will be raised if the specified local file (for upload) or the directory path (for download) does not exist.fixEnsure the local file exists at the specified `filename` path before calling `upload_file`. For `download_file`, ensure that parent directories for the target `filename` path exist before the operation.
affects: All versions
breakingThe local file specified for upload_file or download_file does not exist. This results in a FileNotFoundError when the transfer manager attempts to access the file's metadata (e.g., size) or content.fixEnsure the source file exists on the local filesystem before calling upload_file. For download_file, ensure the destination directory is writable and accessible.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 's3transfer'
The `s3transfer` package is not installed in the current Python environment or is not accessible, even though it's typically a dependency of `boto3`.
fixInstall `boto3` (which includes `s3transfer` as a dependency) or explicitly install `s3transfer`.
pip install boto3
# Or, if only s3transfer is needed:
pip install s3transfer
A client error (InvalidRequest) occurred when calling the PutObject operation: Content-MD5 HTTP header is inconsistent, please remove and try again.
When using `boto3`'s high-level S3 transfer methods (`upload_file`, `upload_fileobj`) which utilize `s3transfer`, the library or S3 service handles checksums automatically; explicitly providing a `ContentMD5` header in `ExtraArgs` can conflict.
fixAvoid manually providing the `ContentMD5` header in `ExtraArgs` when using `boto3`'s high-level S3 transfer methods.
s3_client.upload_file(
'local_file.txt', 'my-bucket', 'remote_file.txt'
# Remove ExtraArgs={'ContentMD5': 'your_md5_hash_base64'}
) TypeError: __init__() got an unexpected keyword argument 'some_invalid_argument'
The `boto3.s3.transfer.TransferConfig` class (part of `s3transfer`'s interface via `boto3`) was initialized with an argument that does not exist or is misspelled for the installed version of `boto3`/`s3transfer`.
fixReview the official `boto3` documentation for the `TransferConfig` class to ensure all arguments used are valid for your installed version, correcting any typos or removing deprecated arguments.
from boto3.s3.transfer import TransferConfig
# Example with valid arguments
config = TransferConfig(
multipart_threshold=1024 * 25, # 25 MB
max_concurrency=10,
use_threads=True
)
# s3_client.upload_file('local_file.txt', 'my-bucket', 'remote_file.txt', Config=config) ModuleNotFoundError: No module named 'awscrt'
AWS CRT-accelerated transfers, an optional feature, require the `awscrt` Python package to be installed in the environment.
fixInstall the `awscrt` package in your Python environment to enable CRT acceleration.
pip install awscrt
Upgrade
Version history
0.19.2latest on PyPI · released Jul 22, 2026
Audit
Dependencies
botocorerequiredRequired core AWS SDK primitives; s3transfer requires botocore>=1.37.4,<2.0a.0
awscrtoptionalOptional CRT-based high-throughput transfer backend; auto-selected when installed and running on an optimized instance type