Registry / aws / ibm-cos-sdk-s3transfer

ibm-cos-sdk-s3transfer

JSON →
library2.16.2pypypi✓ verified 21d ago

The `ibm-cos-sdk-s3transfer` library provides a high-level, multi-part, multi-threaded Amazon S3-like transfer manager optimized for IBM Cloud Object Storage (COS). It handles efficient uploading and downloading of files, especially large objects, managing part sizing and concurrency. It is currently at version 2.16.1 and receives regular updates, typically with minor feature releases every few months and patch releases for bug fixes.

pip install ibm-cos-sdk
INSTALL
IMPORT
SIG · IBM-COS-SDK-S3TRAN
I
ibm-cos-sdk-s3transfer
awspythonv2.16.2
Install
5.0s avg
Import
894ms
Disk
29MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2.16.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
installs and imports cleanly · install 0.0s · import 0.938s · 30.5MB
glibc
py 3.103.910 runs
installs and imports cleanly · install 5.0s · import 0.850s · 31MB
29MB installed
● package 29MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

TransferManager
from ibm_boto3.s3.transfer import TransferManager
from ibm_cos_sdk.s3.transfer import TransferManager

This quickstart demonstrates how to initialize the IBM COS client and then use the `TransferManager` to upload and download a file. It expects IBM COS credentials and bucket name to be set as environment variables.

import os import ibm_cos_sdk from ibm_cos_sdk.s3.transfer import TransferManager # Configure IBM COS credentials and endpoint # Ensure these environment variables are set: # IBM_COS_ENDPOINT, IBM_COS_API_KEY_ID, IBM_COS_SERVICE_INSTANCE_ID, IBM_COS_BUCKET ENDPOINT = os.environ.get('IBM_COS_ENDPOINT', 'https://s3.us.cloud-object-storage.appdomain.cloud') API_KEY_ID = os.environ.get('IBM_COS_API_KEY_ID', '') SERVICE_INSTANCE_ID = os.environ.get('IBM_COS_SERVICE_INSTANCE_ID', '') BUCKET_NAME = os.environ.get('IBM_COS_BUCKET', 'my-test-bucket') if not all([API_KEY_ID, SERVICE_INSTANCE_ID, BUCKET_NAME]): print("Error: IBM COS environment variables not set. Please set IBM_COS_API_KEY_ID, IBM_COS_SERVICE_INSTANCE_ID, IBM_COS_BUCKET.") exit(1) # Initialize COS client cos_client = ibm_cos_sdk.client( 's3', ibm_api_key_id=API_KEY_ID, ibm_service_instance_id=SERVICE_INSTANCE_ID, config=ibm_cos_sdk.config.Config(signature_version='oauth'), endpoint_url=ENDPOINT ) # Create a TransferManager instance transfer_manager = TransferManager(cos_client) # --- Example: Upload a file --- local_file_path = 'local_file.txt' key_name = 'remote_object.txt' # Create a dummy local file for upload with open(local_file_path, 'w') as f: f.write('Hello, IBM COS Transfer Manager!') print(f"Uploading {local_file_path} to {BUCKET_NAME}/{key_name}...") try: future = transfer_manager.upload(local_file_path, BUCKET_NAME, key_name) future.result() # Wait for the transfer to complete print(f"Successfully uploaded {key_name}") except Exception as e: print(f"Upload failed: {e}") # --- Example: Download a file --- download_path = 'downloaded_file.txt' print(f"Downloading {BUCKET_NAME}/{key_name} to {download_path}...") try: future = transfer_manager.download(BUCKET_NAME, key_name, download_path) future.result() # Wait for the transfer to complete print(f"Successfully downloaded {key_name} to {download_path}") except Exception as e: print(f"Download failed: {e}") # Clean up local files (optional) os.remove(local_file_path) os.remove(download_path) # You can also close the transfer manager (optional) transfer_manager.shutdown()
Debug
Known issues
gotchaThe `ibm-cos-sdk-s3transfer` library is generally a transitive dependency of `ibm-cos-sdk`. For typical usage, you should install `ibm-cos-sdk` and import `TransferManager` from `ibm_cos_sdk.s3.transfer`, not directly from `ibm_cos_sdk_s3transfer`.
fix
Install `ibm-cos-sdk` and use `from ibm_cos_sdk.s3.transfer import TransferManager`.
affects: All versions
gotchaThis library requires Python 3.8 or newer. Using older Python versions will result in installation or runtime errors.
fix
Ensure your Python environment is 3.8 or newer. Upgrade Python if necessary.
affects: All versions
gotchaIBM Cloud Object Storage (COS) requires specific configuration parameters that differ from standard AWS S3. Notably, you must provide `ibm_api_key_id`, `ibm_service_instance_id`, and a COS-specific `endpoint_url` when initializing the client.
fix
Refer to IBM COS documentation for correct endpoint URLs and ensure all required `ibm_` prefixed parameters are passed to `ibm_cos_sdk.client('s3', ...)`.
affects: All versions
gotchaWhen transferring very large files or managing many concurrent transfers, `TransferManager` can consume significant system resources (memory, CPU). Monitor resource usage and adjust `max_concurrency` or `multipart_threshold` parameters if encountering issues.
fix
For fine-grained control, consider initializing `TransferManager` with custom `transfer_config` options. For example, `TransferConfig(max_concurrency=N)` to limit concurrent parts or `TransferConfig(multipart_threshold=M_bytes)` to adjust the multipart upload threshold.
affects: All versions
Upgrade
Version history
2.16.2latest on PyPI · released Apr 14, 2026
Audit
Dependencies
ibm-cos-sdkrequiredProvides the IBM COS client and exposes the TransferManager for direct use. This package is typically a transitive dependency of `ibm-cos-sdk`.
s3transferrequiredThis library is a fork and extension of the upstream `s3transfer` library from `boto3`.
Agent activity
23 hits · last 30 days
node
22
Resources
ibm-cos-sdk-s3transfer — pip install ibm-cos-sdk-s3transfer · libregistry