Registry / aws / s3pathlib

s3pathlib

JSON →
library2.3.6pypypi✓ verified 22d ago

s3pathlib is a Python package that provides an intuitive, object-oriented programming (OOP) interface for manipulating AWS S3 objects and directories. Its API closely resembles the Python standard library's `pathlib` module, making S3 interactions feel familiar and Pythonic. The library is actively maintained, with the current version being 2.3.6, and receives regular minor updates.

pip install s3pathlib
INSTALL
IMPORT
SIG · S3PATHLIB
S
s3pathlib
awspythonv2.3.6
Install
5.1s avg
Import
1112ms
Disk
53MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2.3.6 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 1.150s · 54.7MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 5.1s · import 1.074s · 55MB
53MB installed
● package 53MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

S3Path
from s3pathlib import S3Path
context
from s3pathlib import context
Used for attaching a boto3 session or configuring AWS credentials.

This quickstart demonstrates how to initialize `s3pathlib` with AWS credentials, create an `S3Path` object, write content to S3, and read it back. It shows how to use `context.attach_boto_session` for explicit credential management.

import boto3 import os from s3pathlib import S3Path, context # Configure AWS credentials (e.g., from environment variables or a profile) # In a real application, consider using AWS IAM roles or proper credential management. aws_region = os.environ.get("AWS_REGION", "us-east-1") aws_profile = os.environ.get("AWS_PROFILE", None) # Attach a boto3 session for s3pathlib to use if aws_profile: session = boto3.session.Session(region_name=aws_region, profile_name=aws_profile) else: session = boto3.session.Session(region_name=aws_region) context.attach_boto_session(session) # Define an S3 path object bucket_name = os.environ.get("S3_BUCKET_NAME", "your-test-bucket-12345") s3_path = S3Path(bucket_name, "my-folder", "hello.txt") # Example: Write text to S3 print(f"Writing to {s3_path.uri}...") s3_path.write_text("Hello, s3pathlib!") print("Content written.") # Example: Read text from S3 if s3_path.exists(): content = s3_path.read_text() print(f"Content read from S3: '{content}'") # Example: Check if a folder exists and list its contents s3_dir = S3Path(bucket_name, "my-folder/") print(f"Checking if {s3_dir.uri} exists: {s3_dir.exists()}") # Clean up (optional) # s3_path.delete() # Deletes the object # s3_dir.delete_dir() # Deletes all objects under the prefix (careful!) # Detach the boto3 session when done (optional, for explicit resource management) context.detach_boto_session()
Debug
Known issues
gotchaS3Path objects are immutable. Operations that modify the S3 object (like `write_text()` or `write_bytes()`) will return a *new* S3Path object, especially in S3 versioning-enabled buckets. Always reassign the result if you need to work with the updated path object.
fix
Always capture the return value of write operations: `s3_path = s3_path.write_text('new content')`.
affects: All versions
gotchaWrite operations (`write_text()`, `write_bytes()`) will silently overwrite existing files by default. There is no automatic error or warning if the target S3 object already exists.
fix
To prevent accidental overwrites, explicitly check for file existence using `if not s3_path.exists():` before performing write operations.
affects: All versions
gotchaS3 does not have a true directory concept; `s3pathlib` provides a 'logical' or 'soft' directory abstraction. A path ending with `/` is treated as a directory. Understanding this distinction is crucial for directory-related operations.
fix
Be mindful of trailing slashes (`/`) when defining S3 paths to denote directories, e.g., `S3Path('bucket', 'my-folder/')`. Use methods like `.to_dir()` if unsure.
affects: All versions
gotchaWhile `s3pathlib` mimics the `pathlib.Path` API, `S3Path` is not a direct subclass of `pathlib.Path`. This means `isinstance(s3_path_object, pathlib.Path)` checks will return `False` and code expecting a `pathlib.Path` might break.
fix
Avoid `isinstance(obj, pathlib.Path)` checks if you intend to support `S3Path`. Instead, check for `isinstance(obj, S3Path)` or rely on duck typing if only standard pathlib methods are used.
affects: All versions
gotchaImplicit `boto3` session usage can lead to unexpected AWS credential behavior. If you don't explicitly attach a `boto3` session using `context.attach_boto_session()`, `s3pathlib` will rely on `boto3`'s default credential chain (environment variables, shared credential file, IAM roles), which might not be what you intend.
fix
Explicitly configure and attach your `boto3` session using `from s3pathlib import context; context.attach_boto_session(your_boto3_session)` to ensure predictable AWS authentication and region behavior.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 's3pathlib'
The 's3pathlib' package is not installed in the current Python environment.
fix
Install the package using pip: `pip install s3pathlib`
botocore.exceptions.NoCredentialsError: Unable to locate credentials
The AWS SDK (boto3), which s3pathlib uses internally, could not find valid AWS credentials to authenticate with S3.
fix
Configure AWS credentials using environment variables (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), AWS CLI config files (~/.aws/credentials), or by passing a boto3 session/client explicitly to S3Path.
FileNotFoundError: S3 object 's3://your-bucket/non-existent-key' not found
The specified S3 object path does not exist in the bucket, or the user lacks permission to access it, causing s3pathlib to raise a FileNotFoundError similar to local pathlib.
fix
Ensure the S3 bucket and key path are correct and that the AWS credentials have `s3:GetObject` permission for the target object. Verify the object's existence in S3.
s3pathlib.exceptions.S3PathlibError: Cannot rmdir non-empty S3 prefix: 's3://your-bucket/my_folder/'
The user attempted to delete an S3 'directory' (object prefix) using `rmdir()`, but the prefix still contains objects, as `rmdir()` is designed to only remove empty prefixes.
fix
Before calling `rmdir()`, ensure the S3 prefix is empty by deleting all objects within it, or use `delete_dir()` if you intend to recursively delete all contents.
AttributeError: property 'name' of 'S3Path' object has no setter
S3Path objects are immutable, similar to pathlib.Path objects, meaning you cannot directly modify attributes like `name`, `stem`, or `parent` after creation.
fix
To change path components, use methods like `with_name()`, `with_stem()`, `joinpath()`, or create a new `S3Path` object with the desired modifications.
Upgrade
Version history
2.3.6latest on PyPI · released Aug 12, 2025
Audit
Dependencies
boto3requiredRequired for all interactions with AWS S3. It's an implicit dependency that s3pathlib leverages for its underlying AWS calls.
Agent activity
20 hits · last 30 days
node
14
OpenAI (training)
3
Resources