Universal Pathlib is a Python library that extends the `pathlib.Path` interface to provide a unified, Pythonic way of interacting with a wide variety of backend filesystems, including local, cloud storage (S3, GCS, Azure), HTTP(S), and more, by leveraging `fsspec` backends. It is currently at version 0.3.10 and maintains a regular release cadence with frequent updates and bug fixes.
pip install universal-pathlibVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to use `UPath` for both a remote S3 bucket and a local file. It shows common operations like writing text, reading text, checking existence, and iterating directory contents. For S3, ensure `s3fs` is installed and AWS credentials are configured (e.g., via environment variables or `~/.aws/credentials`). For actual usage, replace `your-test-s3-bucket` with an accessible S3 bucket.
Migrate to subclassing `upath.extensions.ProxyUPath` for extending the API. Consult the official documentation for `ProxyUPath` usage examples.
Ensure the protocol in the path string matches the `UPath` subclass being instantiated, or use the base `UPath` class which dispatches to the correct subclass based on the protocol.
Prioritize using protocols with explicitly supported `UPath` implementations listed in the documentation. If using an untested protocol, thoroughly test its behavior and consider contributing an implementation if issues arise.
Ensure AWS credentials are correctly configured for the environment where the Python script is being run. Refer to the AWS documentation on configuring credentials for details (e.g., using environment variables, IAM roles, or the AWS CLI config file).
Ensure that the execution environment has the necessary AWS credentials configured according to AWS best practices (e.g., AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY environment variables, or an ~/.aws/credentials file).
pip install universal-pathlib
from upath import UPath
from upath import UPath
pip install s3fs