Registry / aws / obstore

obstore

JSON →
library0.11.1pypypi✓ verified 23d ago

obstore is a Python library providing a simple, high-throughput interface for various object storage services like Amazon S3, Google Cloud Storage, Azure Blob Storage, and S3-compliant APIs. It features both synchronous and asynchronous APIs, streaming downloads/uploads, and automatic multipart uploads for large files, powered by a Rust backend for performance. The current version is 0.9.2, with a frequent release cadence, often introducing minor updates and fixes.

pip install obstore
INSTALL
IMPORT
SIG · OBSTORE
O
obstore
awspythonv0.11.1
Install
2.0s avg
Import
14ms
Disk
31MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.11.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.014s · 33.9MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 2.0s · import 0.014s · 34MB
31MB installed
● package 31MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

obstore
import obstore as obs
MemoryStore
from obstore.store import MemoryStore
import obstore.MemoryStore
Store classes like MemoryStore, S3Store, GCSStore are located in the `obstore.store` submodule, not directly under `obstore`.
put
obs.put(store, 'path', b'data')
store.put('path', b'data')
All operations (put, get, list, delete, copy, etc.) are top-level functions exported by the `obstore` module, not methods of the store object itself.

This quickstart demonstrates how to initialize an in-memory object store and perform basic synchronous `put` and `get` operations. Note that operations like `put` and `get` are top-level functions within the `obstore` module, taking the store instance as their first argument. Asynchronous counterparts (e.g., `put_async`, `get_async`) are also available.

import obstore as obs from obstore.store import MemoryStore # Initialize an in-memory store for demonstration store = MemoryStore() # Define a file path and content file_path = "my_document.txt" file_content = b"Hello, obstore world!" # Put the object into the store obs.put(store, file_path, file_content) print(f"Object '{file_path}' put into store.") # Get the object from the store response = obs.get(store, file_path) retrieved_content = response.bytes() print(f"Retrieved content: {retrieved_content.decode()}") assert retrieved_content == file_content print("Content matches!") # Asynchronous operations are also available (requires an async context) # async def main(): # await obs.put_async(store, 'async_file.txt', b'async data') # res_async = await obs.get_async(store, 'async_file.txt') # print(f"Async retrieved: {res_async.bytes().decode()}") # import asyncio # asyncio.run(main())
Debug
Known issues
breakingSupport for Python 3.9 was deprecated in `obstore` version 0.9.0. Users on Python 3.9 should upgrade their Python environment.
fix
Upgrade Python to 3.10 or newer.
affects: >=0.9.0
breakingIn older versions (prior to 0.7.0), `S3Store.from_session()` and `S3Store._from_native()` were removed. Users should transition to using credential providers for S3 authentication.
fix
Refactor S3 store construction to use explicit credential providers or environment variable configuration instead of `from_session`.
affects: <0.7.0
breakingPath encoding behavior changed in version 0.8.2 to prevent unintentional double-encoding. Users must now ensure that paths provided to `obstore` are valid and correctly encoded.
fix
Review all path inputs to `obstore` functions and ensure they are properly formed (e.g., not already percent-encoded if they contain special characters).
affects: >=0.8.2
breakingIn version 0.5.0, the `container` parameter for `AzureStore`'s constructor was renamed to `container_name` and became a keyword-only argument. Using `container` will raise an error.
fix
Update `AzureStore` constructor calls to use `container_name='my-container'` instead of `container='my-container'`.
affects: >=0.5.0
gotchaUnlike many object storage libraries, `obstore`'s core operations (`put`, `get`, `list`, `delete`, `copy`, etc.) are top-level functions (e.g., `obs.put(store, ...)`) rather than methods on the store object (e.g., `store.put(...)`).
fix
Always call object storage operations as `obstore.<function_name>(store_instance, ...)`.
affects: All versions
gotchaWhen listing objects with `obstore.list(return_arrow=True)`, the optional `arro3-core` dependency is required. Without it, attempting to use this feature will fail.
fix
If using `return_arrow=True`, ensure `pip install obstore[arrow]` or `pip install arro3-core` is run.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'obstore'
The 'obstore' library is not installed in the Python environment where the code is being executed.
fix
Install the 'obstore' library using pip: `pip install obstore`
GenericError: Generic S3 error: Error performing list request: Received redirect without LOCATION, this normally indicates an incorrectly configured region.
When interacting with an S3 bucket, the AWS region was not specified or was configured incorrectly during `S3Store` initialization.
fix
Explicitly pass the correct `region` parameter when initializing `S3Store`, or ensure the `AWS_REGION` environment variable is correctly set. For example: `store = S3Store("your-bucket-name", region="us-west-2")`
NotImplementedError (when using mode="create" with put_async)
The 'mode' parameter for `put_async` (and `put`) requires specific capitalization (e.g., 'Create' instead of 'create'), or the requested put mode is not yet fully supported by the underlying S3 implementation for the given store configuration.
fix
Use the correct capitalization for `PutMode` values (e.g., `mode="Create"` or `mode="Overwrite"`). Refer to the `PutMode` documentation for supported values. For example: `await obs.put_async(store, key, buf, mode="Create")`
obstore.exceptions.UnauthenticatedError
The credentials used to access the object storage service are invalid, expired, or missing, leading to an authentication failure.
fix
Provide valid authentication credentials via environment variables (e.g., `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), configuration parameters during store initialization, or by setting up a custom credential provider.
obstore.exceptions.NotFoundError
The specified object or path does not exist in the configured object storage location.
fix
Verify that the `path` argument provided to operations like `get`, `head`, or `delete` correctly points to an existing object within the storage bucket.
Upgrade
Version history
0.11.1latest on PyPI · released Aug 21, 2026
Audit
Dependencies
boto3optionalRequired for `S3Store.from_session()` to integrate with AWS sessions.
arro3-coreoptionalRequired for `obstore.list(return_arrow=True)` to enable zero-copy Arrow integration for listing results.
Agent activity
31 hits · last 30 days
node
27
OpenAI (training)
1
Resources
obstore — pip install obstore · libregistry