Registry /
aws / snakemake-storage-plugin-s3
Install & Compatibility
Where this runs
tested against v0.2.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.000s · 52.4MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 4.8s · import 0.000s · 53MB
52MB installed
● package 52MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
S3Storage
✓ This plugin is automatically discovered by Snakemake when installed, and is used via 's3://' URIs in your Snakefile, not by direct Python import of a specific class for usage.
You do not typically import classes directly from this library into your Python code. Snakemake discovers and utilizes the plugin automatically when it's installed in the same environment.
This quickstart demonstrates how to use the S3 storage plugin in a `Snakefile`. Snakemake automatically recognizes `s3://` prefixes for input and output files when the plugin is installed. Ensure your S3 credentials are configured in your environment or via standard `boto3` configuration methods (e.g., `~/.aws/credentials`). For S3-compatible services like MinIO, `AWS_ENDPOINT_URL` is typically required.
# file: Snakefile
# Set S3 credentials via environment variables for a runnable example
# export AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY"
# export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_KEY"
# export AWS_ENDPOINT_URL="http://localhost:9000" # Optional, for MinIO or other S3-compatible storages
rule all:
input: "s3://my-test-bucket/output.txt"
rule generate_s3_file:
output: "s3://my-test-bucket/output.txt"
shell:
"echo 'Hello Snakemake S3!' > {output}"
# To run this example:
# 1. Ensure snakemake and snakemake-storage-plugin-s3 are installed.
# 2. Set your S3 credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, etc.)
# or AWS_ENDPOINT_URL for compatible S3 services.
# 3. Create 'my-test-bucket' in your S3 service.
# 4. Navigate to the directory containing this Snakefile and run: snakemake
Debug
Known issues
gotchaS3 object keys cannot contain `..` (double-dot) for path traversal. The plugin attempts to normalize paths, but directly using `..` in S3 URIs will lead to errors.fixEnsure all S3 object paths used in your Snakefile are absolute or relative paths without `..` segments. For example, use `s3://bucket/dir/file.txt` instead of `s3://bucket/dir/../file.txt`.
affects: All versions prior to 0.3.5, and potentially current if complex paths are not normalized correctly.
gotchaWhen using non-AWS S3 compatible services (e.g., MinIO), `boto3` might automatically attach checksums to uploaded files which can cause issues or unexpected behavior.fixUpgrade to version 0.3.3 or newer, which includes a workaround for this `boto3` behavior. If issues persist, consult `boto3` documentation or your S3 provider for specific configuration requirements regarding checksums.
affects: All versions prior to 0.3.3.
gotchaSnakemake's storage plugin interface versioning requires the plugin to be compatible. If Snakemake is updated to a new major interface version, the S3 plugin might need an update as well.fixAlways keep the `snakemake-storage-plugin-s3` library updated to its latest version to ensure compatibility with recent Snakemake releases and the `snakemake-interface-storage-plugins`.
affects: Any version of the plugin that is not aligned with the Snakemake core's storage interface version (e.g., plugin 0.3.2 adapted to interface 4.0).
breakingIn earlier versions, nested directories were not always retrieved correctly, leading to missing files or `FileNotFoundError`.fixUpgrade to version 0.3.4 or newer to fix issues with nested directory retrieval and `FileExists` errors when getting directories.
affects: Versions prior to 0.2.12 (for nested directories) and 0.3.4 (for `FileExists` bug).
Upgrade
Version history
0.3.6latest on PyPI · released Sep 11, 2025
Audit
Dependencies
snakemakerequiredThis is a plugin for Snakemake and requires Snakemake to be installed.
boto3requiredUsed for interacting with the S3 API.
snakemake-interface-storage-pluginsrequiredProvides the necessary interface for Snakemake storage plugins.