Install & Compatibility
Where this runs
tested against v2026.7.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.925 runs
installs and imports cleanly · install 0.0s · import 0.320s · 87.7MB
glibcpy 3.10–3.925 runs
installs and imports cleanly · install 16.4s · import 0.281s · 88MB
84MB installed
● package 84MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
open
✓ from fsspec import open
✗ import fsspec
get_filesystem_class
✓ from fsspec import get_filesystem_class
✗ import fsspec
AbstractFileSystem
✓ from fsspec import AbstractFileSystem
✗ import fsspec
Open a remote HTTP file, read/list a local filesystem, and demonstrate the memory filesystem — all with the same API.
import os
import fsspec
# 1. Open any URL transparently (protocol auto-detected from URL)
with fsspec.open(
"https://raw.githubusercontent.com/fsspec/filesystem_spec/master/README.md",
"rt",
) as f:
first_line = f.readline()
print("README first line:", first_line.strip())
# 2. Use fsspec.filesystem() for repeated operations on one backend
fs = fsspec.filesystem("memory") # in-process, no I/O
fs.mkdir("/mydir")
with fs.open("/mydir/hello.txt", "wt") as f:
f.write("Hello from fsspec!")
with fs.open("/mydir/hello.txt", "rt") as f:
print(f.read())
print("Files:", fs.ls("/mydir"))
# 3. S3 example (needs pip install fsspec[s3])
# Uses env-var credentials — safe pattern for agents
# aws_key = os.environ.get('AWS_ACCESS_KEY_ID', '')
# aws_secret = os.environ.get('AWS_SECRET_ACCESS_KEY', '')
# fs_s3 = fsspec.filesystem('s3', key=aws_key, secret=aws_secret)
# print(fs_s3.ls('my-bucket/'))
# 4. Zarr-style key-value mapping over any backend
mapper = fsspec.get_mapper("memory://zarr-root/")
mapper["chunk-0"] = b"\x00" * 128
print("Mapper keys:", list(mapper))
Debug
Known issues
gotchaFilesystem instances are cached singletons. fsspec.filesystem('s3', key=K1) and fsspec.filesystem('s3', key=K2) with different credentials may return the same cached instance if arguments hash identically. This can cause silent credential cross-contamination.fixCall fs_class.clear_instance_cache() when you need a fresh instance, or pass skip_instance_cache=True to fsspec.filesystem().
affects: all
breakingCalling sync fsspec methods (e.g. fs.ls()) from inside a running asyncio event loop raises NotImplementedError: 'Calling sync() from within a running loop'. This affects Jupyter notebooks, FastAPI handlers, and async frameworks.fixInside an async context, instantiate with asynchronous=True and await the private _method coroutines (e.g. await fs._ls()), or use fsspec.asyn.fsspec_loop context manager.
affects: all async implementations (s3fs, gcsfs, adlfs, HTTPFileSystem)
gotchafsspec.open() returns an OpenFile placeholder — the remote file is NOT opened until you enter a `with` block. Calling .read() on the raw OpenFile object without a context manager will fail.fixAlways use `with fsspec.open(...) as f: f.read(...)` or call of.open() explicitly and close when done.
affects: all
gotchaCloud backend packages (s3fs, gcsfs, adlfs) are NOT included in the base fsspec install. Attempting to use fsspec.filesystem('s3') without s3fs raises an ImportError with a hint, but the hint may be missed in automated pipelines.fixInstall the required extra: pip install fsspec[s3], fsspec[gcs], or the specific package. Use fsspec.available_protocols() to check what is installed.
affects: all
gotchaAsync filesystem instances (s3fs, gcsfs) are incompatible with os.fork() (used by PyTorch DataLoader, multiprocessing). After fork, cached instances hold references to dead event loops, causing hangs or cryptic errors.fixUse multiprocessing with spawn context, or call fs_class.clear_instance_cache() inside the worker initializer after forking.
affects: all async implementations
deprecatedPassing `trim` to fsspec.spec.AbstractBufferedFile is deprecated.fixRemove the trim argument from any AbstractBufferedFile instantiation.
affects: >=0.9.0
breakingfsspec.asyn.maybe_sync was removed. Older pinned versions of s3fs (<=0.5.2) and other ecosystem libraries that imported maybe_sync will break on recent fsspec.fixUpgrade s3fs, gcsfs, and adlfs to current releases that are compatible with the same fsspec CalVer month.
affects: fsspec>=2021.x removes maybe_sync; s3fs<=0.5.2 affected
Upgrade
Version history
2026.7.0latest on PyPI · released Jul 28, 2026
Audit
Dependencies
s3fsoptionalRequired for s3:// protocol (AWS S3). Install via fsspec[s3] or pip install s3fs.
gcsfsoptionalRequired for gs:// or gcs:// protocol (Google Cloud Storage). Install via fsspec[gcs].
adlfsoptionalRequired for abfs:// or az:// protocol (Azure Blob / ADLS Gen2).
aiohttpoptionalRequired for HTTP/HTTPS async filesystem (HTTPFileSystem). Included in base install.
paramikooptionalRequired for sftp:// or ssh:// protocol. Install via fsspec[ssh].