Registry / serialization / xopen
library2.1.0pypypi✓ verified 23d ago

This Python module provides an `xopen` function that works like Python's built-in `open` function but also transparently deals with compressed files. `xopen` selects the most efficient method for reading or writing a compressed file, often leveraging external tools like `pigz` for parallel processing. It supports gzip (.gz), bzip2 (.bz2), xz (.xz), and optionally Zstandard (.zst) formats. The library is actively maintained, currently at version 2.0.2, and has a consistent release cadence.

pip install xopen
INSTALL
IMPORT
SIG · XOPEN
X
xopen
serializationpythonv2.1.0
Install
1.9s avg
Import
115ms
Disk
18MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2.1.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.124s · 20.8MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 1.9s · import 0.106s · 21MB
18MB installed
● package 18MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

xopen
from xopen import xopen

Demonstrates reading from a gzipped file (auto-detected) and writing to an xz-compressed file with a specified compression level. `xopen` behaves like the built-in `open` but handles compression transparently based on file extension or magic numbers.

from xopen import xopen import os # Create a dummy gzipped file with open('example.txt', 'w') as f: f.write('Hello, world!\nThis is a test.') import gzip with open('example.txt', 'rb') as f_in: with gzip.open('example.txt.gz', 'wb') as f_out: f_out.write(f_in.read()) os.remove('example.txt') # Clean up uncompressed file # Open for reading (auto-detects gzip) with xopen('example.txt.gz', mode='rt') as f: content = f.read() print(f'Read content: {content.strip()}') # Open for writing (creates xz compressed file) output_file = 'output.txt.xz' with xopen(output_file, mode='wt', compresslevel=3) as f: f.write('This is compressed with xz.\nAnother line.') # Verify writing (optional: requires another xopen to read it back) with xopen(output_file, mode='rt') as f: written_content = f.read() print(f'Written content to {output_file}: {written_content.strip()}') # Clean up generated files os.remove('example.txt.gz') os.remove(output_file)
Debug
Known issues
breakingxopen v1.8.0 and later (including 2.x) dropped support for Python 3.7. Users on older Python versions must either upgrade Python or pin `xopen` to a version prior to 1.8.0.
fix
Upgrade Python to 3.8 or later, or downgrade `xopen` to `<1.8.0`.
affects: >=1.8.0
gotchaUnlike Python's built-in `open()`, `xopen()` defaults the `encoding` parameter to 'utf-8' when opening files in text mode ('rt', 'wt', 'at'). If your files are not UTF-8 encoded, this can lead to decoding/encoding errors.
fix
Explicitly specify the correct `encoding` parameter (e.g., `encoding='latin-1'` or `encoding=locale.getpreferredencoding(False)`) when working with non-UTF-8 text files.
affects: >=1.3.0
gotchaThe `threads` parameter affects (de)compression performance and backend choice. `threads=None` (default) uses up to 4 CPU cores (potentially with external tools like `pigz`). `threads=0` forces single-threaded operation using only Python-based backends (`python-isal` or standard library modules). Behavior and performance will vary significantly.
fix
Understand your performance requirements. For maximum speed on multi-core systems, leave `threads=None`. For predictable single-threaded behavior (e.g., in resource-constrained environments or when external tools are undesirable), use `threads=0`.
affects: All
gotchaFor gzip files, `compresslevel=0` (no compression) could previously cause crashes if the external `gzip` application backend was used, as it lacked a `--0` flag. `xopen` now attempts to defer to other backends in this scenario, but unexpected behavior might still occur with specific system configurations or older `xopen` versions.
fix
Ensure `xopen` is updated to at least v2.0.0. If using `compresslevel=0` with gzip, test thoroughly, especially if external gzip utilities are involved.
affects: <2.0.0 (fixed in 2.0.0, but still a consideration for older deployments)
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'xopen'
The 'xopen' package has not been installed or is not available in the current Python environment.
fix
pip install xopen
ValueError: Suffix '.zst' not supported. Install the 'python-zstandard' package to enable .zst support.
The 'xopen' library attempts to open a Zstandard compressed file (.zst), but the optional 'python-zstandard' dependency is not installed.
fix
pip install python-zstandard
ModuleNotFoundError: No module named 'xopen.xopen'; 'xopen' is not a package
The user is attempting to import the 'xopen' function directly as a submodule, but 'xopen' is the module, and the primary function within it is also named 'xopen'.
fix
import xopen # then use xopen.xopen()
# or
from xopen import xopen # then use xopen()
_compression.BadGzipFile: Not a gzipped file (b'...')
The 'xopen' function was used to open a file with a '.gz' extension for reading, but the file's content is not actually in gzip format or is corrupted.
fix
Ensure the file truly is a gzip-compressed file, verify its integrity, or remove the '.gz' extension if it is a plain text file.
Upgrade
Version history
2.1.0latest on PyPI · released Jun 3, 2026
Audit
Dependencies
python-isalrequiredProvides highly optimized gzip (de)compression when threads=0 or external tools are not preferred/available. It's a direct dependency for optimal gzip performance.
zstandardoptionalEnables support for Zstandard (.zst) compressed files.
Agent activity
48 hits · last 30 days
node
42
OpenAI (training)
1
Resources
xopen — pip install xopen · libregistry