hdf5plugin

library6.0.0pypypi✓ verified 84d ago

hdf5plugin is a Python library that provides additional HDF5 compression filters for use with h5py, enabling reading and writing of compressed datasets with various algorithms like Blosc, Bitshuffle, LZ4, Zstd, and more. It is actively maintained with frequent releases, currently at version 6.0.0, and often updates its embedded compression libraries and introduces new filters.

data serialization

pip install hdf5plugin

github ↗package ↗docs ↗

INSTALL

IMPORT

SIG · HDF5PLUGIN

hdf5plugin

datapythonv6.0.0

Install

—

Import

—

Disk

—

Pass rate

0/ 10

Env Coverage0 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v6.0.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

4/8 runs

py 3.11

✕ build_error

4/8 runs

py 3.12

✕ build_error

4/8 runs

py 3.13

✕ build_error

4/8 runs

py 3.9

✕ build_error

4/8 runs

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

hdf5plugin

✓ import hdf5plugin

Importing the package automatically registers the supported HDF5 compression filters with the HDF5 library used by h5py, making them available for use. No explicit filter registration calls are typically needed.

h5py

✓ import h5py

Required to interact with HDF5 files and datasets.

This quickstart demonstrates how to write and read a dataset compressed using one of the filters provided by hdf5plugin (LZ4 in this case). The `import hdf5plugin` statement is crucial as it registers the filters with h5py.

import numpy import h5py import hdf5plugin # Create a dummy dataset data_to_write = numpy.arange(100, dtype='i4') # Write compressed data to an HDF5 file using an hdf5plugin filter (e.g., LZ4) file_name = 'test_compressed.h5' with h5py.File(file_name, 'w') as f: dset = f.create_dataset('data', data=data_to_write, compression=hdf5plugin.LZ4()) print(f"Dataset 'data' written to {file_name} with LZ4 compression.") # Read the compressed data back with h5py.File(file_name, 'r') as f: read_data = f['data'][()] print(f"Data read successfully: {read_data}") assert numpy.array_equal(data_to_write, read_data) print("Original and read data match.")

Debug

Known issues

breakingVersion 6.0.0 requires Python >= 3.9. Prior versions supported older Python versions (e.g., v5.0.0 required >=3.8, v4.0.0 required >=3.7).

fix

Ensure your Python environment is 3.9 or newer. Downgrade hdf5plugin if an older Python version is strictly required.

affects: >=6.0.0

breakingVersion 5.0.0 requires h5py >= 3.0.0. This was a significant bump from previous versions.

fix

Upgrade h5py to version 3.0.0 or higher. You may also need to upgrade Python if h5py's requirements for newer versions conflict with your setup.

affects: >=5.0.0

breakingDeprecated constants `hdf5plugin.config`, `hdf5plugin.date`, `hdf5plugin.hexversion`, and `hdf5plugin.strictversion` were removed in version 5.0.0.

fix

Replace usage of these constants. For `hdf5plugin.config`, use `hdf5plugin.get_config()` instead. Other removed constants do not have direct replacements and should be removed if no longer necessary.

affects: >=5.0.0

deprecatedThe SZ filter has been deprecated in version 6.0.0. While it might still function, its continued support is not guaranteed.

fix

Migrate away from using the SZ filter to other supported filters like SZ3, Zfp, or Blosc for future compatibility and better performance/features.

affects: >=6.0.0

gotchaData compressed with newer versions of the H5Z-ZFP filter (e.g., v1.1.0 in hdf5plugin v4.0.0) might not be readable by older versions of the filter, though newer versions can read older data.

fix

Ensure that the hdf5plugin version used for reading ZFP-compressed data is at least as new as, or newer than, the version used for writing.

affects: >=4.0.0

gotchaSome advanced Blosc2 compression codecs (e.g., blosc2-grok, blosc2-openhtj2k) might require additional, separately installed plugins to be present in the HDF5_PLUGIN_PATH environment variable for decompression to work correctly. hdf5plugin itself may not bundle all possible Blosc2 sub-filters.

fix

If encountering errors with specific Blosc2 codecs, consult the Blosc2 documentation for required external plugins and ensure they are installed and discoverable via the HDF5_PLUGIN_PATH environment variable.

affects: All

gotchaPoorly chosen HDF5 chunking strategies (e.g., very large/small chunks, or chunk shapes misaligned with common access patterns) can significantly degrade performance, even with efficient compression filters. This is a general HDF5/h5py concern but applies directly to hdf5plugin usage.

fix

Design chunking to align with expected read/write access patterns. Refer to HDF5 and h5py documentation on chunking best practices for optimal performance with compressed datasets.

affects: All

Errors

Common errors & fixes

IOError: Can't read data (Can't open directory) - Missing gzip compression filter

This error, or similar messages like 'Can't read data (filter not available)', occurs when `h5py` attempts to open an HDF5 file compressed with a filter provided by `hdf5plugin`, but the underlying HDF5 library cannot locate or load the required plugin.

fix

Ensure `hdf5plugin` is installed (`pip install hdf5plugin` or `conda install -c conda-forge hdf5plugin`) and explicitly import it in your Python script before any `h5py` operations: `import hdf5plugin`. This import registers the necessary filters with the HDF5 library.

ValueError: Compression filter "<class 'hdf5plugin.Blosc'>" is unavailable

This error indicates that while you are trying to create a dataset with a specific `hdf5plugin` compression filter (e.g., Blosc, Zstd, LZ4), the HDF5 library does not recognize or cannot load that filter for writing. This can be due to `hdf5plugin` not being correctly registered or an incompatibility with the underlying HDF5 installation.

fix

First, ensure `hdf5plugin` is installed and imported: `import hdf5plugin`. If the problem persists, consider reinstalling `hdf5plugin` from source or via `conda-forge` to ensure all underlying C libraries are correctly linked and compatible with your `h5py` and HDF5 installation.

ModuleNotFoundError: No module named 'hdf5plugin'

This is a standard Python error indicating that the `hdf5plugin` package is not installed or is not accessible in the Python environment where your script is being run.

fix

Install the `hdf5plugin` package using your preferred package manager: `pip install hdf5plugin` (for pip) or `conda install -c conda-forge hdf5plugin` (for Anaconda/Miniconda).

Upgrade

Version history

6.0.0latest on PyPI · released Oct 8, 2025

Audit

Dependencies

h5pyrequiredhdf5plugin registers filters with h5py; it's a core dependency for its functionality.

numpyoptionalCommonly used for data manipulation with h5py, especially in examples.

Agent activity

4 hits · last 30 days

node

Resources

githubhttps://github.com/silx-kit/hdf5plugin ↗

packagehttps://pypi.org/project/hdf5plugin/ ↗

docshttp://www.silx.org/doc/hdf5plugin/latest/ ↗