Registry / data / blosc2

blosc2

JSON →
library4.11.0pypypi✓ verified 24d ago

Blosc2 is a high-performance compressed ndarray library for Python, using the C-Blosc2 compression backend. It provides efficient storage and manipulation of arbitrarily large N-dimensional datasets, following the Array API standard, and includes a flexible compute engine for complex calculations on compressed data. Currently at version 4.1.2, it maintains an active development pace with frequent updates and feature enhancements.

pip install blosc2 --upgrade
INSTALL
IMPORT
SIG · BLOSC2
B
blosc2
datapythonv4.11.0
Install
7.4s avg
Import
665ms
Disk
125MB
Pass rate
6/ 10
Env Coverage6 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v4.3.3 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
glibc
py 3.10
✕ build_error
✓ 7.9s
py 3.11
✕ build_error
✓ 9.2s
py 3.12
✕ build_error
✓ 7.2s
py 3.13
✕ build_error
✓ 7.7s
py 3.9
✓ —
✓ 4.9s
125MB installed
● package 125MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

blosc2
import blosc2
NDArray
import blosc2 array = blosc2.zeros((10, 10))
from blosc2 import NDArray
NDArray is generally accessed as `blosc2.NDArray` or through creation functions like `blosc2.zeros` or `blosc2.asarray`.
TreeStore
from blosc2 import TreeStore
Codec
blosc2.Codec.BLOSCLZ
blosc2.compress(data, codec='blosclz')
Compression codecs should be passed using the `blosc2.Codec` enum members, not string literals.
Filter
blosc2.Filter.SHUFFLE
blosc2.compress(data, filter='shuffle')
Compression filters should be passed using the `blosc2.Filter` enum members, not string literals.

This quickstart demonstrates creating a compressed NDArray, performing a simple computation, and decompressing it. It also includes an example of using `blosc2.TreeStore` for hierarchical data persistence, converting NumPy arrays and Blosc2 NDArrays to a file-based storage format.

import blosc2 import numpy as np # Create a Blosc2 NDArray from a NumPy array data = np.arange(1_000_000, dtype=np.float64) ndarray = blosc2.asarray(data) print(f"Original data size: {data.nbytes / (1024**2):.2f} MB") print(f"Compressed data size: {ndarray.nbytes / (1024**2):.2f} MB") # Perform a computation (e.g., sum) on the compressed array computed_sum = ndarray.sum() print(f"Sum of array elements: {computed_sum}") # Decompress the array back to a NumPy array decompressed_data = ndarray[:] assert np.allclose(data, decompressed_data) print("Data compressed and decompressed successfully.") # Example of using TreeStore for hierarchical data storage with blosc2.TreeStore("my_data.b2z", mode="w") as ts: ts["/group1/dataset_a"] = np.random.rand(100, 100) ts["/group2/dataset_b"] = blosc2.zeros((50, 50), dtype=np.int32) print("Data stored in TreeStore 'my_data.b2z'.") with blosc2.TreeStore("my_data.b2z", mode="r") as ts: ds_a = ts["/group1/dataset_a"] ds_b = ts["/group2/dataset_b"] print(f"Read dataset_a shape: {ds_a.shape}") print(f"Read dataset_b dtype: {ds_b.dtype}") os.remove("my_data.b2z")
Debug
Known issues
breakingBuffers generated with C-Blosc2 are generally not format-compatible with C-Blosc1 (i.e., forward compatibility is not supported). While C-Blosc2 is backward compatible with the C-Blosc1 API and in-memory format, users upgrading or sharing data between versions should be aware of this limitation.
fix
Ensure all consumers of Blosc2-generated data are using Blosc2. For C-Blosc1 compatibility, consider defining `BLOSC1_COMPAT` during C-Blosc2 compilation if using the C API directly.
affects: All versions where C-Blosc2 is used with C-Blosc1
breakingThe `NDArray.size` property changed its behavior in version 3.11.0. It now returns the number of elements in the array (Array API standard compliant) instead of the size of the array in bytes. Code relying on `NDArray.size` for byte size will need to be updated.
fix
For the size in bytes, use `ndarray.nbytes`. Update any code that used `.size` expecting byte count.
affects: >=3.11.0
breakingThe `blosc2.concatenate()` function was renamed to `blosc2.concat()` in version 3.5.0 to align with the Array API. While `concatenate` is still available for backward compatibility, it will be removed in a future release.
fix
Migrate code to use `blosc2.concat()` for future compatibility.
affects: >=3.5.0
gotchaWhen specifying compression codecs or filters, users must pass members of the `blosc2.Codec` and `blosc2.Filter` enums, respectively, not string literals. Passing strings will result in an `AttributeError`.
fix
Always use `blosc2.Codec.CODEC_NAME` (e.g., `blosc2.Codec.BLOSCLZ`) and `blosc2.Filter.FILTER_NAME` (e.g., `blosc2.Filter.SHUFFLE`).
affects: All versions
gotchaFor in-memory tasks, Blosc2's overhead can sometimes make it slower than pure NumPy/Numexpr, especially on x86 CPUs. However, it consistently outperforms them for on-disk operations or on modern ARM architectures (e.g., Apple Silicon) due to its efficient use of compression and cache optimization.
fix
Evaluate performance for specific workloads and hardware. Blosc2 shines in I/O-bound scenarios and for out-of-core computations. Consider the trade-offs between compression overhead and I/O savings.
affects: All versions
gotchaWhen using Blosc2 as an HDF5 filter, it is important not to activate the shuffle filter directly within HDF5. Blosc2 uses an internal SIMD shuffle that is much faster and should be handled by Blosc2 itself for optimal performance.
fix
Configure Blosc2's compression parameters directly, ensuring its internal shuffle is used when writing HDF5 datasets with the Blosc2 filter.
affects: All versions with HDF5 integration
Upgrade
Version history
4.11.0latest on PyPI · released Aug 12, 2026
Audit
Dependencies
numpyoptionalCrucial for creating and interacting with blosc2.NDArray objects, which are designed to be compatible with NumPy arrays. While not a direct install_requires, it's a de-facto dependency for most use cases.
numexproptionalUsed by Blosc2's compute engine (miniexpr) for hyper-fast multithreaded element-wise computations and reductions.
Agent activity
23 hits · last 30 days
node
20
OpenAI (training)
1
Resources