lzfse

library0.4.2pypypi✓ verified 23d ago

The `lzfse` library provides Python bindings to Apple's LZFSE (Lempel–Ziv Finite State Entropy) compression algorithm. LZFSE is an open-source lossless data compression algorithm developed by Apple, designed for faster decompression and higher energy efficiency compared to zlib, particularly optimized for modern micro-architectures like arm64. The Python bindings are currently at version 0.4.2 (as of April 19, 2024), with infrequent releases often tied to updates in the underlying C reference implementation or improvements to the Python packaging.

serialization

pip install lzfse

github ↗package ↗

INSTALL

IMPORT

SIG · LZFSE

lzfse

serializationpythonv0.4.2

Install

1.6s avg

Import

—

Disk

17MB

Pass rate

8/ 10

Env Coverage8 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.4.2 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✓ —

✓ 1.6s

py 3.11

✓ —

✓ 1.7s

py 3.12

✓ —

✓ 1.4s

py 3.13

✕ build_error

py 3.9

✓ —

✓ 1.7s

17MB installed

● package 17MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

lzfse

✓ import lzfse

✗ import liblzfse

The original project was named 'pyliblzfse' and imported as `liblzfse`. The current `lzfse` package on PyPI is a fork and rename, which should be imported directly as `lzfse`.

This example demonstrates basic LZFSE compression and decompression of a byte string, including error handling for corrupted data.

import lzfse original_data = b"This is some data to be compressed using LZFSE! " * 100 # Compress data compressed_data = lzfse.compress(original_data) print(f"Original size: {len(original_data)} bytes") print(f"Compressed size: {len(compressed_data)} bytes") # Decompress data decompressed_data = lzfse.decompress(compressed_data) print(f"Decompressed size: {len(decompressed_data)} bytes") # Verify data integrity assert original_data == decompressed_data print("Data compressed and decompressed successfully!") # Handle decompression errors corrupted_data = b'\xde\xad\xbe\xef' + compressed_data[4:] # Corrupt header try: lzfse.decompress(corrupted_data) except lzfse.error as e: print(f"Caught expected error during decompression: {e}")

Debug

Known issues

breakingThe current `lzfse` package is a fork and rename of an earlier project, `pyliblzfse`. Codebases written for `pyliblzfse` might use `import liblzfse`, which will break with the `lzfse` package. The correct import is now `import lzfse`.

fix

Update import statements from `import liblzfse` to `import lzfse`.

affects: <0.4.0 (for pyliblzfse users), all (for lzfse users migrating from pyliblzfse)

gotchaOn platforms where pre-built wheels are not available (e.g., some Linux distributions, or when specific Python versions are not covered), installing `lzfse` might attempt to build from source. This requires a C/C++ compiler and potentially system-specific development headers. On Windows, this often means installing 'Desktop Development with C++' and the Windows 10/11 SDK via Visual Studio Build Tools, failing with errors like 'stddef.h: No such file or directory'.

fix

Ensure a compatible C/C++ build environment is installed on your system. For Windows, install the 'Desktop Development with C++' workload and the Windows 10/11 SDK using the Visual Studio Build Tools installer. For Linux, ensure `build-essential` or similar developer tools are installed.

affects: All versions when installing from source.

gotchaThe underlying LZFSE C reference implementation has known limitations and potential issues, including degraded compression performance for inputs larger than 2GiB and documented memory-related crashes (e.g., double free when compressing empty files, malloc errors). As the Python library is a binding, these issues can manifest when using `lzfse`.

fix

Be aware of these limitations when processing very large files or unusual inputs. Consider pre-processing large data into smaller chunks if 2GiB+ performance degradation is a concern. Refer to the upstream LZFSE C library's GitHub issues for more details and potential workarounds.

affects: All versions.

Errors

Common errors & fixes

ERROR: Failed building wheel for lzfse

The `lzfse` library contains C extensions that require a C/C++ compiler and associated development tools (like the Windows SDK or `build-essential` on Linux) to compile during installation if a pre-built wheel is not available for your specific Python version and operating system.

fix

Install the necessary build tools for your operating system. On Windows, install 'Desktop Development with C++' from Visual Studio Build Tools. On Debian/Ubuntu Linux, run `sudo apt-get install build-essential`. On macOS, ensure Xcode Command Line Tools are installed (`xcode-select --install`).

ModuleNotFoundError: No module named 'lzfse'

The `lzfse` package is not installed in the active Python environment, or the environment where the code is being executed does not have access to the installed package.

fix

Install the package using pip: `pip install lzfse` or `python3 -m pip install lzfse`. Ensure you are running this in the correct Python environment.

ImportError: cannot import name 'compress' from 'lzfse'

While the `lzfse` module is found, the specific function or object named 'compress' (or 'decompress') is not directly importable or is not exposed as a top-level function. The `lzfse` library exposes `compress` and `decompress` as methods of the `lzfse` module itself rather than individual functions to be imported by name. This can also happen if there's a version mismatch or confusion with other LZFSE-related libraries that might use different import patterns.

fix

Instead of `from lzfse import compress`, you should import the module and call the function directly on it: `import lzfse; compressed_data = lzfse.compress(data)`. Similarly for `decompress`: `decompressed_data = lzfse.decompress(compressed_data)`.

Upgrade

Version history

0.4.2latest on PyPI · released Apr 19, 2024

Audit

Dependencies

C/C++ Build ToolsoptionalThe library is a wrapper around a C implementation; compiling from source (e.g., when wheels aren't available for your platform/Python version, especially on Windows) requires a C/C++ compiler and associated SDKs (e.g., Windows 10/11 SDK for Visual Studio Build Tools).

Agent activity

5 hits · last 30 days

node

Resources

githubhttps://github.com/m1stadev/lzfse ↗

packagehttps://pypi.org/project/lzfse/ ↗