Registry / serialization / gzip-stream

gzip-stream

JSON →
library1.2.0pypypi✓ verified 84d ago

gzip-stream is a lightweight Python library (version 1.2.0) for compressing and decompressing data on the fly using the GZIP format. It provides both synchronous and asynchronous stream interfaces, allowing efficient handling of large data without loading it all into memory. Releases are infrequent, often adding new features or compatibility updates rather than breaking changes.

pip install gzip-stream
INSTALL
IMPORT
SIG · GZIP-STREAM
G
gzip-stream
serializationpythonv1.2.0
Install
2.5s avg
Import
196ms
Disk
17MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v1.2.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
installs and imports cleanly · install 0.0s · import 0.209s · 19.2MB
glibc
py 3.103.910 runs
installs and imports cleanly · install 2.5s · import 0.184s · 20MB
17MB installed
● package 17MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

GZIPCompressedStream
from gzip_stream import GZIPCompressedStream
GZIPDecompressedStream
from gzip_stream import GZIPDecompressedStream
AsyncGZIPCompressedStream
from gzip_stream import AsyncGZIPCompressedStream
For asynchronous compression in asyncio applications.
AsyncGZIPDecompressedStream
from gzip_stream import AsyncGZIPDecompressedStream
For asynchronous decompression in asyncio applications.

This quickstart demonstrates synchronous compression and decompression of a byte stream using `gzip-stream`. It shows how to create an iterable of bytes, pass it to `GZIPCompressedStream`, and then take the resulting compressed chunks and pass them to `GZIPDecompressedStream` to retrieve the original data.

import gzip_stream import io # Original data (must be bytes) original_data = b"This is some data that will be compressed and then decompressed using gzip-stream." * 5 # Make it longer # --- Compression Example (Synchronous) --- # Create an iterable of bytes. In real apps, this could be reading from a file or network. def generate_bytes_for_compression(data_bytes): chunk_size = 15 # Arbitrary chunk size for i in range(0, len(data_bytes), chunk_size): yield data_bytes[i : i + chunk_size] # Compress the data by passing the iterable to GZIPCompressedStream compressed_chunks = list(gzip_stream.GZIPCompressedStream(generate_bytes_for_compression(original_data))) compressed_data = b"".join(compressed_chunks) print(f"Original size: {len(original_data)} bytes") print(f"Compressed size: {len(compressed_data)} bytes") # --- Decompression Example (Synchronous) --- # Create an iterable of compressed bytes. def generate_bytes_for_decompression(compressed_data_bytes): chunk_size = 20 # Arbitrary chunk size for decompression for i in range(0, len(compressed_data_bytes), chunk_size): yield compressed_data_bytes[i : i + chunk_size] # Decompress the data by passing the iterable to GZIPDecompressedStream decompressed_chunks = list(gzip_stream.GZIPDecompressedStream(generate_bytes_for_decompression(compressed_data))) decompressed_data = b"".join(decompressed_chunks) print(f"Decompressed data (first 50 bytes): {decompressed_data[:50].decode()}") # Verify data integrity assert original_data == decompressed_data print("\nSuccess: Original and decompressed data match!")
Debug
Known issues
gotchaThe library exclusively handles byte streams. Providing strings directly to `GZIPCompressedStream` or expecting strings from `GZIPDecompressedStream` without explicit encoding/decoding will lead to `TypeError` or `UnicodeDecodeError`.
fix
Ensure all input iterables yield `bytes` objects. Decode output chunks to `str` (e.g., `chunk.decode('utf-8')`) when necessary.
affects: All
gotchaSynchronous and asynchronous stream classes (e.g., `GZIPCompressedStream` vs `AsyncGZIPCompressedStream`) are distinct and not interchangeable. Attempting to use synchronous iteration (`for`) on an async stream will fail, and vice-versa.
fix
Use `async for` with `AsyncGZIPCompressedStream` and `AsyncGZIPDecompressedStream` within an `async def` function, run with `asyncio.run()`. Use standard `for` with synchronous streams.
affects: All
gotchaWhen compressing, it's crucial to fully iterate through the `GZIPCompressedStream` (or `AsyncGZIPCompressedStream`) to ensure the GZIP footer, which contains checksums and length information, is properly written. If you stop iterating prematurely, the resulting compressed data may be truncated or corrupted.
fix
Always exhaust the stream by reading all chunks, for example, by converting it to a list (`list(stream)`) or iterating until completion (`for chunk in stream: ...`).
affects: All
Errors
Common errors & fixes
TypeError: a bytes-like object is required, not 'str'
Attempting to pass string data directly to `GZIPCompressedStream` or `GZIPDecompressedStream`, which expect `bytes`.
fix
Encode your string data to bytes before passing it to the stream. Example: `my_string.encode('utf-8')`.
TypeError: 'AsyncGZIPCompressedStream' object is not async iterable
Trying to iterate an asynchronous stream object (`AsyncGZIPCompressedStream` or `AsyncGZIPDecompressedStream`) using a synchronous `for` loop.
fix
Ensure you are using `async for chunk in stream:` within an `async def` function, and run the asynchronous code using `asyncio.run()`.
Upgrade
Version history
1.2.0latest on PyPI · released Dec 8, 2021
Audit
Dependencies

No dependency data recorded yet.

Agent activity
6 hits · last 30 days
node
6
Resources
gzip-stream — pip install gzip-stream · libregistry