Registry /
serialization / msgpack-numpy-opentensor
Install & Compatibility
Where this runs
tested against v0.5.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.005s · 90.4MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 3.7s · import 0.002s · 87MB
90MB installed
● package 90MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
patch
✓ from msgpack_numpy import patch
✗ import msgpack_numpy_opentensor as m; m.patch()
Packer
✓ from msgpack_numpy import Packer
✗ import msgpack_numpy_opentensor as m; m.Packer
Unpacker
✓ from msgpack_numpy import Unpacker
✗ import msgpack_numpy_opentensor as m; m.Unpacker
This quickstart demonstrates how to serialize a NumPy array into MessagePack binary format and then deserialize it back into a NumPy array using the `msgpack-numpy-opentensor` library. It shows both packing (`packb`) and unpacking (`unpackb`) using the provided encoder and decoder functions, highlighting that deserialized arrays are typically read-only.
import numpy as np
import msgpack
import msgpack_numpy_opentensor as m
# Create a NumPy array
x = np.random.rand(5, 5)
# Pack the NumPy array using msgpack-numpy-opentensor's encoder
# Optionally, you can call m.patch() to monkey-patch msgpack globally
# m.patch()
packed_x = msgpack.packb(x, default=m.encode)
# Unpack the bytes back into a NumPy array using the decoder
unpacked_x = msgpack.unpackb(packed_x, object_hook=m.decode, raw=False)
print("Original array:\n", x)
print("Unpacked array:\n", unpacked_x)
print("Arrays are equal:", np.array_equal(x, unpacked_x))
print("Unpacked array is read-only:", not unpacked_x.flags['WRITEABLE'])
Debug
Known issues
breakingThe upstream `opentensor/msgpack-numpy` GitHub repository, linked as this package's source, released `v1.0.0` with a breaking change: it disables `pickle` by default. This will prevent deserialization of NumPy arrays with `dtype='O'` (object arrays) that were serialized with pickle enabled in older versions. While `msgpack-numpy-opentensor` on PyPI is currently `0.5.0`, this change may propagate to future versions.fixReview `dtype='O'` usage and consider custom serializers for such arrays if you rely on pickle for compatibility. Be prepared to update serialization/deserialization logic if upgrading to a version with this change.
affects: Potentially `1.0.0+` (if adopted by `msgpack-numpy-opentensor`)
gotchaNumPy arrays with `dtype='O'` (object arrays) are serialized/deserialized using Python's `pickle` module as a fallback by `msgpack-numpy` (and, by extension, likely `msgpack-numpy-opentensor`). This introduces significant performance overhead and poses security risks when deserializing data from untrusted sources due to pickle's arbitrary code execution capabilities.fixAvoid `dtype='O'` for sensitive or performance-critical data. Consider explicitly converting object arrays to more primitive types or implementing custom, secure encoders/decoders for specific object types.
affects: All versions
gotchaNumPy arrays deserialized by `msgpack-numpy` (and thus, `msgpack-numpy-opentensor`) are read-only by default. Attempting to modify them directly will raise a `ValueError` or `AttributeError`.fixIf modification is required, explicitly create a writable copy of the array after deserialization, e.g., `modified_array = unpacked_array.copy()`.
affects: All versions
gotchaThe underlying `msgpack` library has limitations on the maximum size of individual binary or string objects, typically around 4.3 GB. Attempting to serialize a single NumPy array that exceeds this limit may result in serialization errors.fixFor extremely large NumPy arrays, consider chunking them into smaller pieces before serialization, using alternative serialization formats designed for larger-than-memory data, or streaming solutions.
affects: All versions
Upgrade
Version history
0.5.0latest on PyPI · released Oct 2, 2023
Audit
Dependencies
msgpackrequiredCore MessagePack serialization library.
numpyrequiredProvides the array and numerical types for serialization.