Registry / serialization / pycapnp

pycapnp

JSON →
library2.2.3pypypi✓ verified 87d ago

pycapnp is a Python wrapper for the C++ implementation of the Cap'n Proto data interchange format and RPC system. It provides insanely fast serialization and deserialization, often outperforming Protocol Buffers. The library is actively maintained, with regular releases bringing performance improvements, new features, and compatibility updates.

pip install pycapnp
INSTALL
IMPORT
SIG · PYCAPNP
P
pycapnp
serializationpythonv2.2.3
Install
2.1s avg
Import
221ms
Disk
35MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2.2.3 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.920 runs
installs and imports cleanly · install 0.0s · import 0.231s · 37.2MB
glibc
py 3.103.920 runs
installs and imports cleanly · install 2.1s · import 0.211s · 37MB
35MB installed
● package 35MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

capnp
import capnp

This quickstart demonstrates how to define a Cap'n Proto schema (dynamically for brevity), build a message by initializing a root object and its fields, and then serialize and deserialize it. It highlights the use of `capnp.load()` for schemas, initializing structs and lists, and safely reading messages using `capnp.alloc_builder()` and `capnp.alloc_reader()` context managers.

import capnp import os # Define a Cap'n Proto schema dynamically for demonstration # In a real application, this would be loaded from a .capnp file # e.g., addressbook = capnp.load('addressbook.capnp') SCHEMA_PATH = 'addressbook.capnp' addressbook_schema_content = ''' @0xd411d7353f406691; struct Person { id @0 :UInt32; name @1 :Text; email @2 :Text; phones @3 :List(PhoneNumber); struct PhoneNumber { number @0 :Text; type @1 :Type; enum Type { mobile @0; home @1; work @2; } } employment @4 :union { unemployed @5 :Void; employer @6 :Text; school @7 :Text; selfEmployed @8 :Void; } } struct AddressBook { people @0 :List(Person); } ''' # Write the schema to a temporary file with open(SCHEMA_PATH, 'w') as f: f.write(addressbook_schema_content) try: # Load the Cap'n Proto schema addressbook = capnp.load(SCHEMA_PATH) # 1. Build a message with capnp.alloc_builder() as msg_builder: addresses = msg_builder.init_root(addressbook.AddressBook) people = addresses.init('people', 2) alice = people[0] alice.id = 123 alice.name = 'Alice' alice.email = 'alice@example.com' alice_phones = alice.init('phones', 1) alice_phones[0].number = '555-1212' alice_phones[0].type = 'mobile' alice.employment.employer = 'Google' bob = people[1] bob.id = 456 bob.name = 'Bob' bob.email = 'bob@example.com' bob_phones = bob.init('phones', 2) bob_phones[0].number = '555-4567' bob_phones[0].type = 'home' bob_phones[1].number = '555-7654' bob_phones[1].type = 'work' bob.employment.unemployed = None # Serialize the message to bytes serialized_bytes = msg_builder.to_bytes_packed() print(f"Serialized message size: {len(serialized_bytes)} bytes") # 2. Read a message # Using capnp.alloc_reader() as a context manager is important for memory safety with capnp.alloc_reader(serialized_bytes) as msg_reader: read_addresses = msg_reader.get_root(addressbook.AddressBook) for person in read_addresses.people: print(f"\nPerson ID: {person.id}") print(f"Name: {person.name}") print(f"Email: {person.email}") print("Phones:") for phone in person.phones: print(f" - {phone.number} ({phone.type})") which_employment = person.employment.which() if which_employment == 'employer': print(f"Employment: Employer - {person.employment.employer}") elif which_employment == 'unemployed': print("Employment: Unemployed") else: print(f"Employment: {which_employment}") finally: # Clean up the temporary schema file if os.path.exists(SCHEMA_PATH): os.remove(SCHEMA_PATH)
Debug
Known issues
breakingStarting with pycapnp v2.0.0, the use of `asyncio` is mandatory for all RPC calls, and the synchronous RPC mode has been removed. Existing synchronous RPC code will break.
fix
Migrate all RPC implementations and calls to use Python's `asyncio` event loop. Refer to the updated RPC documentation and examples.
affects: >=2.0.0
breakingpycapnp v2.0.0 and later versions have dropped support for Python 3.7.
fix
Upgrade your Python environment to Python 3.8 or newer.
affects: >=2.0.0
gotchaInstallation of pycapnp requires a C++ Cap'n Proto library and a compatible C++14 compiler. While `pip install pycapnp` can often bundle and build the C++ library, issues may arise if the build environment is not correctly set up (e.g., missing development headers, incorrect compiler versions, or 32-bit Linux `fPIC` requirements).
fix
Ensure you have a C++14 compatible compiler (GCC 6.1+, Clang 6+, MSVC 2017+) and Python development headers. If facing issues, consider pre-installing the C++ Cap'n Proto library (version 1.0+) or explicitly controlling the bundling process during installation (e.g., `pip install . -C force-bundled-libcapnp=True`).
affects: All versions
gotchaAll `capnp` code that involves I/O operations should be wrapped within a `capnp.alloc_builder()` or `capnp.alloc_reader()` context manager to prevent potential segmentation faults and ensure proper resource management.
fix
Always use `with capnp.alloc_builder() as builder:` or `with capnp.alloc_reader(data) as reader:` patterns when interacting with Cap'n Proto messages.
affects: All versions, critical for >=2.0.0
gotchaWhen working with older versions (pre-Python 3.8, though pycapnp v2+ requires 3.8+), 'Text' type fields were treated as byte strings under Python 2 and unicode strings under Python 3. 'Data' fields consistently return byte strings across all Python versions. For modern `pycapnp` versions, 'Text' fields are always unicode strings.
fix
For current versions (>=2.0.0) running Python 3.8+, 'Text' fields are unicode strings. Ensure proper encoding/decoding if interfacing with older systems or different language bindings.
affects: <2.0.0 for Python 2/3 behavioral difference; informational for >=2.0.0
Errors
Common errors & fixes
error: command 'gcc' failed with exit status 1
This error frequently occurs during `pycapnp` installation when the system lacks the necessary C++ development tools (like a C++14 compatible compiler, CMake, or Ninja) or the underlying Cap'n Proto C++ library, preventing the Cython extensions from compiling.
fix
Ensure you have a C++14 compatible compiler (e.g., GCC 6+ or Clang 6+), CMake, and Ninja installed. On Linux, also ensure Python development headers are installed (e.g., `sudo apt-get install python3-dev` for Python 3). If `pycapnp` still fails to build, try forcing it to bundle the C++ Cap'n Proto library: `pip install pycapnp --no-binary :all: -C force-bundled-libcapnp=True`.
ModuleNotFoundError: No module named 'schema_capnp'
This error occurs when Python cannot find the generated module for a Cap'n Proto schema file (`.capnp`). This typically means the schema was not compiled correctly or the import path is wrong.
fix
After defining your schema in a `.capnp` file (e.g., `myschema.capnp`), you need to compile it into a Python module. This is usually done by importing it directly in your Python code, which triggers `pycapnp`'s import hook to compile it: `import myschema_capnp` (assuming `myschema.capnp` exists in the Python path or current directory).
Segmentation fault
A segmentation fault in `pycapnp` typically indicates an issue in the underlying C++ Cap'n Proto library or the Cython bindings, often due to incorrect memory access or object lifecycle management when interacting with Cap'n Proto messages or RPC clients/servers from Python.
fix
Ensure proper object ownership and lifecycle, especially with readers and builders. Avoid accessing union members without first checking `which()`. For RPC, ensure the event loop is running correctly, and if experiencing issues with older versions, update `pycapnp` to the latest version, as many memory-related bugs and segfaults have been addressed in recent releases.
'initializer_list' file not found
This specific compilation error, often seen on macOS, indicates that the C++ compiler cannot find the `<initializer_list>` header, usually because it's using an older or incompatible C++ standard library (e.g., libstdc++) instead of `libc++`.
fix
When installing, explicitly tell `pip` to use `libc++` by setting environment variables: `export CXXFLAGS="-stdlib=libc++" && export CFLAGS="-stdlib=libc++" && pip install pycapnp`.
Upgrade
Version history
2.2.3latest on PyPI · released May 31, 2026
Audit
Dependencies
C++ Cap'n Proto libraryrequiredpycapnp is a Cython wrapper around the C++ library; it can bundle and build Cap'n Proto or link to a system-wide installation.
C++14 supported compiler (e.g., GCC 6.1+, Clang 6+, Visual Studio 2017+)requiredRequired to compile the C++ Cap'n Proto library, either bundled or system-installed.
cmakeoptionalRequired if pycapnp needs to bundle and build the C++ Cap'n Proto library.
Python development headersrequiredRequired for compiling the Cython extensions.
Agent activity
4 hits · last 30 days
node
4
Resources
pycapnp — pip install pycapnp · libregistry