Registry / serialization / msgpackr-extract

msgpackr-extract

JSON →
library3.0.3jsnpmunverified

msgpackr-extract is a Node.js native addon designed for highly optimized, C-level extraction of strings from MessagePack binary data. It acts as an optional native acceleration component for the `msgpackr` package, significantly speeding up string deserialization. The current stable version is 3.0.3, with releases often coinciding with updates to its parent project, `msgpackr`, or new Node.js ABI versions. Its key differentiator is its ability to perform partial MessagePack parsing to quickly locate and extract string data, especially leveraging V8's Latin-1 string optimizations by returning contiguous multi-string buffers. This low-level approach offers superior performance for applications heavily processing MessagePack data streams in Node.js environments. It's not a standalone MessagePack parser but a specialized utility for improving string extraction performance.

npm install msgpackr-extract
INSTALL
IMPORT
SIG · MSGPACKR-EXTRACT
M
msgpackr-extract
serializationjavascriptv3.0.3
Install
Import
Disk
Pass rate
0/ 6
Env Coverage0 / 6
glibc
1822
musl
1822
Install & Compatibility
Where this runs
tested against v? · npm install
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
node 18226 runs
build_error
glibc
node 18226 runs
build_error
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

extractStrings
import { extractStrings } from 'msgpackr-extract';
const { extractStrings } = require('msgpackr-extract');
While CommonJS `require` might work in older Node.js contexts, ESM `import` is the recommended modern approach since v3, especially as msgpackr itself favors ESM.
Node.js addon entry point
const extractStrings = require('msgpackr-extract').extractStrings;
import extractStrings from 'msgpackr-extract';
The module exports a named `extractStrings` function. A default import is incorrect. CommonJS `require` still functions for many native modules but is being phased out in new Node.js projects.

This example demonstrates how to use `msgpackr-extract` to extract strings from a MessagePack-encoded buffer. It first uses `msgpackr` to create a sample buffer, then calls `extractStrings` to get all strings within a specified range, highlighting its core functionality.

import { pack } from 'msgpackr'; import { extractStrings } from 'msgpackr-extract'; async function runExtraction() { const dataToPack = { name: 'Alice', city: 'New York', country: 'USA', tags: ['developer', 'typescript', 'nodejs'], description: 'A long string that might contain some non-latin characters like é or ö for testing purposes.' }; // Pack the data into a MessagePack buffer const packedBuffer = pack(dataToPack); console.log('Original packed buffer length:', packedBuffer.length); // Extract strings from the entire buffer. start and end are optional. const extracted = extractStrings(packedBuffer, 0, packedBuffer.length); console.log('Extracted strings:'); extracted.forEach(str => console.log(`- ${str}`)); // Example with a subset of the buffer (if you knew string offsets) // This module is usually used internally by msgpackr, but this shows direct usage. const partialExtracted = extractStrings(packedBuffer, 20, 50); // Arbitrary range if (partialExtracted.length > 0) { console.log('\nPartial extracted strings (from offset 20 to 50):'); partialExtracted.forEach(str => console.log(`- ${str}`)); } else { console.log('\nNo strings found in the partial range (offset 20 to 50).'); } } runExtraction().catch(console.error);
Debug
Known issues
gotchamsgpackr-extract is a native Node.js addon. This means it requires compilation with `node-gyp` during installation if prebuilt binaries are not available for your specific Node.js version, operating system, and architecture. Compilation failures are common in constrained environments (e.g., CI/CD without build tools, Docker containers without full toolchains).
fix
Ensure your environment has necessary build tools (Python, C++ compiler, Node.js development headers). Consider using Docker images with pre-installed build essentials or Node.js versions for which prebuilts are guaranteed. Update `node-gyp` if necessary (`npm install -g node-gyp`).
affects: >=2.0.0
breakingMajor version 3 of msgpackr-extract moved towards a more modern ESM-first approach, aligning with broader Node.js ecosystem changes. While CommonJS `require` might still function, new projects and best practices strongly recommend using ESM `import` statements.
fix
Update import statements from `const { extractStrings } = require('msgpackr-extract');` to `import { extractStrings } from 'msgpackr-extract';`.
affects: >=3.0.0
gotchaThe `extractStrings` function has an internal limit and will return a maximum of 256 strings per call. If your MessagePack data contains more strings than this within the specified range, you must call `extractStrings` again with a new offset to continue extraction.
fix
Implement a loop or recursive function to repeatedly call `extractStrings`, advancing the `start` offset based on the length of the previously extracted data, until all strings are processed or the end of the buffer is reached.
affects: >=2.0.0
gotchamsgpackr-extract is specifically designed for high-performance string *extraction* and partial parsing of MessagePack data. It is not a full MessagePack deserializer on its own. It is intended to be used as an internal component or acceleration for a complete MessagePack library like `msgpackr`.
fix
For full MessagePack encoding and decoding, use a comprehensive library such as `msgpackr` which integrates `msgpackr-extract` for accelerated string handling. Do not attempt to use `msgpackr-extract` for full object deserialization.
affects: >=2.0.0
Errors
Common errors & fixes
Error: The N-API module 'msgpackr-extract.node' was compiled against a different Node.js version. This version of Node.js may not be compatible with this module.
The prebuilt binary for msgpackr-extract (or one built during install) is incompatible with your current Node.js runtime's ABI.
fix
Try deleting `node_modules` and `package-lock.json` and reinstalling to force a rebuild (`npm rebuild msgpackr-extract` or `npm install`). If it persists, ensure your Node.js version is widely supported by prebuilts or that your build environment is fully set up.
`node-gyp` failed to rebuild `msgpackr-extract`
Compilation from source failed due to missing build tools (e.g., Python, C++ compiler, Node.js development headers) or environment issues.
fix
Install necessary build tools: on Windows, use 'npm install --global windows-build-tools'; on macOS, ensure Xcode Command Line Tools are installed; on Linux, install `build-essential` and `python3`. Verify `node-gyp` is up-to-date (`npm install -g node-gyp`).
Upgrade
Version history
3.0.3latest on npm
Audit
Dependencies
node-gyp-build-optional-packagesrequiredHandles optional native addon compilation and loading for different platforms and Node.js versions.
msgpackroptionalWhile not a direct runtime dependency, msgpackr-extract is primarily used to accelerate string parsing within the msgpackr ecosystem.
Agent activity
8 hits · last 30 days
node
8
Resources