Registry / http-networking / stream-concat

stream-concat

JSON →
library2.0.0jsnpmunverified

The `stream-concat` library provides a simple and efficient mechanism for concatenating multiple Node.js readable streams into a single output readable stream. Currently at version `2.0.0`, it maintains an active development status, though a specific release cadence isn't explicitly defined. A core differentiator is its ability to handle streams in two ways: either by accepting an array of pre-existing streams or, more efficiently, by using a user-supplied function that returns the next stream on demand. This 'on-the-fly' stream creation significantly reduces memory consumption and improves performance, particularly when dealing with large datasets or numerous files, by preventing all streams from being buffered simultaneously. It builds upon Node.js's `Transform` stream, leveraging its underlying options like `highWaterMark` and `objectMode` for fine-grained control over stream behavior.

npm install stream-concat
INSTALL
IMPORT
SIG · STREAM-CONCAT
S
stream-concat
http-networkingjavascriptv2.0.0
Install
Import
Disk
Pass rate
0/ 6
Env Coverage0 / 6
glibc
1822
musl
1822
Install & Compatibility
Where this runs
tested against v? · npm install
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
node 18226 runs
build_error
glibc
node 18226 runs
build_error
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

StreamConcat
const StreamConcat = require('stream-concat');
import StreamConcat from 'stream-concat';
This package is CommonJS only. Using ES Modules `import` syntax directly will likely result in a TypeError or undefined import without a transpilation step or specific Node.js loader configuration.

This example demonstrates how to concatenate multiple files efficiently using a `nextStream` function. This approach defers opening new streams until they are needed, optimizing memory usage compared to providing an array of all streams upfront.

const fs = require('fs'); const StreamConcat = require('stream-concat'); const path = require('path'); // Create some dummy files for demonstration fs.writeFileSync('file1.csv', 'header1,data1a\nheader2,data1b\n'); fs.writeFileSync('file2.csv', 'header3,data2a\nheader4,data2b\n'); fs.writeFileSync('file3.csv', 'header5,data3a\nheader6,data3b\n'); const fileNames = ['file1.csv', 'file2.csv', 'file3.csv']; let fileIndex = 0; // Use a function to defer stream creation, reducing memory footprint const nextStream = () => { if (fileIndex === fileNames.length) { return null; } console.log(`Reading: ${fileNames[fileIndex]}`); return fs.createReadStream(fileNames[fileIndex++]); }; const output = fs.createWriteStream('combined.csv'); const combinedStream = new StreamConcat(nextStream); combinedStream.pipe(output) .on('finish', () => { console.log('All files combined into combined.csv'); // Clean up dummy files fileNames.forEach(file => fs.unlinkSync(file)); fs.unlinkSync('combined.csv'); }) .on('error', (err) => { console.error('Error combining streams:', err); });
Debug
Known issues
breakingVersion `2.0.0` and newer requires Node.js `v12` or higher. Older versions of Node.js (`v0.12` to `v10`) are only supported by `stream-concat` versions prior to `1.0.0`. Ensure your Node.js runtime meets the `engines` requirement.
fix
Upgrade your Node.js environment to version 12 or newer, or use an older `stream-concat` version if you are constrained to an older Node.js runtime.
affects: >=1.0.0
gotchaWhen concatenating many or very large streams, providing an array of all streams directly to the `StreamConcat` constructor can lead to high memory consumption and reduced performance. All streams' read queues will be buffered simultaneously.
fix
Instead of providing an array of streams, pass a function to the `StreamConcat` constructor that returns the next stream (or a Promise resolving to a stream) only when it's needed. This allows the library to manage buffering sequentially.
affects: >=0.12
Errors
Common errors & fixes
TypeError: StreamConcat is not a constructor
Attempting to use `import StreamConcat from 'stream-concat'` in an ES Modules (ESM) context without proper CJS-ESM interop.
fix
Since `stream-concat` is a CommonJS package, you must use `const StreamConcat = require('stream-concat');` to import it correctly in both CJS and ESM environments (where ESM allows CJS `require`).
Upgrade
Version history
2.0.0latest on npm
Audit
Dependencies

No dependency data recorded yet.

Agent activity
10 hits · last 30 days
node
8
OpenAI (training)
1
Resources
stream-concat — npm install stream-concat · libregistry