Registry / devops / shardy-mc-shard-face

shardy-mc-shard-face

JSON →
library2.1.0jsnpmunverified

ShardyMcShardFace is a JavaScript/TypeScript utility library designed to efficiently shard arrays of items across multiple parallel processes, primarily for Continuous Integration (CI) environments. It leverages standard CI environment variables like `CI_NODE_INDEX` and `CI_NODE_TOTAL` (via the `ci-parallel-vars` dependency) to determine the current shard and total shard count, distributing items as evenly as possible. The current stable version is 2.1.0, released in November 2019. Since then, the project appears to be in a maintenance-only state, with no new releases in over five years. Key features include stable, random-seeded item distribution, graceful handling of scenarios where the item count is less than the total shard count, and full TypeScript support, ensuring type safety. It offers both a programmatic API for direct integration into JavaScript/TypeScript applications and a command-line interface for simple shell scripting. Its primary differentiator is its direct, opinionated integration with common CI parallelization parameters, simplifying the setup for distributed test suites or build steps.

npm install shardy-mc-shard-face
INSTALL
IMPORT
SIG · SHARDY-MC-SHARD-FA
S
shardy-mc-shard-face
devopsjavascriptv2.1.0
Install
Import
Disk
Pass rate
0/ 6
Env Coverage0 / 6
glibc
1822
musl
1822
Install & Compatibility
Where this runs
tested against v? · npm install
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
node 18226 runs
build_error
glibc
node 18226 runs
build_error
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

shard
import { shard } from 'shardy-mc-shard-face';
import shardyMcShardFace from 'shardy-mc-shard-face';
The primary sharding function is a named export. There is no default export.
shard (CommonJS)
const { shard } = require('shardy-mc-shard-face');
For CommonJS environments, destructure the named 'shard' export. This package ships with TypeScript types, but CommonJS is still supported in Node.js environments.
shardy (CLI)
cat items.txt | shardy shard
The CLI is installed as 'shardy'. Input is newline-separated to stdin, output is newline-separated to stdout. Environment variables like CI_NODE_INDEX and CI_NODE_TOTAL are crucial for its operation.

This quickstart demonstrates how to programmatically shard an array of items using `shardy-mc-shard-face`. It simulates CI environment variables and shows both basic usage and how to apply custom options like a stable seed.

import { shard as shardyMcShardFace } from 'shardy-mc-shard-face'; // Simulate CI environment variables for local testing process.env.CI_NODE_INDEX = '0'; // Current shard index (0-based) process.env.CI_NODE_TOTAL = '2'; // Total number of shards const allItems = [ 'test_file_A.spec.ts', 'test_file_B.spec.ts', 'test_file_C.spec.ts', 'test_file_D.spec.ts', 'test_file_E.spec.ts', 'test_file_F.spec.ts' ]; // Basic sharding, options are optional const myShard = shardyMcShardFace(allItems); console.log(`Running on CI node ${process.env.CI_NODE_INDEX} of ${process.env.CI_NODE_TOTAL}`); console.log('My assigned shard:', myShard); // Example with custom options and seed for stable distribution const customShard = shardyMcShardFace(allItems, { throwOnEmpty: false, // Don't throw if shard is empty seed: 'my-stable-seed' // Ensures same items always go to same shard }); console.log('My assigned shard with custom seed:', customShard);
shardy --version
Debug
Known issues
breakingThe `shard` function's last parameter changed from a string to an options map.
fix
If you were passing a string as the last argument (e.g., for a seed), you must now pass an object `{ seed: 'your-seed' }` or similar. Refer to the updated function signature: `shardyMcShardFace(items, options)`.
affects: >=2.0.0
gotchaThis package relies heavily on CI environment variables (`CI_NODE_INDEX`, `CI_NODE_TOTAL`, etc.) to determine sharding parameters. If these are not set, it will default to a single shard (index 0 of 1).
fix
Ensure that your CI system correctly sets these environment variables. For local development or testing, you may need to manually set `process.env.CI_NODE_INDEX` and `process.env.CI_NODE_TOTAL` or use the `concurrency` option within the API if available in newer versions.
affects: >=1.0.0
gotchaThe project has not seen a new release since November 2019. While functional, active development and rapid bug fixes are unlikely.
fix
Assess the stability and existing features carefully. Consider contributing fixes or new features if needed, or fork the repository. Be aware that future JavaScript/TypeScript ecosystem changes might not be immediately supported.
affects: >=2.1.0
Errors
Common errors & fixes
TypeError: Expected options to be an object, but got string.
Attempting to pass a seed string directly as the last argument to the `shard` function in versions 2.0.0 or later.
fix
Wrap the seed string in an options object: `shard(items, { seed: 'your-seed' });`
Items are not being sharded; all items are returned in every execution.
The `CI_NODE_INDEX` or `CI_NODE_TOTAL` environment variables are not correctly set or are missing, causing the library to assume a single execution node.
fix
Verify that your CI environment or local setup correctly defines `CI_NODE_INDEX` and `CI_NODE_TOTAL` (e.g., `process.env.CI_NODE_INDEX = '0'; process.env.CI_NODE_TOTAL = '2';`). The library uses `ci-parallel-vars` to detect these.
Upgrade
Version history
2.1.0latest on npm
Audit
Dependencies
ci-parallel-varsrequiredRequired for automatically detecting CI parallelization parameters (e.g., current shard index and total shard count) from environment variables.
Agent activity
3 hits · last 30 days
node
2
Resources
shardy-mc-shard-face — npm install shardy-mc-shard-face · libregistry