clustershell

library1.9.3pypypi✓ verified 86d ago

ClusterShell is a Python library and set of tools designed for efficient command execution on Linux clusters. It provides robust NodeSet manipulation, parallel command execution, and a powerful CLI for High-Performance Computing (HPC) environments. The current version is 1.9.3, with releases typically occurring a few times a year, indicating active maintenance and development.

devops database workflow data

pip install clustershell

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · CLUSTERSHELL

clustershell

devopspythonv1.9.3

Install

3.0s avg

Import

41ms

Disk

21MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.9.3 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.930 runs

installs and imports cleanly · install 0.0s · import 0.044s · 23MB

glibc

py 3.10–3.930 runs

installs and imports cleanly · install 3.0s · import 0.039s · 24MB

21MB installed

● package 21MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

NodeSet

✓ from ClusterShell.NodeSet import NodeSet

✗ from clustershell.NodeSet import NodeSet

The top-level Python module is named 'ClusterShell' (uppercase C), not 'clustershell' (lowercase c), despite the PyPI package name.

Task

✓ from ClusterShell.Task import Task

✗ from clustershell.Task import Task

The top-level Python module is named 'ClusterShell' (uppercase C), not 'clustershell' (lowercase c), despite the PyPI package name.

This quickstart demonstrates the core NodeSet functionality of ClusterShell, allowing you to parse, manipulate, and query groups of node names efficiently. This is a foundational step before performing distributed tasks.

from ClusterShell.NodeSet import NodeSet # Create a NodeSet from a string representation nodes = NodeSet("node[0-3],host[10-12]") print(f"Original NodeSet: {nodes}") # Add individual nodes or other NodeSets nodes.add("server05") print(f"After adding server05: {nodes}") # Remove nodes nodes.remove("node1") print(f"After removing node1: {nodes}") # Check for membership print(f"Is node2 in the set? {'node2' in nodes}") # Iterate over nodes print("Nodes in the set:") for node in nodes: print(f"- {node}")

clush --version

Debug

Known issues

breakingClusterShell dropped support for Python 2.7.

fix

Upgrade your environment to Python 3.6 or newer. Previous ClusterShell versions (1.7.x and below) are required for Python 2 compatibility.

affects: >= 1.8.0

breakingNodeSet API changes: methods like `intersection()`, `difference()`, `union()`, and `symmetric_difference()` no longer modify NodeSets in-place but return a new NodeSet object.

fix

Ensure you assign the result of these methods back to a variable, e.g., `nodes = nodes.intersection(other_nodes)`. Code expecting in-place modification will behave unexpectedly.

affects: >= 1.7.0

deprecatedThe `NodeSet.clear()` method was deprecated.

fix

To empty a NodeSet, re-initialize it: `nodes = NodeSet()`.

affects: >= 1.7.0

gotchaThe `ClusterShell.Task` object's `run()` method performs actual SSH connections to specified nodes, which requires proper SSH setup (keys, agents, host reachability).

fix

Before using `Task.run()`, ensure you can manually SSH to your target nodes. For debugging, use `task.set_info("ssh_options", "-vvv")` to see detailed SSH output. For isolated testing, consider using `localhost` or mock objects.

affects: all

Errors

Common errors & fixes

ImportError: No module named 'clustershell.NodeSet'

Attempting to import from the lowercase package name 'clustershell' instead of the module's actual name 'ClusterShell' (uppercase 'C').

fix

Change your import statements to use 'ClusterShell', e.g., `from ClusterShell.NodeSet import NodeSet`.

TypeError: 'NodeSet' object is not subscriptable

Attempting to access elements of a NodeSet using indexing (e.g., `nodes[0]`). NodeSets are iterable but not directly indexable like lists.

fix

Convert the NodeSet to a list first if you need indexed access: `node_list = list(nodes)`, then `node_list[0]`.

Command fails to execute on remote nodes during `Task.run()` without clear error message in stdout

Often due to underlying SSH connectivity issues (e.g., firewall, incorrect hostnames, lack of proper SSH key authentication, sshd not running on target), or issues with the command's execution path on the remote host.

fix

Verify SSH connectivity manually (e.g., `ssh user@node01 hostname`). Check SSH agent forwarding and host key configuration. For deeper diagnostics, enable verbose SSH output within ClusterShell: `task.set_info("ssh_options", "-vvv")`.

Upgrade

Version history

1.9.3latest on PyPI · released Jan 25, 2025

Audit

Dependencies

PyYAMLoptionalRequired for parsing YAML configuration files if using advanced ClusterShell configuration.

Agent activity

30 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/cea-hpc/clustershell ↗

packagehttps://pypi.org/project/clustershell/ ↗

homepagehttps://clustershell.readthedocs.io/ ↗