Install & Compatibility
Where this runs
tested against v3.0.6 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
installs and imports cleanly · install 0.0s · import 0.358s · 20.2MB
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 2.0s · import 0.322s · 23MB
19MB installed
● package 19MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
Client
✓ from snakebite.client import Client
✗ from snakebite import Client
The primary client class is located within the `snakebite.client` submodule. Directly importing from `snakebite` will fail.
Client
✓ from snakebite.client import Client
✗ from snakebite_py3.client import Client
The package name on PyPI is `snakebite-py3`, but the internal import path remains `snakebite` for compatibility with the original project's API.
This quickstart demonstrates how to establish a connection to an HDFS NameNode and perform basic file system operations like listing directories and creating a new directory. It uses environment variables for host and port for flexibility, defaulting to `localhost:8020`. Ensure your HDFS cluster is running and accessible from where you execute this code.
import os
from snakebite.client import Client
# Configure HDFS NameNode host and port
# Default HDFS RPC port is 8020
host = os.environ.get('HDFS_NAMENODE_HOST', 'localhost')
port = int(os.environ.get('HDFS_NAMENODE_PORT', '8020'))
try:
# Initialize the HDFS client
# It's recommended to set use_trash=False for non-interactive scripts
# Or explicitly set hadoop_version if not the default (9)
client = Client(host, port, use_trash=False)
print(f"Connected to HDFS NameNode at {host}:{port}")
# Example: List contents of the root directory
print("Listing /:")
for item in client.ls(['/']):
print(item)
# Example: Create a directory
test_dir = '/user/test_snakebite_py3'
if not list(client.ls([test_dir])):
print(f"Creating directory {test_dir}")
list(client.mkdir([test_dir], create_parents=True))
else:
print(f"Directory {test_dir} already exists.")
except Exception as e:
print(f"An error occurred: {e}")
print("Please ensure your HDFS NameNode is running and accessible at the specified host and port.")
snakebite --version
Debug
Known issues
breakingThe `snakebite-py3` library is a Python 3 fork of the original `snakebite`, which was Python 2 only. Projects migrating from `snakebite` must switch to `snakebite-py3` and ensure their codebase is Python 3 compatible. Attempting to use the original `snakebite` in Python 3 environments will result in errors.fixInstall `snakebite-py3` and update import statements to `from snakebite.client import Client`. Review code for any Python 2 specific constructs.
affects: < 3.0.0 (original snakebite)
gotchaMany methods within `snakebite.client` (e.g., `ls`, `mkdir`, `rm`) return generators. The actual HDFS operation is only executed when the generator is consumed (e.g., by iterating over it with a `for` loop or converting it to a `list()`). Failing to consume the generator means the operation will not be performed.fixAlways iterate over the returned generator or wrap it in `list()` to ensure the HDFS command executes: `list(client.mkdir(['/new_dir']))`.
affects: All versions
gotchaUnlike the standard Hadoop client, `snakebite-py3` disables CRC (Cyclic Redundancy Check) for data transfers by default to improve performance. This means data integrity is not verified during transfer unless explicitly enabled.fixFor operations requiring CRC checks (e.g., `cat`), pass `check_crc=True` as an argument to the method: `for chunk in client.cat(['/path/to/file'], check_crc=True): ...`
affects: All versions
gotcha`snakebite-py3` has primarily been tested with specific Hadoop distributions like CDH5 and supports Hadoop 2.2.0+ (protocol version 9). Compatibility with newer Hadoop versions or different distributions might vary and may require specifying the `hadoop_version` parameter in the `Client` constructor.fixIf encountering connection issues or unexpected behavior, try explicitly setting the `hadoop_version` parameter: `client = Client(host, port, hadoop_version=some_version)`.
affects: All versions
gotchaThe `Client` constructor parameter `use_trash` is often set to `False` in examples and defaults to `False` in many contexts, meaning file deletions can be permanent without moving to the HDFS trash.fixFor safer deletion, explicitly set `use_trash=True` when initializing the client or when performing delete operations if supported by the method: `client = Client(host, port, use_trash=True)`.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'snakebite'
The `snakebite-py3` library has not been installed in the current Python environment or the Python interpreter cannot find it in its path.
fixInstall the library using pip: `pip install snakebite-py3`
ConnectionRefusedError: [Errno 111] Connection refused
The HDFS NameNode service is either not running or is inaccessible from the client machine at the specified host and port (e.g., due to incorrect configuration or a firewall).
fixEnsure the HDFS NameNode is running, verify the NameNode's host and port in your client configuration (e.g., `Client('namenode_host', 8020)`), and check network connectivity or firewall rules. snakebite.errors.HDFSError: Permission denied
The user or process running the snakebite client lacks the necessary HDFS permissions to perform the requested operation on the target file or directory.
fixGrant appropriate HDFS permissions to the user or group executing the snakebite client using `hdfs dfs -chmod`, `hdfs dfs -chown`, or `hdfs dfs -setfacl` commands on the HDFS cluster.
NameError: name 'string_types' is not defined
This error typically occurs when trying to use Python 2 compatibility code (like `string_types`) in a Python 3 environment without properly importing it from `six`, or if the `six` library is missing.
fixEnsure the `six` library is installed (`pip install six`) and properly imported if you are using `string_types` directly. For `snakebite-py3` internal use, ensure a compatible version of the library is installed.
Upgrade
Version history
3.0.6latest on PyPI · released Feb 18, 2025
Audit
Dependencies
protobufrequiredUsed for communication with the HDFS NameNode via RPC protocol.
pyasn1optionalRequired for Kerberos/SASL authentication, if enabled.
pykerberosoptionalRequired for Kerberos/SASL authentication, if enabled.