webhdfs

library1.2.0jsnpmunverified

Node.js WebHDFS REST API client library implementing Hadoop WebHDFS REST API (2.2.0) with an fs-like asynchronous interface. Current stable version is 1.2.0. The library provides streaming file operations (createReadStream, createWriteStream) and supports user, host, port, and path configuration. It offers a familiar Node.js filesystem API pattern for interacting with Hadoop HDFS via WebHDFS. Release cadence is low; last release was several years ago. Differentiators include simplicity, minimal dependencies, and stream-based operations.

storage

npm install webhdfs

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · WEBHDFS

webhdfs

storagejavascriptv1.2.0

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

WebHDFS

✓ import { createClient } from 'webhdfs'

✗ import WebHDFS from 'webhdfs'

Package exports a single function createClient; there is no default export.

createClient

✓ const { createClient } = require('webhdfs')

✗ const WebHDFS = require('webhdfs').default

CommonJS require returns an object with createClient method.

WebHDFSClient

✓ // TypeScript type: declare module 'webhdfs' { function createClient(config?: ClientConfig): WebHDFSClient; }

Type definitions are not shipped; use @types/webhdfs or declare your own.

Demonstrates creating a client and performing streaming read/write operations to HDFS.

import { createClient } from 'webhdfs'; import * as fs from 'fs'; const hdfs = createClient({ user: process.env.HDFS_USER ?? 'webuser', host: process.env.HDFS_HOST ?? 'localhost', port: parseInt(process.env.HDFS_PORT ?? '80'), path: '/webhdfs/v1' }); // Write local file to HDFS const local = fs.createReadStream('/path/to/local/file'); const remote = hdfs.createWriteStream('/path/to/remote/file'); local.pipe(remote); remote.on('error', (err) => console.error('Write error:', err)); remote.on('finish', () => console.log('Write complete')); // Read from HDFS and pipe to stdout const reader = hdfs.createReadStream('/path/to/remote/file'); reader.pipe(process.stdout); reader.on('error', (err) => console.error('Read error:', err)); reader.on('finish', () => console.log('Read complete'));

Debug

Known issues

deprecatedPackage is no longer actively maintained; last release was in 2016.

fix

Consider using 'hdfs' or 'hdfs-webhdfs' packages for newer alternatives.

affects: >=1.0.0

gotchaDefault port is 50070 for Hadoop 2.x (not 80). The example uses port 80 which may not work unless configured.

fix

Set port to 50070 or your cluster's WebHDFS port.

affects: >=1.0.0

gotchaThe path option must include '/webhdfs/v1' to match Hadoop's WebHDFS endpoint.

fix

Always set path to '/webhdfs/v1' or your cluster's webhdfs prefix.

affects: >=1.0.0

breakingIn version 1.0.0, the API changed from constructor to factory function (createClient).

fix

Use createClient instead of new WebHDFS().

affects: >=1.0.0

Errors

Common errors & fixes

Error: connect ECONNREFUSED <host>:<port>

HDFS WebHDFS service not running or wrong host/port.

fix

Verify WebHDFS is enabled (dfs.webhdfs.enabled=true) and correct host/port.

Error: Authentication failed

Invalid or missing user parameter.

fix

Provide a valid Hadoop user via the user option.

TypeError: WebHDFS.createClient is not a function

Importing as default instead of named import.

fix

Use `const { createClient } = require('webhdfs')` or `import { createClient } from 'webhdfs'`.

Error: File not found

The remote path does not exist or is not accessible.

fix

Verify the path exists on HDFS and that the user has permissions.

Upgrade

Version history

1.2.0latest on npm

Audit

Dependencies

No dependency data recorded yet.

Agent activity

41 hits · last 30 days

node

Resources

githubhttps://github.com/harrisiirak/webhdfs ↗

packagewebhdfs ↗

homepagehttps://github.com/harrisiirak/webhdfs#readme ↗