Registry / other / hadoop-streaming

hadoop-streaming

library3.5.0javamavenunverified

Utility for running MapReduce jobs with custom executables as mapper and reducer via standard input/output.

<dependency>
  <groupId>org.apache.hadoop</groupId>
  <artifactId>hadoop-streaming</artifactId>
  <version>3.5.0</version>
</dependency>

docs ↗

INSTALL

IMPORT

SIG · HADOOP-STREAMING

hadoop-streaming

otherjavav3.5.0

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

StreamJob

✓ org.apache.hadoop.streaming.StreamJob

Minimal usage of Hadoop Streaming to run a MapReduce job with shell commands.

import org.apache.hadoop.streaming.StreamJob; public class Example { public static void main(String[] args) throws Exception { String[] jobArgs = {"-input", "/input", "-output", "/output", "-mapper", "/usr/bin/cat", "-reducer", "/usr/bin/wc"}; StreamJob job = new StreamJob(); job.run(jobArgs); System.out.println("Streaming job completed"); } }

Debug

Known issues

gotchaHadoop Streaming requires the executables to be available on all cluster nodes; missing dependencies cause failures.

fix

Ensure mapper/reducer scripts are distributed via DistributedCache or available on all nodes.

affects: >=2.0.0

Upgrade

Version history

3.5.0latest on Maven

Audit

Dependencies

No dependency data recorded yet.

Agent activity

14 hits · last 30 days

node

Resources

docshttps://appdoc.app/artifact/org.apache.hadoop/hadoop-streaming/ ↗