Registry / workflow / apache-airflow-providers-apache-hdfs

apache-airflow-providers-apache-hdfs

library4.11.5pypypiunverified

A provider package for Apache Airflow that integrates with Apache HDFS, providing hooks and operators for HDFS file operations. Current version 4.11.5 requires Python >=3.10. Released on Airflow provider schedule.

workflow data database

pip install apache-airflow-providers-apache-hdfs

github ↗package ↗docs ↗changelog ↗

INSTALL

IMPORT

SIG · APACHE-AIRFLOW-PRO

apache-airflow-providers-apache-hdfs

workflowpythonv4.11.5

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

HDFSHook

✓ from airflow.providers.apache.hdfs.hooks.hdfs import HDFSHook

✗ from airflow.providers.apache.hdfs.hooks.hdfs import HDFSHook

Basic usage of HDFSHook to list files

from airflow.providers.apache.hdfs.hooks.hdfs import HDFSHook hook = HDFSHook(conn_id='hdfs_default') # List files in a directory files = hook.list_directory('/tmp') print(files)

Debug

Known issues

breakingHDFSHook and operators were moved from `airflow.hooks.hdfs` and `airflow.operators.hdfs` to `airflow.providers.apache.hdfs` in version 2.0.0. Old imports will break.

fix

Use new import paths: `from airflow.providers.apache.hdfs.hooks.hdfs import HDFSHook`

affects: <2.0.0

gotchaThe HDFSHook's get_conn() returns a HDFSClient that may require explicit authentication; default uses 'hdfs' connection string without kerberos if not configured.

fix

Ensure your Airflow connection has extra parameters for kerberos if needed.

affects: all

deprecatedHdfsMkdirFileOperator and HdfsPutFileOperator are deprecated as of provider version 4.0.0 in favor of generic FileTransferOperator.

fix

Use `FileTransferOperator` from `airflow.providers.apache.hdfs.operators.hdfs` or implement custom logic.

affects: >=4.0.0

Upgrade

Version history

4.11.5latest on PyPI · released Apr 12, 2026

Audit

Dependencies

apache-airflowrequiredProvider requires Airflow core

hdfsrequiredPython HDFS client library

apache-airflow-providers-apache-hiveoptionalOptional integration with Hive

Agent activity

52 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/apache/airflow ↗

packagehttps://pypi.org/project/apache-airflow-providers-apache-hdfs/ ↗

docshttps://airflow.apache.org/docs/apache-airflow-providers-apache-hdfs/4.11.5 ↗

changeloghttps://airflow.apache.org/docs/apache-airflow-providers-apache-hdfs/4.11.5/changelog.html ↗