Install & Compatibility
Where this runs
tested against v1.0.3 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.732s · 21.4MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 2.3s · import 0.656s · 22MB
19MB installed
● package 19MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
ResourceManager
✓ from yarn_api_client.resource_manager import ResourceManager
ApplicationMaster
✓ from yarn_api_client.application_master import ApplicationMaster
HistoryServer
✓ from yarn_api_client.history_server import HistoryServer
NodeManager
✓ from yarn_api_client.node_manager import NodeManager
This quickstart demonstrates how to initialize the `ResourceManager` and fetch a list of applications currently running or finished on the YARN cluster. It highlights configuring YARN endpoints and includes basic error handling for connection issues.
import os
from yarn_api_client.resource_manager import ResourceManager
# Configure YARN ResourceManager endpoints
# These can also be discovered automatically if YARN_CONF_DIR or HADOOP_CONF_DIR
# environment variables are set and point to valid Hadoop configuration.
# For local testing, ensure a YARN ResourceManager is running or adjust endpoint.
# Using a dummy endpoint if not set, for demonstration purposes.
# In a real scenario, replace with your actual YARN ResourceManager URL(s).
# Example for HA: ['http://rm1.example.com:8088', 'http://rm2.example.com:8088']
rm_endpoints = os.environ.get('YARN_RM_ENDPOINTS', 'http://localhost:8088').split(',')
if not rm_endpoints or rm_endpoints == ['']:
print("Warning: YARN_RM_ENDPOINTS environment variable not set. Using 'http://localhost:8088' as default.")
rm_endpoints = ['http://localhost:8088']
print(f"Attempting to connect to YARN ResourceManager at: {rm_endpoints}")
try:
resource_manager = ResourceManager(rm_endpoints)
# Fetch cluster applications
applications_response = resource_manager.cluster_applications()
if applications_response.apps:
print(f"Found {len(applications_response.apps)} applications.")
for app in applications_response.apps[:3]: # Print first 3 apps
print(f" Application ID: {app.id}, Name: {app.name}, State: {app.state}")
else:
print("No applications found on the YARN cluster.")
except Exception as e:
print(f"Error connecting to YARN ResourceManager or fetching applications: {e}")
print("Please ensure a YARN ResourceManager is running and accessible at the configured endpoint(s).")
print("You can set the YARN_RM_ENDPOINTS environment variable, e.g., export YARN_RM_ENDPOINTS='http://your-rm-host:8088'")
Debug
Known issues
breakingPython 2.7 support was officially dropped in version 1.0.3. Code written for Python 2.7 will likely fail with syntax errors or missing features.fixMigrate your codebase to Python 3.6+ to use versions 1.0.3 and later. If Python 2.7 is strictly required, pin the library version to <1.0.3.
affects: 1.0.3 and later
breakingVersion 1.0.0 introduced a major API cleanup. The `ResourceManager`, `ApplicationMaster`, `HistoryServer`, and `NodeManager` constructors no longer accept separate `address` and `port` parameters. Instead, they require complete endpoint URLs (e.g., `['http://localhost:8088']`). `ResourceManager` also now accepts a list of endpoints for HA support.fixUpdate constructor calls to pass full endpoint URLs as a list (even for a single endpoint). For example, `ResourceManager('localhost', 8088)` becomes `ResourceManager(['http://localhost:8088'])`. affects: 1.0.0 and later
gotchaWhen using YARN in High Availability (HA) mode, ensure you provide a list of all active ResourceManager endpoints to the `ResourceManager` constructor. The client will attempt to connect to the active RM from the provided list.fixPass a list of all ResourceManager URLs: `ResourceManager(['http://rm1.example.com:8088', 'http://rm2.example.com:8088'])`.
affects: 1.0.0 and later
gotchaThe library can automatically discover Hadoop configuration by checking `YARN_CONF_DIR` or `HADOOP_CONF_DIR` environment variables. If these are set, explicit endpoints provided in the constructor might be overridden or interact unexpectedly with discovered configurations.fixBe mindful of these environment variables. If you wish to explicitly control endpoints, ensure these variables are not set or that your explicit endpoint configuration takes precedence as expected. Consult the official documentation for precedence rules.
affects: All versions with configuration discovery (from 0.3.4, enhanced in 1.0.3)
gotchaOlder YARN deployments or certain API calls might return empty JSON responses, which could cause parsing errors in the client. Version 1.0.2 improved handling of such cases.fixUpgrade to version 1.0.2 or later to benefit from improved robustness when handling empty or malformed YARN responses. Implement robust error handling and check for empty content in your application logic.
affects: Prior to 1.0.2
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'yarn_api_client'
The Python `yarn-api-client` library is either not installed in your environment or there is a discrepancy between the package name used for installation (yarn-api-client) and the import name (yarn_api_client).
fixEnsure the library is correctly installed using `pip install yarn-api-client` and that your virtual environment is active.
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused'))
The `yarn-api-client` failed to establish a connection with the YARN ResourceManager, likely due to an incorrect URL, the ResourceManager service not running, or network/firewall issues.
fixVerify that the YARN ResourceManager URL (e.g., `http://localhost:8088`) is correct and accessible, ensure the ResourceManager service is running, and check any relevant network configurations or firewalls.
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The YARN API endpoint returned a response that was not valid JSON (e.g., an HTML error page for a 404 or 500 error), which the client attempted to parse as JSON.
fixDouble-check the YARN ResourceManager URL and the specific API path being accessed (e.g., `/ws/v1/cluster/info`) to ensure it's correct and that the YARN service is returning expected JSON responses.
AttributeError: 'YARNClient' object has no attribute 'get_all_applications'
You are attempting to call a method that does not exist on the `YARNClient` object. The method name is likely misspelled or refers to a non-existent function.
fixConsult the `yarn-api-client` documentation for the correct method names. For retrieving applications, use `client.get_applications()`.
TypeError: 'NoneType' object is not subscriptable
This error occurs when you try to access an item (e.g., using `[]`) on an object that is `None`. In `yarn-api-client`, this often means an API call like `get_application()` returned `None` because the requested resource (e.g., application) was not found.
fixAlways check if the result of an API call is `None` before attempting to access its properties. For example: `app = client.get_application(app_id); if app: print(app['state']) else: print(f'Application {app_id} not found.')` Upgrade
Version history
1.0.3latest on PyPI · released Nov 16, 2021
Audit
Dependencies
requestsrequiredUsed for making HTTP requests to the YARN API.
requests-kerberosoptionalProvides Kerberos/SPNEGO authentication support, optional since v0.3.2.