pydruid

library0.6.9pypypi✓ verified 26d ago

PyDruid is a Python connector for Apache Druid, providing a simple API to create, execute, and analyze Druid queries. It can parse query results into Pandas DataFrame objects for seamless integration with the SciPy stack. The library offers both synchronous and asynchronous clients, implements the Python DB API 2.0, and includes a SQLAlchemy dialect. The current version is 0.6.9, and it maintains an active development and release cadence.

database data serialization

pip install pydruid

github ↗package ↗homepage ↗docs ↗

INSTALL

IMPORT

SIG · PYDRUID

pydruid

databasepythonv0.6.9

Install

7.8s avg

Import

11ms

Disk

119MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.6.9 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.910 runs

installs and imports cleanly · install 0.0s · import 0.012s · 22.7MB

glibc

py 3.10–3.910 runs

installs and imports cleanly · install 7.8s · import 0.011s · 23MB

119MB installed

● package 119MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

PyDruid

✓ from pydruid.client import PyDruid

connect

✓ from pydruid.db import connect

doublesum

✓ from pydruid.utils.aggregators import doublesum

Filter

✓ from pydruid.utils.filters import Filter

This quickstart demonstrates how to establish a connection to Druid using the Python DB API 2.0 interface and execute a basic SQL query. Ensure your Druid cluster is running and accessible at the specified host and port. Connection parameters can be provided via environment variables DRUID_HOST, DRUID_PORT, DRUID_PATH, and DRUID_SCHEME.

import os from pydruid.db import connect # Configure connection details (replace with your Druid Broker/Router info) DRUID_HOST = os.environ.get('DRUID_HOST', 'localhost') DRUID_PORT = int(os.environ.get('DRUID_PORT', '8082')) DRUID_PATH = os.environ.get('DRUID_PATH', '/druid/v2/sql/') DRUID_SCHEME = os.environ.get('DRUID_SCHEME', 'http') try: conn = connect( host=DRUID_HOST, port=DRUID_PORT, path=DRUID_PATH, scheme=DRUID_SCHEME ) curs = conn.cursor() curs.execute("SELECT COUNT(*) FROM wikipedia") result = curs.fetchone() print(f"Query result: {result}") except Exception as e: print(f"Error connecting to Druid or executing query: {e}")

pydruid --version

Debug

Known issues

breakingFor Druid SQL versions >= 0.13.0, to ensure correct column name inference, especially with empty result sets, you should set `header=true` in the query context. PyDruid defaults to `header=false` for backward compatibility, which can lead to issues inferring column names.

fix

Pass `context={'sqlQueryId': '...', 'header': True}` in your query parameters, or ensure your Druid configuration forces header output if applicable for all queries.

affects: <= 0.6.x (when interacting with Druid >= 0.13.0)

breakingOlder versions of pydruid (< 0.6.0) experienced compatibility issues with Python 3.8.2 and newer due to changes in how `collections.abc` was imported, leading to `AttributeError: module 'collections' has no attribute 'abc'`.

fix

Upgrade pydruid to version 0.6.0 or newer to ensure compatibility with modern Python versions.

affects: < 0.6.0 with Python >= 3.8.2

deprecatedThe `export_pandas()` and `export_tsv()` methods directly on `BaseDruidClient` (e.g., `PyDruid` instances) are deprecated. These methods now reside on the `Query` object returned by query methods.

fix

After executing a query (e.g., `query_result = client.timeseries(...)`), call `query_result.export_pandas()` or `query_result.export_tsv(dest_path)` instead.

affects: All versions where `Query` object exists

gotchaWhen using `pydruid`'s SQLAlchemy dialect, `sqlalchemy` is an optional dependency. If you install `pydruid` without the `[sqlalchemy]` extra (e.g., `pip install pydruid`), the SQLAlchemy integration will not work due to missing dependencies.

fix

Install `pydruid` with the `[sqlalchemy]` extra: `pip install pydruid[sqlalchemy]`.

affects: All versions

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'pydruid'

The `pydruid` library is not installed in the current Python environment or the Python interpreter cannot locate it in its search path.

fix

Install the library using pip: `pip install pydruid` or `pip install pydruid[pandas]` if planning to use Pandas integration.

OSError: HTTP Error 400: Bad Request

This error typically indicates that the Druid server received a request that it considered malformed or invalid, often due to an incorrectly formatted query or an inaccessible endpoint.

fix

Verify the Druid query syntax, ensure the `url` and `endpoint` provided to the `PyDruid` client are correct (e.g., `http://localhost:8082/druid/v2/sql/` for SQL queries), and check Druid server logs for more specific error details.

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This error occurs when `pydruid` expects a JSON response from the Druid server but receives something else, such as an empty response, an HTML error page, or plain text.

fix

Check the Druid server status and logs to ensure it's running and responding with valid JSON. This can also happen if the client is pointed to an incorrect URL that serves non-JSON content, like a human-readable web page.

AttributeError: module 'urllib' has no attribute 'request'

This error arises from compatibility issues, specifically when `pydruid` code written for Python 2's `urllib` module is executed in a Python 3 environment, where `urllib`'s structure (e.g., `urllib.request`, `urllib.error`) changed significantly.

fix

Ensure you are using a `pydruid` version compatible with your Python 3 environment, or update your Python 2 code to use the Python 3 `urllib` syntax. This specific `AttributeError` was common in older `pydruid` versions run on Python 3.

Upgrade

Version history

0.6.9latest on PyPI · released May 16, 2024

Audit

Dependencies

pandasoptionalRequired for exporting query results to Pandas DataFrames.

tornadooptionalRequired for the asynchronous client (pydruid.async_client).

sqlalchemyoptionalRequired for using the SQLAlchemy dialect.

prompt_toolkitoptionalRequired for the command line interface (CLI).

Agent activity

10 hits · last 30 days

node

Resources

githubhttps://github.com/druid-io/pydruid ↗

packagehttps://pypi.org/project/pydruid/ ↗

homepagehttps://druid.apache.org ↗

docshttps://pythonhosted.org/pydruid/ ↗