Registry / database / clickhouse-sqlalchemy

clickhouse-sqlalchemy

JSON →
library0.3.2pypypi✓ verified 25d ago

The `clickhouse-sqlalchemy` library provides a SQLAlchemy dialect for connecting to and interacting with ClickHouse databases. It enables users to leverage SQLAlchemy's ORM and SQL Expression Language for querying and manipulating data in ClickHouse. The current version is `0.3.2`. The project releases updates on an as-needed basis rather than a fixed schedule.

pip install clickhouse-sqlalchemy
INSTALL
IMPORT
SIG · CLICKHOUSE-SQLALCH
C
clickhouse-sqlalchemy
databasepythonv0.3.2
Install
6.0s avg
Import
702ms
Disk
99MB
Pass rate
8/ 10
Env Coverage8 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.3.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
glibc
py 3.10
✕ build_error
✓ 6.1s
py 3.11
✓ —
✓ 5.4s
py 3.12
✓ —
✓ 5.8s
py 3.13
✓ —
✓ 5.8s
py 3.9
✕ build_error
✓ 7s
99MB installed
● package 99MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

create_engine
from sqlalchemy import create_engine
from clickhouse_sqlalchemy import create_engine
SQLAlchemy dialects are activated via the engine connection string; `create_engine` itself is from `sqlalchemy`.

This quickstart demonstrates how to establish a connection to a ClickHouse database using SQLAlchemy's `create_engine` and execute basic SQL queries, including DDL and DML operations. It uses environment variables for connection details for secure and flexible configuration. A running ClickHouse instance is required.

import os from sqlalchemy import create_engine, text CH_HOST = os.environ.get('CLICKHOUSE_HOST', 'localhost') CH_PORT = os.environ.get('CLICKHOUSE_PORT', '8123') CH_USER = os.environ.get('CLICKHOUSE_USER', 'default') CH_PASSWORD = os.environ.get('CLICKHOUSE_PASSWORD', '') CH_DATABASE = os.environ.get('CLICKHOUSE_DATABASE', 'default') try: # Establish a connection to ClickHouse engine = create_engine( f'clickhouse://{CH_USER}:{CH_PASSWORD}@{CH_HOST}:{CH_PORT}/{CH_DATABASE}' ) # Execute a simple query with engine.connect() as connection: result = connection.execute(text('SELECT 1 as one')).scalar() print(f"Query result: {result}") # Example: Create a table and insert data with engine.connect() as connection: connection.execute(text('DROP TABLE IF EXISTS my_test_table')) connection.execute(text('CREATE TABLE my_test_table (id Int32, name String) ENGINE = MergeTree ORDER BY id')) connection.execute(text('INSERT INTO my_test_table (id, name) VALUES (1, \'Alice\'), (2, \'Bob\')')) connection.commit() # Query data rows = connection.execute(text('SELECT id, name FROM my_test_table ORDER BY id')).fetchall() print(f"Table data: {rows}") except Exception as e: print(f"An error occurred: {e}") print("Ensure a ClickHouse instance is running and connection details are correct.") print("You can set environment variables like CLICKHOUSE_HOST, CLICKHOUSE_PORT, etc.")
Debug
Known issues
breakingThe aliases `ClickHouseEngine` and `create_session` were removed in version `0.3.0`. Users should directly use `sqlalchemy.create_engine` and standard SQLAlchemy session management.
fix
Replace `ClickHouseEngine(...)` with `create_engine('clickhouse://...')` and `create_session(...)` with `sqlalchemy.orm.sessionmaker(bind=engine)()`.
affects: >=0.3.0
breakingThe default string type mapping changed from `Nullable(String)` to `String` in version `0.3.0`. This affects DDL generation; columns defined as `String` in SQLAlchemy will no longer implicitly be `Nullable(String)` in ClickHouse.
fix
If nullable strings are desired, explicitly define columns using `sa.Column('name', sa.String(255), nullable=True)` or import `NullableString` from `clickhouse_sqlalchemy.types`.
affects: >=0.3.0
gotchaClickHouse's `DateTime` and `DateTime64` types have strict requirements for timezones and precision. SQLAlchemy's default `DateTime` might not always align perfectly, leading to timezone or precision issues during data insertion/retrieval.
fix
Use specific types like `DateTime(timezone=...)` or `DateTime64(precision=..., timezone=...)` from `clickhouse_sqlalchemy.types` for precise control over timestamp columns.
affects: <all>
gotchaAdvanced ClickHouse data types (e.g., `Nested`, `Array`, `Map`, `AggregateFunction`) and features (e.g., `FINAL` modifier, complex table engines) often lack direct, high-level ORM support. Attempting to use them with the ORM might require complex workarounds or fail.
fix
For complex types and features, it's often more robust to use `engine.execute(text('RAW SQL HERE'))` to run native ClickHouse queries directly, bypassing the ORM layer.
affects: <all>
Errors
Common errors & fixes
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:clickhouse.native
This error typically occurs when SQLAlchemy cannot find and load the `clickhouse` dialect plugin. This can happen due to an incomplete installation, issues with how Python environments are packaged (e.g., PyInstaller), or if the dialect isn't properly registered.
fix
Ensure `clickhouse-sqlalchemy` is correctly installed. If packaging, ensure the dialect is included. Sometimes, explicitly importing `clickhouse_sqlalchemy` or registering the dialect can help: `from clickhouse_sqlalchemy import dialect` (though usually not necessary with a proper install).
sqlalchemy.exc.OperationalError: (OperationalError) (Code: 516, Authentication failed)
This specific authentication failure, particularly with `Code: 516`, has been linked to incompatibilities with the `infi.clickhouse_orm` library, where newer versions of `infi.clickhouse_orm` do not pass the password as expected by `clickhouse-sqlalchemy`.
fix
Downgrade the `infi.clickhouse_orm` package to version `1.0.4` or earlier. Add `infi.clickhouse_orm==1.0.4` to your `requirements.txt` and reinstall.
ERROR: Cannot install -r requirements.txt (line X), clickhouse-sqlalchemy==0.3.2 and sqlalchemy==1.4.54 because these package versions have conflicting dependencies.
This indicates a dependency conflict where `clickhouse-sqlalchemy` version 0.3.2 requires a specific range of SQLAlchemy versions (e.g., `>=2.0.0, <2.1.0`), but the project is attempting to install an incompatible SQLAlchemy version (e.g., `1.4.x`).
fix
Adjust your `requirements.txt` or `pip install` commands to use a SQLAlchemy version compatible with `clickhouse-sqlalchemy==0.3.2`. For example, use `sqlalchemy==2.0.x` (where `x` is a compatible patch version).
AttributeError: 'GenericTypeCompiler' object has no attribute 'visit_nullable'
This error occurs when attempting to reflect tables with `Nullable` columns, indicating that the dialect's type compiler cannot properly handle the `Nullable` type during metadata processing.
fix
This was a bug in earlier versions of `clickhouse-sqlalchemy` related to column compilation and `Nullable` types. Update `clickhouse-sqlalchemy` to a version where this fix has been merged (e.g., 0.1.6 or newer, or the latest 0.3.2).
AttributeError: 'NoneType' object has no attribute 'encode'
This error typically arises when attempting to insert an object into a table via `session.add()` and `session.commit()`, where a `Nullable(String)` column is not explicitly set (remaining `None`), and the underlying `clickhouse-driver` attempts to encode `None` as a string.
fix
While `session.execute(table.insert(), data)` often works correctly, when using ORM-style `session.add()`, ensure that `Nullable(String)` columns are either explicitly set to `None` or an empty string (`''`) if `None` is not handled gracefully by the driver in that context. This issue was reported in earlier versions and might be resolved by updating the library.
Upgrade
Version history
0.3.2latest on PyPI · released Jun 12, 2024
Audit
Dependencies

No dependency data recorded yet.

Agent activity
8 hits · last 30 days
node
6
OpenAI (training)
1
Resources
clickhouse-sqlalchemy — pip install clickhouse-sqlalchemy · libregistry