Registry / database / databricks-sql-connector

databricks-sql-connector

library4.4.0pypypi✓ verified 27d ago

The Databricks SQL Connector for Python is a Python library that enables running SQL commands on Databricks clusters and SQL warehouses. It is a Thrift-based client, conforms to the Python DB API 2.0 specification, and uses Apache Arrow for efficient data exchange. The library is actively maintained with frequent releases, often multiple times a month.

database

pip install databricks-sql-connector

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · DATABRICKS-SQL-CON

databricks-sql-connector

databasepythonv4.4.0

Install

11.0s avg

Import

38ms

Disk

333MB

Pass rate

5/ 10

Env Coverage5 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v4.4.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.910 runs

build_error

glibc

py 3.10–3.910 runs

installs and imports cleanly · install 11.0s · import 0.038s · 331MB

333MB installed

● package 333MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

sql

✓ from databricks import sql

This quickstart demonstrates how to establish a connection to a Databricks SQL warehouse using a Personal Access Token (PAT) and execute a simple query. Ensure that `DATABRICKS_SERVER_HOSTNAME`, `DATABRICKS_HTTP_PATH`, and `DATABRICKS_TOKEN` environment variables are set with your Databricks connection details.

import os from databricks import sql # Ensure these environment variables are set: # DATABRICKS_SERVER_HOSTNAME, DATABRICKS_HTTP_PATH, DATABRICKS_TOKEN host = os.environ.get('DATABRICKS_SERVER_HOSTNAME', 'your_server_hostname.databricks.com') http_path = os.environ.get('DATABRICKS_HTTP_PATH', '/sql/1.0/endpoints/your_sql_warehouse_id') access_token = os.environ.get('DATABRICKS_TOKEN', 'dapiXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX') if not all([host, http_path, access_token]): print("Please set DATABRICKS_SERVER_HOSTNAME, DATABRICKS_HTTP_PATH, and DATABRICKS_TOKEN environment variables.") else: try: with sql.connect( server_hostname=host, http_path=http_path, access_token=access_token ) as connection: with connection.cursor() as cursor: cursor.execute("SELECT 1 as id, 'hello' as message") result = cursor.fetchall() for row in result: print(row) except Exception as e: print(f"An error occurred: {e}")

Debug

Known issues

breakingPyArrow is no longer a default dependency since version 4.0.0. Users must explicitly install `databricks-sql-connector[pyarrow]` or `pip install pyarrow` to enable Arrow-based features like CloudFetch and `fetchmany_arrow`. Failing to do so may impact performance for large datasets.

fix

Install with `pip install databricks-sql-connector[pyarrow]` or `pip install pyarrow` separately.

affects: >=4.0.0

breakingThe SQLAlchemy dialect for Databricks was split into a separate `databricks-sqlalchemy` package in version 4.0.0. Users leveraging SQLAlchemy must now explicitly install `databricks-sqlalchemy` alongside the core connector.

fix

Install the separate SQLAlchemy dialect: `pip install databricks-sqlalchemy`.

affects: >=4.0.0

gotchaWhen using `pandas.read_sql` directly with a `databricks-sql-connector` connection object (v3.0.0+), a `UserWarning` regarding DB API 2.0 support might appear. While this warning can generally be ignored due to PyArrow's efficiency, using `databricks-sqlalchemy` as the engine provides a warning-free experience and broader compatibility.

fix

Optionally, use the `databricks-sqlalchemy` engine for `pandas.read_sql` to suppress the warning or if broader SQLAlchemy compatibility is needed.

affects: >=3.0.0

breakingThe `pandas.DataFrame.to_sql()` method effectively broke for inserts exceeding 255 values (not rows) into Delta tables with `databricks-sql-connector` versions 3.0.0 and above, due to the introduction of native parameters and a server-side limitation.

fix

Consider using alternative methods for large inserts, or a dedicated fix package like `pandas-tosql-dbx-fix` which compiles the SQL query before sending it.

affects: >=3.0.0

breakingVersion 4.2.0 introduced changes to `autocommit` properties on the connection object. Directly setting `connection.autocommit = False` via API calls (e.g., in an Airflow hook) can now lead to `TransactionError: [CONFIG_NOT_AVAILABLE] Configuration AUTOCOMMIT is not available. SQLSTATE: 42K0I`. Multi-statement transaction control should be managed through `connection.autocommit = False` then `connection.commit()` and `connection.rollback()`.

fix

Ensure `autocommit` is managed correctly by the connector's transaction methods. For explicit control, set `autocommit=False` during connection establishment or directly on the connection object, then use `commit()`/`rollback()`. Avoid direct setting if it's causing the `CONFIG_NOT_AVAILABLE` error.

affects: >=4.2.0

gotchaWhile PyPI metadata states Python >=3.8.0 is supported, the official GitHub README for the latest versions (4.x.x) recommends Python 3.9 or above for full compatibility and access to the latest features and stability.

fix

Ensure your environment uses Python 3.9 or newer when working with `databricks-sql-connector` versions 4.x.x.

affects: >=4.0.0

breakingInstallation fails with `error: command 'gcc' failed` when building the `lz4` dependency (e.g., in minimal environments like Alpine Linux). This occurs because `lz4` requires C compilation, and build tools are not present.

fix

Ensure C build tools (e.g., `gcc`, `build-base` for Alpine) are installed in your environment before installing `databricks-sql-connector`.

affects: >=4.0.0

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'databricks'

This error occurs when you try to import `databricks.sql` but the `databricks-sql-connector` package is not installed, or you are attempting to use an outdated import path or a different Databricks-related package like `databricks-connect` which has a different import structure.

fix

Ensure `databricks-sql-connector` is installed using `pip install databricks-sql-connector`. The correct import for the SQL connector is `from databricks.sql import connect`.

databricks.sql.exc.RequestError: Error during request to server

This is a generic connection error, often indicating issues with the provided `server_hostname`, `http_path`, or an invalid or expired `access_token` (Personal Access Token).

fix

Double-check that `server_hostname`, `http_path`, and `access_token` are correct and have the necessary permissions. Ensure there are no typos, leading/trailing spaces, or incorrect environment variable configurations. Also, confirm network connectivity to the Databricks workspace.

AttributeError: 'NoneType' object has no attribute 'startswith'

This error typically occurs when one of the required connection parameters (like `server_hostname`, `http_path`, or `access_token`) is passed as `None` to the `sql.connect()` function, often because an environment variable was not set or retrieved correctly.

fix

Verify that all connection parameters are explicitly set and are not `None`. For example, if using environment variables, ensure they are defined in your execution environment: `os.getenv("DATABRICKS_SERVER_HOSTNAME")` must return a string, not `None`.

ModuleNotFoundError: No module named 'packaging'

This error means that a required dependency, the `packaging` module, is missing from your Python environment, which `databricks-sql-connector` relies on.

fix

Install the missing `packaging` library: `pip install packaging`.

AttributeError: 'Cursor' object has no attribute 'active_op_handle'

This `AttributeError` indicates an incompatibility, typically with older versions of libraries that depend on `databricks-sql-connector` (e.g., `dbt-databricks`). These dependent libraries might be trying to access the `active_op_handle` attribute on the `Cursor` object, which was present in earlier versions of `databricks-sql-connector` but has since been removed or changed.

fix

Pin the `databricks-sql-connector` version to an earlier compatible version (e.g., `pip install databricks-sql-connector==4.0.5`) or update the dependent library (like `dbt-databricks`) to a version that is compatible with your current `databricks-sql-connector` version.

Upgrade

Version history

4.4.0latest on PyPI · released Jul 22, 2026

Audit

Dependencies

pythonrequiredRuntime environment

pyarrowoptionalOptional, for CloudFetch and Apache Arrow-based data transfer features. Not installed by default in v4.0.0+.

databricks-sdkoptionalOptional, required for OAuth Machine-to-Machine (M2M) and User-to-Machine (U2M) authentication.

Agent activity

7 hits · last 30 days

node

Resources

githubhttps://github.com/databricks/databricks-sql-python ↗

packagehttps://pypi.org/project/databricks-sql-connector/ ↗

homepagehttps://www.databricks.com ↗