Install & Compatibility
Where this runs
tested against v1.12.4 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
build_error
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 26.0s · import 9.768s · 482MB
488MB installed
● package 488MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
DatabricksAdapter
✓ from dbt.adapters.databricks import DatabricksAdapter
This is an internal dbt-core import for adapter registration and is not typically used directly by end-users. `dbt-databricks` functionality is primarily accessed via the dbt CLI and `profiles.yml` configuration.
This quickstart demonstrates setting up environment variables for Databricks authentication (using a Personal Access Token) and outlines the necessary `profiles.yml` configuration and dbt CLI commands to initialize a project, test connectivity, and run dbt models on Databricks. Ensure you replace placeholder values with your actual Databricks workspace URL, SQL warehouse ID, and Unity Catalog name (if applicable).
import os
import sys
# --- Step 1: Set up environment variables for authentication ---
# In a real scenario, set DBT_DATABRICKS_TOKEN securely,
# e.g., via your shell or CI/CD secrets.
# For local testing, replace 'YOUR_DATABRICKS_PAT' with an actual PAT.
# Using os.environ.get for compliance with auth checks.
databricks_token = os.environ.get('DBT_DATABRICKS_TOKEN', 'YOUR_DATABRICKS_PAT_FOR_QUICKSTART_ONLY')
if databricks_token == 'YOUR_DATABRICKS_PAT_FOR_QUICKSTART_ONLY':
print("WARNING: DBT_DATABRICKS_TOKEN not set in environment. Using placeholder. Ensure you replace it.")
os.environ['DBT_DATABRICKS_TOKEN'] = databricks_token
# --- Step 2: Verify dbt-databricks installation ---
try:
import dbt.adapters.databricks # Check if the package is findable
print("dbt-databricks is installed and accessible.")
except ImportError:
print("ERROR: dbt-databricks not found. Please run 'pip install dbt-databricks'")
sys.exit(1)
print("\n--- Quickstart: Next Steps (run these in your terminal) ---")
print("1. Configure your dbt profile in `~/.dbt/profiles.yml`:")
print(" Replace placeholders like `<your-databricks-workspace-url>`, etc.")
print(" Example `profiles.yml` snippet (named `my_databricks_project`):")
print("\n my_databricks_project:")
print(" target: dev")
print(" outputs:")
print(" dev:")
print(" type: databricks")
print(" host: <your-databricks-workspace-url> # e.g., dbc-xxxx.cloud.databricks.com")
print(" http_path: /sql/1.0/warehouses/<your-sql-warehouse-id> # For SQL Endpoints")
print(" token: ""{{ env_var('DBT_DATABRICKS_TOKEN') }}"" # Uses the env var set above")
print(" catalog: <your-unity-catalog-name> # Optional, if using Unity Catalog")
print(" schema: dbt_quickstart_schema")
print(" threads: 4")
print("\n2. Initialize a new dbt project and link your profile:")
print(" mkdir my_dbt_project_databricks && cd my_dbt_project_databricks")
print(" dbt init --skip-profile-setup")
print(" # Edit `dbt_project.yml` to set `profile: 'my_databricks_project'`")
print("\n3. Test your Databricks connection:")
print(" dbt debug --profile my_databricks_project")
print("\n4. Run your dbt models (after creating some in the `models` directory):")
print(" dbt run --profile my_databricks_project")
dbt --version
Debug
Known issues
breakingdbt-databricks adapter versions must align with the `dbt-core` version they are built for. Upgrading `dbt-core` without upgrading `dbt-databricks` (or vice-versa) can lead to unexpected errors, incompatible syntax, or unhandled features.fixAlways install the `dbt-databricks` version that is explicitly compatible with your `dbt-core` version. Check the official dbt Labs release notes and documentation for compatibility matrices before upgrading.
affects: All versions, particularly across major `dbt-core` upgrades (e.g., from `dbt-core` 1.x to 1.y).
gotchaIncorrect `http_path` configuration in `profiles.yml` is a frequent cause of connection failures. The required path varies significantly between Databricks SQL Endpoints (recommended) and older cluster-based connections.fixFor Databricks SQL Endpoints, use `/sql/1.0/warehouses/<YOUR_SQL_WAREHOUSE_ID>`. For legacy interactive clusters, use `/sql/protocolv1/o/<YOUR_WORKSPACE_ID>/<YOUR_CLUSTER_ID>`. Ensure the ID is correct and accessible to your user/service principal.
affects: All versions
gotchaDatabricks authentication methods (Personal Access Tokens via `token`, Azure AD Service Principals via `client_id`/`client_secret`, or OAuth) have different security and configuration requirements. Using PATs is simpler for development but less secure for production. Azure AD Service Principals or OAuth are preferred for production Azure Databricks deployments.fixFor production environments, prioritize more secure methods like Azure AD Service Principal or OAuth. Ensure environment variables (`DBT_DATABRICKS_TOKEN`, `DBT_DATABRICKS_CLIENT_ID`, `DBT_DATABRICKS_CLIENT_SECRET`) are correctly set and referenced in `profiles.yml` using `{{ env_var('...') }}`. affects: All versions
gotchaWhen working with Unity Catalog, the `catalog` parameter in `profiles.yml` is crucial. Omitting it or providing an incorrect catalog name can lead to 'schema not found' or 'permission denied' errors, even if the schema exists in a different catalog.fixExplicitly set the `catalog` parameter in your `profiles.yml` to the correct Unity Catalog name. Verify that your Databricks user or service principal has the necessary `USE CATALOG`, `USE SCHEMA`, and other required permissions on the target catalog and schema.
affects: All versions supporting Unity Catalog
gotchaSome Python packages required by dbt-databricks (or its dependencies, like `lz4`) contain C extensions that need to be compiled during installation. This requires system-level build tools (e.g., `gcc`, `python3-dev`) which are often missing in minimal Docker images (e.g., `alpine`).fixEnsure your Dockerfile or environment has the necessary build tools installed. For Alpine Linux, add `RUN apk add --no-cache gcc musl-dev python3-dev`. For Debian/Ubuntu, use `apt-get update && apt-get install -y build-essential python3-dev`.
affects: All versions, particularly when installing in minimal container environments (e.g., `alpine` based images).
Errors
Common errors & fixes
Database error, failed to connect
This general connection error often indicates issues with the authentication details (host, http_path, token) provided in your `profiles.yml` file, network restrictions, expired credentials, or insufficient permissions for the Databricks user/service principal.
fixVerify that your `profiles.yml` has the correct `host` (without 'https://'), `http_path` (with a leading slash), and a valid `token` or OAuth configuration. Ensure your Databricks Personal Access Token (PAT) has not expired and has the necessary permissions. Check any network firewalls or IP access lists that might be blocking the connection. Running `dbt debug` provides more detailed diagnostics.
Credentials in profile "<profile_name>", target "<target_name>" invalid: xxx is not of type 'string'
A numeric-looking value (e.g., a long ID) in your `profiles.yml` that should be treated as a string is being implicitly converted to an integer by the YAML parser, causing a type mismatch error.
fixEnclose the problematic value (e.g., `warehouse_id`, `organization` ID) in double quotes in your `profiles.yml` file to explicitly define it as a string. For example, change `warehouse_id: 1234567890` to `warehouse_id: "1234567890"`.
ModuleNotFoundError: No module named 'dbt.adapters.databricks'
The `dbt-databricks` adapter, which allows dbt-core to connect to Databricks, is either not installed or is not accessible in the Python environment where dbt commands are being run.
fixEnsure that `dbt-databricks` is installed in your active Python environment using `pip install dbt-databricks`. If you are using virtual environments, make sure the correct environment is activated before executing any `dbt` commands.
TABLE_ALREADY_EXISTS
When running `dbt run` with `view` materializations, especially in repeated executions without code changes, the `dbt-databricks` adapter might encounter a conflict where it attempts to create a view that already exists, rather than replacing it idempotently.
fixThis can sometimes be mitigated by ensuring a clean state (e.g., dropping views manually before a `dbt run`) or by running `dbt run --full-refresh`. Review the dbt-databricks GitHub issues for similar problems, as this might be a known bug or a specific behavior that requires an adapter update or configuration tweak.
Your administrator has only allowed sql and python commands on this cluster. This execution contained at least one disallowed language.
The Databricks cluster or SQL Endpoint where dbt is attempting to execute commands is configured with language restrictions, typically allowing only SQL and Python, but a part of the dbt job (e.g., a pre-hook, post-hook, or an internal dbt mechanism) tried to run code in another disallowed language.
fixVerify that all dbt models and any custom hooks or macros exclusively use SQL or Python. If the issue persists, consult your Databricks administrator to review the cluster's language settings. If another language is critical for your workflow, the cluster configuration might need to be adjusted to include it, though this is generally not recommended for dbt operations.
Upgrade
Version history
1.12.4latest on PyPI · released Aug 12, 2026
Audit
Dependencies
dbt-corerequireddbt-databricks is an adapter for dbt-core and depends on it implicitly for functionality. Installing dbt-databricks will typically pull in a compatible dbt-core version.