Registry / devops / pulumi-databricks

pulumi-databricks

JSON →
library1.95.0pypypi✓ verified 87d ago

The `pulumi-databricks` Python package is an Infrastructure as Code (IaC) tool that enables developers to define, deploy, and manage Databricks cloud resources programmatically. It wraps the Databricks Terraform provider, providing Pythonic access to resources such as notebooks, clusters, jobs, and Unity Catalog entities. The library is actively maintained with frequent updates, typically released multiple times per month, reflecting the rapid development of its upstream Terraform provider. The current version is 1.90.0.

pip install pulumi-databricks
INSTALL
IMPORT
SIG · PULUMI-DATABRICKS
P
pulumi-databricks
devopspythonv1.95.0
Install
8.6s avg
Import
11420ms
Disk
112MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v1.95.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.920 runs
installs and imports cleanly · install 0.0s · import 12.341s · 115.9MB
glibc
py 3.103.920 runs
installs and imports cleanly · install 8.6s · import 10.499s · 102MB
112MB installed
● package 112MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

Provider
import pulumi_databricks as databricks
from pulumi_databricks.provider import Provider
While technically possible, the idiomatic Pulumi Python usage is to import the package as 'databricks' and refer to resources and types via `databricks.<ResourceName>` or `databricks.get_...` functions.
Notebook
from pulumi_databricks import Notebook
It is also common to use `import pulumi_databricks as databricks` and then `databricks.Notebook`.
Cluster
from pulumi_databricks import Cluster
It is also common to use `import pulumi_databricks as databricks` and then `databricks.Cluster`.

This quickstart program demonstrates how to configure the Pulumi Databricks provider and create a simple Databricks Notebook. It assumes Databricks authentication is handled via environment variables `DATABRICKS_HOST` and `DATABRICKS_TOKEN`. The example retrieves the current user's home directory to place the notebook, showcasing a common pattern for dynamic resource naming.

import pulumi import pulumi_databricks as databricks import os # Configure Databricks authentication using environment variables # DATABRICKS_HOST and DATABRICKS_TOKEN are typically set. # Example: export DATABRICKS_HOST="https://adb-YOUR_WORKSPACE_ID.1.azuredatabricks.net" # export DATABRICKS_TOKEN="dapiXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" # Retrieve current user information (useful for creating resources in user's home directory) current_user = databricks.get_current_user() # Create a Databricks Notebook hello_notebook = databricks.Notebook( "my-first-notebook", path=current_user.then(lambda user: f"{user.home}/pulumi_hello_world_notebook"), language="PYTHON", content_base64="# Databricks notebook created by Pulumi\nprint('Hello from Pulumi!')\n", # Note: content_base64 is discouraged for large notebooks; consider 'source' argument instead. # For real applications, you would typically load content from a file using base64.b64encode(open('path/to/notebook.py', 'rb').read()).decode('utf-8') ) pulumi.export("notebook_url", hello_notebook.url)
pulumi --version
Debug
Known issues
breakingMajor version upgrades of the underlying `terraform-provider-databricks` (often monthly) can introduce breaking changes in resource properties, behavior, or required arguments, even if `pulumi-databricks` itself remains at v1.x. Always review the `terraform-provider-databricks` changelog when upgrading `pulumi-databricks`.
fix
Consult the `terraform-provider-databricks` release notes for detailed breaking changes and adjust your Pulumi program accordingly. Test upgrades in a staging environment.
affects: All v1.x when underlying Terraform provider is upgraded.
gotchaUsing `content_base64` for `databricks.Notebook` is discouraged for large notebooks as it increases the Pulumi state file size and memory footprint. Consider using the `source` argument instead to specify a local file path for the notebook content.
fix
Prefer the `source` argument for `databricks.Notebook` to point to a local notebook file. Example: `source="./notebooks/my_notebook.py"`.
affects: All v1.x
gotchaWhen configuring authentication, ensure `databricks:token` is marked as a secret if using `pulumi config set` to prevent it from being stored in plaintext in the Pulumi state file.
fix
Use `pulumi config set databricks:token YYYYYYYYYYYYYY --secret` when setting the token via Pulumi configuration. Alternatively, use `DATABRICKS_TOKEN` environment variable which is the recommended approach for CI/CD.
affects: All v1.x
gotchaThe `accountId` provider argument should *not* be set for workspace-level providers. Setting it incorrectly can lead to 'invalid Databricks Account configuration errors' when performing workspace-specific operations.
fix
Only set the `databricks:accountId` configuration (or `DATABRICKS_ACCOUNT_ID` environment variable) when interacting with account-level Databricks resources (e.g., creating workspaces). Omit it for workspace-level resources.
affects: All v1.x
deprecatedThe `basic` authentication type for the provider, specified via `databricks:authType`, is deprecated.
fix
Use recommended authentication methods such as Personal Access Tokens (`pat`), Azure Service Principal (`azure-client-secret`), or other modern OIDC-based methods.
affects: All v1.x
Errors
Common errors & fixes
Error: Cannot access cluster ####-######-####### that was terminated or unpinned more than 30 days ago
This error occurs when trying to manage a Databricks cluster that has been terminated or unpinned and its metadata is no longer available in the Databricks API, typically after 30 days. The Pulumi state still references this old cluster.
fix
Upgrade `pulumi-databricks` to v0.5.5 or later if possible. If not, manually remove the cluster from the Pulumi state using `pulumi state rm urn:pulumi:<stack>::<project>::databricks:index/cluster:Cluster::<resource_name>` and then import the current state of your desired cluster, or create a new cluster resource.
Error: configuration for 'databricks:host' is required
The Databricks provider needs to know the host URL of your Databricks workspace to authenticate and interact with it. This configuration is missing.
fix
Set the `DATABRICKS_HOST` environment variable (e.g., `export DATABRICKS_HOST="https://adb-YOUR_WORKSPACE_ID.1.azuredatabricks.net"`) or configure it via Pulumi config (`pulumi config set databricks:host "https://..."`).
Error: configuration for 'databricks:token' is required
The Databricks provider requires an authentication token (Personal Access Token or similar) to authorize requests to your Databricks workspace.
fix
Set the `DATABRICKS_TOKEN` environment variable (e.g., `export DATABRICKS_TOKEN="dapiXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"`) or configure it securely via Pulumi config (`pulumi config set databricks:token YYYYYYYYYYYYYY --secret`).
Error: more than one authorization method configured
This typically happens when multiple authentication-related environment variables or Pulumi configuration settings are present and conflict (e.g., `DATABRICKS_HOST` and `DATABRICKS_CONFIG_FILE` are both set).
fix
Ensure you are using only one set of authentication credentials. For example, if using `~/.databrickscfg`, only set `DATABRICKS_CONFIG_FILE` or `databricks:configFile` and `databricks:profile`. If using host/token, only set `DATABRICKS_HOST`/`DATABRICKS_TOKEN` or `databricks:host`/`databricks:token`.
Upgrade
Version history
1.95.0latest on PyPI · released Jun 4, 2026
Audit
Dependencies
pulumirequiredCore Pulumi SDK for IaC operations.
terraform-provider-databricksoptionalThis Python provider is a wrapper around the official Databricks Terraform Provider.
Agent activity
38 hits · last 30 days
node
34
OpenAI (training)
1
Resources
pulumi-databricks — pip install pulumi-databricks · libregistry