Install & Compatibility
Where this runs
tested against v2.0.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.13
✕ build_error
✕ build_error
99MB installed
● package 99MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
Workflow
✓ from brickflow import Workflow
TaskSettings
✓ from brickflow import TaskSettings
Cluster
✓ from brickflow import Cluster
Project
✓ from brickflow import Project
ctx
✓ from brickflow.context import ctx
Used for accessing Databricks utilities and secrets within tasks.
This quickstart demonstrates how to define a simple 'Hello World' Databricks workflow using Brickflows. It includes defining a workflow with a default cluster and a single task. To deploy, you must first set up a Brickflow project using the `brickflow` CLI and ensure the Databricks CLI is configured to authenticate with your workspace.
from datetime import timedelta
from brickflow import Workflow, Cluster, TaskSettings
import os
# Configure Databricks host and token via environment variables or databricks configure --token
# For this example, ensure your ~/.databrickscfg is set up or environment variables are available.
# Example: export DATABRICKS_HOST="https://<your-workspace-url>.cloud.databricks.com"
# export DATABRICKS_TOKEN="dapi..."
wf = Workflow(
"hello_world_workflow",
default_cluster=Cluster(
name="brickflow-example-cluster",
spark_version='12.2.x-scala2.12',
node_type_id='Standard_DS3_v2',
num_workers=1
),
default_task_settings=TaskSettings(
timeout_seconds=timedelta(hours=2).seconds
)
)
@wf.task()
def hello_task():
print(f"Hello from Databricks! Host: {os.environ.get('DATABRICKS_HOST', 'N/A')}")
return "Task completed successfully"
# To run this, you would typically use the brickflows CLI:
# 1. Create a project: `mkdir my_brickflow_project && cd my_brickflow_project && brickflow projects add`
# 2. Place this code in `workflows/hello_world_wf.py` (assuming workflows is your workflows directory).
# 3. Deploy: `brickflow deploy --deploy-mode=bundle -p <your_databricks_profile>`
brickflows --version
Debug
Known issues
breakingDeployment failure if 'health' rules were not set (regression in some v1.4.x releases).fixUpgrade to v1.4.1 or later. The regression introduced by #237 was fixed in #245 (v1.4.1), which allows workflows to deploy without explicit health rules.
affects: v1.4.0 (and potentially immediate patches leading up to v1.4.1)
gotchaThe `WorkflowDependencySensor` might not fail when an invalid `dependency_job_id` is provided, leading to silent failures or unexpected behavior.fixUpgrade to v1.3.2 or later, which includes a fix for this behavior. Always thoroughly test dependency sensors.
affects: Up to v1.3.1
gotchaDefault task settings like `timeout_seconds` might not apply correctly to 'If/Else' or 'For Each' task types in older versions.fixUpgrade to v1.3.3 or later to ensure default task settings are applied consistently across all task types.
affects: Up to v1.3.2
gotchaRunning a single task directly via programmatic methods in Databricks workflows is not straightforward due to architectural differences from tools like Airflow. Brickflows provides a specific UI-based mechanism.fixTo run specific tasks for debugging or testing, use the Databricks UI's 'Run now with different parameters' option and set `brickflow_internal_only_run_tasks` with a comma-separated list of task names. Avoid modifying other parameters.
affects: All versions
gotchaProject setup is crucial for correct module resolution and deployment. Incorrectly configured project roots or workflow directories can lead to import errors or deployment issues.fixAlways initialize your project using `brickflow projects add` or `bf init` and follow the recommended project structure. Ensure `path_from_repo_root_to_project_root` and `path_project_root_to_workflows_dir` in `.brickflow-project-root.yml` are correctly set.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'your_module'
This error often occurs due to an incorrect project structure, missing `__init__.py` files, or issues with how `brickflows` resolves Python modules within the Databricks environment or local development setup.
fixEnsure your `brickflows` project is initialized correctly using `bf init` or `brickflow projects add`. Verify that all Python modules are correctly placed within the configured `path_project_root_to_workflows_dir` in your project configuration or are properly installed as part of your Databricks libraries.
Error: PERMISSION_DENIED: User does not have CAN MANAGE permissions on job.
This Databricks Asset Bundle (DAB) deployment error indicates that the user or service principal attempting to deploy the workflow lacks the necessary permissions on the target Databricks job, or other Unity Catalog permissions are missing.
fixGrant the required `CAN MANAGE` permissions to the deploying identity (user or service principal). Additionally, ensure all necessary Unity Catalog permissions (e.g., `USE CATALOG`, `USE SCHEMA`) are correctly assigned and verify that job ownership aligns with the deploying identity.
Error: variable "env" is not defined.
This error typically occurs when environment variables or other parameters expected by the Databricks Asset Bundle (which `brickflows` utilizes) are not explicitly defined within the `databricks.yml` file for the specific deployment target, or are not passed correctly during the deployment command.
fixExplicitly define all required variables within the `targets` section of your `databricks.yml` configuration file for each environment (e.g., `dev`, `prod`). Ensure that you are deploying with the correct target specified, for example, `databricks bundle deploy -t prod`.
Error: failed to read provider configuration schema for registry.terraform.io/databricks/databricks: failed to instantiate provider "registry.terraform.io/databricks/databricks" to obtain schema: Unrecognized remote plugin message
This deployment error suggests an issue with the underlying Terraform setup used by Databricks Asset Bundles (which `brickflows` leverages), possibly due to an incorrect Terraform folder structure, a corrupted Databricks Terraform provider, or a version mismatch of Terraform components.
fixInspect the Terraform folder within your `.databricks/bundle` directory to ensure it's correctly structured. Confirm that the Databricks CLI and Terraform are properly installed and configured. Clearing your build workspace or checking for any leftover Terraform processes that might be locking files can also resolve this issue.
Upgrade
Version history
2.0.0latest on PyPI · released Aug 7, 2026
Audit
Dependencies
pythonrequiredRequires Python versions 3.9 to 3.12 for compatibility and execution.
databricks-clirequiredThe Brickflows CLI relies on the Databricks CLI for configuration and interaction with Databricks workspaces.