Install & Compatibility
Where this runs
tested against v0.3.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.13
✕ build_error
✕ build_error
425MB installed
● package 425MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
dlt
✓ import dlt
✗ from dlt import table, read_json
While 'import dlt' is correct for existing DLT code, Databricks now recommends 'from pyspark import pipelines as dp' for new Lakeflow Spark Declarative Pipelines (SDP) development.
pipelines (as dp)
✓ from pyspark import pipelines as dp
This is the recommended import path for new Lakeflow Spark Declarative Pipelines (SDP) development, replacing the 'dlt' module.
This quickstart demonstrates a typical Delta Live Tables pipeline structure using Python decorators within a Databricks notebook. It outlines a medallion architecture (bronze, silver, gold) for data processing. Note that while the `databricks-dlt` library provides IDE support for such code, actual execution and data processing occur only when deployed and run as a DLT pipeline on a Databricks workspace.
# This code snippet is designed to run within a Databricks DLT Notebook environment.
# The 'databricks-dlt' library provides stubs for local development,
# but actual execution requires a Databricks workspace.
import dlt
from pyspark.sql.functions import *
# Define a streaming table (Bronze layer)
@dlt.table
def raw_data():
# In a real scenario, this would read from a source like Auto Loader
# e.g., spark.readStream.format('cloudFiles').option('cloudFiles.format', 'json').load('/databricks-datasets/retail-org/sales_orders/')
# For demonstration, we'll simulate a static DataFrame read as this is a stub example
return spark.read.format('json').load('/databricks-datasets/retail-org/sales_orders/')
# Define a cleansed table (Silver layer) with expectations
@dlt.table(comment='Cleansed sales orders with valid order numbers')
@dlt.expect_or_drop('valid_order_number', 'order_number IS NOT NULL')
def cleansed_data():
return dlt.read('raw_data').select(col('customer_id'), col('order_number'), col('order_date'))
# Define an aggregated table (Gold layer)
@dlt.table(name='daily_sales_summary')
def daily_sales():
return (
dlt.read('cleansed_data')
.groupBy('order_date')
.agg(count('order_number').alias('total_orders'))
)
Debug
Known issues
gotchaThe `databricks-dlt` PyPI library provides Python *stubs* for local development tools (IDE autocompletion, type checking) and *does not contain functional implementations*. DLT pipelines defined using this stub must be deployed and executed on a Databricks workspace; they cannot be run locally.fixUnderstand that `databricks-dlt` is a development-time aid. To run your pipelines, you need a Databricks workspace and the DLT runtime. Refer to Databricks documentation for deploying and running DLT pipelines.
affects: All versions of `databricks-dlt`.
deprecatedThe underlying product "Delta Live Tables (DLT)" has been rebranded to "Lakeflow Spark Declarative Pipelines (SDP)". While existing Python code using `import dlt` will continue to function, Databricks officially recommends migrating new development to use `from pyspark import pipelines as dp` and the corresponding `@dp` decorators and functions for future compatibility and to leverage new features.fixFor new DLT/SDP development, use `from pyspark import pipelines as dp` at the top of your Python pipeline files. Replace `@dlt.table` with `@dp.table`, `@dlt.streaming_table` with `@dp.create_streaming_table`, `@dlt.materialized_view` with `@dp.materialized_view`, etc.
affects: From Databricks Runtime 15.4+ (January 2026 onwards).
gotchaThe `dlt` (or `pyspark.pipelines`) module and its decorators are only available when your Python code is executed within the context of a Databricks DLT/SDP pipeline. Attempting to import or use these modules in a standalone Python script or a regular Databricks notebook (not configured as a DLT pipeline) will result in an `ImportError` or `NameError`.fixEnsure your Python code is part of a DLT/SDP pipeline definition within a Databricks workspace. Local development with the `databricks-dlt` stub is for syntax validation only, not execution.
affects: All versions related to Databricks DLT/SDP Python API.
gotchaManaging external Python dependencies within DLT/SDP pipelines directly with `%pip install` or init scripts on the cluster can be problematic due to potential conflicts and maintenance issues. This can lead to unexpected pipeline failures or inconsistent environments.fixFollow Databricks best practices for dependency management: package Python dependencies as a wheel or egg and specify them in the pipeline libraries settings, or use a `requirements.txt` file attached to the pipeline configuration. Avoid dynamic installation methods within the pipeline code.
affects: All versions.
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'dlt'
The `dlt` module is only available within a Delta Live Tables pipeline runtime on Databricks. The `databricks-dlt` PyPI package provides local development stubs for IDE assistance but does not contain a functional implementation of the `dlt` module for local execution or on standard Databricks clusters.
fixDLT pipelines must be created and run on a Databricks workspace as a DLT pipeline, not as a standard notebook on an interactive cluster or locally. For local development, `import dlt` will work with the stub library, but the code will not execute. To perform syntax checks locally, consider mocking the `dlt` object.
DLTImportException: Delta Live Tables module is not supported on Spark Connect clusters.
This error occurs when attempting to import the `dlt` module on a Databricks cluster that is not specifically a DLT pipeline runtime, such as a Spark Connect cluster or a standard interactive cluster. The `dlt` module has specific runtime requirements within the DLT framework.
fixEnsure that your DLT code is executed as part of a Delta Live Tables pipeline on a Databricks workspace. The `dlt` module is only fully functional in this dedicated environment.
AttributeError: module 'dlt' has no attribute 'table'
This error occurs when trying to use the `@dlt.table` decorator (or other `dlt` functions like `dlt.read`) in an environment where the full DLT runtime is not present, such as a local Python environment with only the `databricks-dlt` stub library installed, or a standard Databricks notebook that isn't part of a DLT pipeline. The local `databricks-dlt` package provides the interface for `dlt.table` for autocompletion but no functional implementation.
fixDLT decorators and functions are only executed and interpreted within a Delta Live Tables pipeline on a Databricks workspace. When developing locally, the `databricks-dlt` stub library provides type hints and autocompletion. To avoid `AttributeError` during local testing or syntax checking, you might need to mock the `dlt` object and its methods.
cannot run a cell when connected to pipeline
Delta Live Tables notebooks are designed to be executed as part of a managed pipeline, not interactively cell-by-cell like standard Databricks notebooks. The DLT runtime manages the execution flow, and individual cells (especially those not directly defining tables or views) are not intended for arbitrary interactive execution.
fixDLT pipelines should be executed via the DLT pipeline interface in the Databricks workspace. For interactive debugging and development of supporting Python code (non-DLT specific functions), consider placing them in separate modules that can be tested independently or using a mocked `dlt` environment.
Upgrade
Version history
0.3.0latest on PyPI · released Nov 27, 2024
Audit
Dependencies
No dependency data recorded yet.