dbt-duckdb

library1.10.1pypypiunverified

dbt-duckdb is the adapter that connects dbt (data build tool) with DuckDB, an embedded analytical database. It allows users to manage and transform data using SQL or Python models, enabling a powerful local data stack or data lakehouse architecture. The library is actively maintained with frequent releases, currently at version 1.10.1.

database data workflow

pip install dbt-core dbt-duckdb

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · DBT-DUCKDB

dbt-duckdb

databasepythonv1.10.1

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

This quickstart guides you through initializing a dbt project, configuring the `dbt-duckdb` adapter in your `profiles.yml` for local storage, and running a simple dbt model. The `path` field is crucial for data persistence.

mkdir my_dbt_project cd my_dbt_project dbt init my_dbt_project_name # Edit ~/.dbt/profiles.yml (or profiles.yml in project root) # Add the following profile: # # my_dbt_project_name: # target: dev # outputs: # dev: # type: duckdb # path: /tmp/dbt.duckdb # Or './dbt_warehouse.duckdb' for project-local persistence # threads: 4 # dbt debug --profile my_dbt_project_name # Expected output: Connection test successful. # Create a simple SQL model (e.g., models/my_model.sql) # -- my_model.sql -- # SELECT 1 as id, 'hello' as message # Run your dbt models dbt run

dbt --version

Debug

Known issues

breakingPython 3.8 and 3.9 are no longer supported. Version 1.10.1 dropped support for both, and 1.9.2 dropped Python 3.8. Users must upgrade to Python 3.10 or newer.

fix

Upgrade your Python environment to version 3.10 or later.

affects: >=1.10.1 (Python 3.8, 3.9); >=1.9.2 (Python 3.8)

breakingExplicit dbt-core Installation Required: As of dbt-duckdb 1.8, dbt-core is no longer automatically installed as a dependency. Users must explicitly install `dbt-core` alongside `dbt-duckdb` (e.g., `pip install dbt-core dbt-duckdb`). Ensure `dbt-core` version is compatible; current dbt-duckdb versions target `dbt-core >= 1.8.x`.

fix

Modify your installation command to `pip install dbt-core dbt-duckdb` and ensure your `dbt-core` version meets the adapter's requirements.

affects: >=1.8.0

gotchaDefault In-Memory Database (No Persistence): By default, `dbt-duckdb` runs against an in-memory DuckDB database, meaning any data generated by `dbt run` will not be persisted after the process exits. To store your data to disk, you *must* specify a `path` in your `profiles.yml` configuration (e.g., `path: /path/to/my/database.duckdb`).

fix

Always set the `path` configuration in your `profiles.yml` to a file path on your filesystem to enable data persistence. Use `:memory:` explicitly if non-persistence is desired.

affects: All versions

gotchalogbook Dependency Pin: Version 1.10.1 introduced a dependency pin `logbook < 1.9`. This is due to `pytest-logbook`'s reliance on a compat module removed in `logbook` 1.9.x. This might cause dependency conflicts if other libraries in your environment require a newer version of `logbook`.

fix

If encountering conflicts, consider isolating dbt-duckdb in a dedicated virtual environment or checking if other dependencies can tolerate `logbook < 1.9`. Report issues if this impacts critical functionality.

affects: >=1.10.1

deprecatedDeltaLake Functions Removed: Legacy DeltaLake functions were removed in `dbt-duckdb` 1.8.4. Models or workflows relying on these older functions will break.

fix

Update your dbt models and configurations to use the current, supported methods for interacting with DeltaLake if applicable. Refer to the dbt-duckdb documentation for updated patterns.

affects: >=1.8.4

gotchaMotherDuck Limitations: When connecting to MotherDuck (cloud-hosted DuckDB), certain features are limited compared to a local DuckDB instance. This includes lack of support for custom extensions/user-defined functions and potential differences in behavior for some advanced SQL features like `dbt.listagg` macros and foreign-key constraints.

fix

Be aware of these limitations when designing solutions for MotherDuck. Test thoroughly if using advanced SQL features or custom extensions.

affects: All versions when using MotherDuck

Errors

Common errors & fixes

Runtime Error Could not find adapter type duckdb!

This error typically occurs when the 'dbt-duckdb' adapter is not correctly installed in the Python environment where dbt-core is running, or there is an incompatibility between the installed versions of dbt-core and dbt-duckdb.

fix

Ensure both 'dbt-core' and 'dbt-duckdb' are installed together in a dedicated Python virtual environment using compatible versions. The recommended installation command is `python -m pip install dbt-core dbt-duckdb`.

No module named 'pandas'

This error arises in dbt Python models when a required Python package, such as 'pandas', is imported but is not available in the specific Python environment that dbt-duckdb uses to execute the model.

fix

Create a virtual environment named 'dbtenv' within your dbt project directory (or ensure it's in your environment path), activate it, and install all necessary Python packages (e.g., `pip install pandas`). dbt automatically detects and uses this `dbtenv` environment for Python model execution.

IO Error: Could not set lock on file "your_database.duckdb": Resource temporarily unavailable

This issue occurs when another process (like a concurrent dbt run, an IDE like DBeaver, or a separate Python script) has an exclusive lock on the DuckDB database file, preventing dbt from accessing or writing to it.

fix

Ensure that no other applications or dbt processes are currently accessing the specified `.duckdb` file. Disconnect any database clients or terminate other dbt runs that might be holding a lock on the database file.

Runtime Error: Catalog Error: Table with name your_model_name__dbt_tmp does not exist!

This can happen in incremental Python models if temporary tables created during a run are not correctly managed or referenced, or with external tables (e.g., on S3) if the configuration for accessing the data source is incorrect (e.g., wrong credentials, region, or path) leading to the table not being found.

fix

For Python models, verify the incremental logic, ensuring `dbt.is_incremental` is handled properly and that the model returns a `pandas.DataFrame` or `polars.DataFrame`. For external tables, double-check your `profiles.yml` for correct S3 configuration (region, access keys, required extensions like `httpfs`, `parquet`, `aws`) and confirm the `external_location` path is valid and accessible with the provided credentials.

Upgrade

Version history

1.10.1latest on PyPI · released Feb 17, 2026

Audit

Dependencies

dbt-corerequiredThis is a dbt adapter; it requires dbt-core. Current versions target dbt-core >= 1.8.x. As of dbt-duckdb 1.8, dbt-core must be installed explicitly.

duckdbrequiredThe underlying analytical database engine. Required for functionality. Current versions target duckdb >= 1.1.x.

pyarrowrequiredA dependency for efficient data handling, often bumped in releases. [cite: GitHub release notes]

logbookrequiredPinned below 1.9.x in 1.10.1 due to compatibility with pytest-logbook. [cite: GitHub release notes]

Agent activity

23 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/jwills/dbt-duckdb ↗

packagehttps://pypi.org/project/dbt-duckdb/ ↗

homepagehttps://docs.getdbt.com ↗