Registry / data / dagster-duckdb-pandas

dagster-duckdb-pandas

JSON →
library0.29.10pypypi✓ verified 86d ago

Dagster library for storing Pandas DataFrames in DuckDB. Current version 0.29.3, requires Python >=3.10,<3.15. Part of the Dagster ecosystem, releases follow Dagster core versions (libraries version 0.x). Cadence: monthly releases alongside Dagster core.

pip install dagster-duckdb-pandas
INSTALL
IMPORT
SIG · DAGSTER-DUCKDB-PAN
D
dagster-duckdb-pandas
datapythonv0.29.10
Install
19.4s avg
Import
3524ms
Disk
333MB
Pass rate
5/ 10
Env Coverage5 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.29.10 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
build_error
glibc
py 3.103.95 runs
installs and imports cleanly · install 19.4s · import 3.524s · 319MB
333MB installed
● package 333MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

DuckDBPandasIOManager
from dagster_duckdb_pandas import DuckDBPandasIOManager
from dagster_duckdb import DuckDBIOManager
DuckDBIOManager from dagster-duckdb is base class; DuckDBPandasIOManager handles pandas DataFrames specifically

Initialize a DuckDB I/O manager that stores pandas DataFrames in a local DuckDB database.

from dagster_duckdb_pandas import DuckDBPandasIOManager from dagster import Definitions, asset @asset def my_table() -> str: import pandas as pd return pd.DataFrame({"col1": [1, 2], "col2": ["a", "b"]}) resources = {"io_manager": DuckDBPandasIOManager(database="my_db.duckdb")} defs = Definitions(assets=[my_table], resources=resources)
Debug
Known issues
breakingUpgrading from dagster-duckdb 0.x to 0.23+ changes I/O manager import paths. Use `DuckDBPandasIOManager` from `dagster_duckdb_pandas` instead of legacy imports from `dagster_duckdb`.
fix
Change import to `from dagster_duckdb_pandas import DuckDBPandasIOManager` and update resource configuration.
affects: >=0.23.0
gotchaDuckDBPandasIOManager expects a `database` argument (string path) instead of `conn` or `connection`. Using a pre-existing connection object will fail.
fix
Provide the database file path (e.g., `DuckDBPandasIOManager(database='path/to/db.duckdb')`).
affects: all
deprecatedThe `mode` parameter (e.g., `'overwrite'`, `'append'`) has been deprecated in favor of per-asset metadata or explicit overwrite strategies.
fix
Use asset metadata or custom I/O manager configurations instead of the `mode` parameter.
affects: >=0.28.0
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'dagster_duckdb_pandas'
dagster-duckdb-pandas not installed or installed in wrong environment.
fix
Run `pip install dagster-duckdb-pandas`.
The `database` argument refer to a different database connection parameter. The `database` parameter expects a DuckDB database file path or `:memory:`.
Misunderstanding of the `database` parameter; users may try to pass a connection string or existing connection.
fix
Instantiate with `DuckDBPandasIOManager(database='my_database.duckdb')` or `database=':memory:'` for in-memory.
Upgrade
Version history
0.29.10latest on PyPI · released Jun 18, 2026
Audit
Dependencies
dagster-duckdbrequiredRequired for DuckDB I/O manager functionality
dagsterrequiredCore Dagster framework
pandasrequiredRequired for DataFrame operations
Agent activity
30 hits · last 30 days
node
26
OpenAI (training)
2
Resources
dagster-duckdb-pandas — pip install dagster-duckdb-pandas · libregistry