Dagster is an orchestration platform designed for the development, production, and observation of data assets. It emphasizes defining pipelines as data assets and provides a rich development experience with built-in UI and testing capabilities. The library is actively maintained with frequent minor releases, often on a weekly basis, incorporating new features, bug fixes, and improvements.
pip install dagster dagster-webserverVerified import paths — ran on the pinned version, not inferred.
This quickstart defines a simple asset and a job that materializes it. It includes a directly runnable Python snippet to execute the asset in memory. For local development with the full Dagster UI (Dagit), save the commented `Definitions` block to a file and run `dagster dev`.
Replace `@solid` with `@op` and `PipelineDefinition` with `JobDefinition` or `@job` decorator. Refer to the Dagster 1.0 migration guide for full details.
Ensure both `dagster` and `dagster-webserver` are installed: `pip install dagster dagster-webserver`.
Explicitly add `psycopg2-binary` (or `psycopg2`) to your project's dependencies if you use `dagster-postgres`: `pip install psycopg2-binary`.
For open-source Dagster, use `dagster dev`, `dagster job launch`, etc. For Dagster Cloud, install `dagster-cloud` and use `dg deploy`, `dg api`, etc.
Define configuration using `class MyConfig(Config): ...` and type-hint your resource/op functions with it: `def my_resource(config: MyConfig): ...`.