Registry / workflow / jobflow

jobflow

JSON →
library0.3.1pypypi✓ verified 84d ago

jobflow is a free, open-source Python library (v0.3.1) for writing and executing computational workflows. It enables defining complex workflows using simple Python functions and executing them locally or on remote resources via managers like `jobflow-remote` or `FireWorks`. Key features include dynamic workflows, easy compositing and nesting of workflows, and the ability to store workflow outputs across various databases (MongoDB, S3, GridFS, etc.) through the `Maggma` package. The library is actively maintained with regular updates.

pip install jobflow
INSTALL
IMPORT
SIG · JOBFLOW
J
jobflow
workflowpythonv0.3.1
Install
20.3s avg
Import
2528ms
Disk
369MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.3.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.960 runs
installs and imports cleanly · install 0.0s · import 2.605s · 359.8MB
glibc
py 3.103.960 runs
installs and imports cleanly · install 20.3s · import 2.450s · 347MB
369MB installed
● package 369MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

job
from jobflow import job
Flow
from jobflow import Flow
run_locally
from jobflow.managers.local import run_locally
SETTINGS
from jobflow import SETTINGS
from jobflow.settings import SETTINGS_CLASS_NAME
Configuration settings are usually accessed via `jobflow.SETTINGS` directly after `v0.1.3`.

This quickstart defines two simple jobs using the `@job` decorator. These jobs are then chained together using `Flow` objects, where the output of one job serves as the input for the next. The `run_locally` function executes the entire flow in your local environment. For persistent storage and more advanced execution, a `JobStore` (e.g., MongoDB, S3) should be configured, typically via a `~/.jobflow.yaml` file.

from jobflow import job, Flow from jobflow.managers.local import run_locally @job def add(a, b): return a + b @job def multiply(a, b): return a * b # Create Job objects job1 = add(1, 2) job2 = multiply(job1.output, 3) job3 = add(job2.output, 10) # Create a Flow from the jobs flow = Flow([job1, job2, job3], name="my_first_flow") # Run the Flow locally # For persistent storage, configure ~/.jobflow.yaml with a JobStore # e.g., JOB_STORE: { _fw_name: "JSONStore", path: "./jobstore" } responses = run_locally(flow) # Access results final_result = responses[job3.uuid].output print(f"Final result: {final_result}")
Debug
Known issues
breakingAs of `v0.1.3`, jobflow migrated its settings handling to `Pydantic`. The default `JOB_STORE` and other configurations are now managed through a `~/.jobflow.yaml` file. Existing setups using older configuration methods may break.
fix
Create or update your `~/.jobflow.yaml` file according to the latest documentation, particularly for `JOB_STORE` configuration (e.g., `JOB_STORE: { _fw_name: "JSONStore", path: "./jobstore" }`).
affects: >=0.1.3
gotchaJob outputs accessed via `.output` are `OutputReference` objects, not the direct computed values. These references are used by jobflow to build the workflow graph and automatically resolve dependencies during execution. Attempting to use `.output` as an immediate value before the job has run will result in errors.
fix
Only access the actual output value from the `JobResponse` object returned after a flow has been successfully executed (e.g., `responses[job.uuid].output`).
affects: All versions
gotchaThe `draw_graph()` method for visualizing flows requires the `graphviz` package and the `jobflow[vis]` extra to be installed. Without it, attempting to draw a graph will likely result in an `AttributeError` or `ModuleNotFoundError`.
fix
Install the visualization dependencies: `pip install jobflow[vis]`.
affects: All versions
breakingWhile `jobflow` itself supports Python >=3.10, its companion package `jobflow-remote` (used for remote execution) dropped support for Python 3.9 as of its `v1.0` release. There were also warnings regarding Python 3.14.1 due to `networkx` issues.
fix
Ensure you are using Python >=3.10 and avoid Python 3.14.1 if using `jobflow-remote`. Always check the specific Python compatibility for all `jobflow` ecosystem packages.
affects: jobflow-remote >=1.0, Python 3.9, Python 3.14.1
gotchaMisconfiguration of the `JobStore` (the database where job inputs/outputs are stored) can lead to jobs not being persisted, not being found by managers, or data loss. This is especially critical for remote execution or long-running workflows.
fix
Carefully configure your `JOB_STORE` in `~/.jobflow.yaml`. For production, use robust backends like MongoDB or S3 via `Maggma` rather than the default `JSONStore`.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'jobflow.managers.fireworks'
Attempting to use the FireWorks manager without installing the optional 'fireworks' dependencies.
fix
Install jobflow with the FireWorks extra: `pip install jobflow[fireworks]`.
TypeError: 'OutputReference' object is not callable
Incorrectly attempting to call or directly access the value of a `job.output` object before the job has been executed and its output resolved.
fix
Remember that `job.output` returns an `OutputReference` for dependency tracking. Access the actual result from the `JobResponse` after running the flow, e.g., `responses[my_job.uuid].output`.
jobflow.settings.JobflowSettingsValidationError: The job store must be configured.
The `JOB_STORE` setting is not properly defined in the `~/.jobflow.yaml` configuration file, which is required for any persistent or remote workflow execution.
fix
Create or update `~/.jobflow.yaml` in your home directory with a valid `JOB_STORE` configuration, for example:
```yaml
JOB_STORE:
  _fw_name: "JSONStore"
  path: "./my_job_data"
```
or a `MongoStore` configuration if `Maggma` is installed.
AttributeError: 'Flow' object has no attribute 'draw_graph'
Attempting to call `flow.draw_graph()` without having installed the optional visualization dependencies.
fix
Install the visualization dependencies: `pip install jobflow[vis]`.
Upgrade
Version history
0.3.1latest on PyPI · released Feb 5, 2026
Audit
Dependencies
pydanticrequiredUsed for settings management and data validation.
maggmaoptionalEnables integration with various databases (MongoDB, S3, GridFS) for storing job outputs, highly recommended for persistent storage.
fireworksoptionalOptional workflow manager for executing jobflow workflows on remote clusters.
jobflow-remoteoptionalOptional package for running jobflow workflows on remote resources.
Agent activity
13 hits · last 30 days
node
12
OpenAI (training)
1
Resources
jobflow — pip install jobflow · libregistry