Registry / workflow / apache-airflow-providers-papermill

apache-airflow-providers-papermill

JSON →
library3.13.2pypypi✓ verified 22d ago

The Apache Airflow Papermill Provider integrates Papermill with Apache Airflow, enabling users to parameterize and execute Jupyter Notebooks as part of their Airflow DAGs. This allows for automated, reproducible, and scalable execution of notebooks within data pipelines. The current version is 3.12.3, and it follows the release cadence of Apache Airflow providers, with updates typically aligned with Airflow releases or independent fixes and features.

pip install apache-airflow-providers-papermill
INSTALL
IMPORT
SIG · APACHE-AIRFLOW-PRO
A
apache-airflow-providers-papermill
workflowpythonv3.13.2
Install
24.3s avg
Import
9136ms
Disk
742MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v3.13.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.1s · import 9.854s · 765.4MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 48.6s · import 8.418s · 714MB
742MB installed
● package 742MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

PapermillOperator
from airflow.providers.papermill.operators.papermill import PapermillOperator
This is the primary operator for executing Jupyter notebooks.

This quickstart demonstrates how to define a simple DAG that uses the `PapermillOperator` to execute a Jupyter notebook. The notebook should have a cell tagged as 'parameters' to receive inputs. Remember to replace `/tmp/hello_world.ipynb` with the actual path to your notebook.

from __future__ import annotations import pendulum from airflow.models.dag import DAG from airflow.providers.papermill.operators.papermill import PapermillOperator # For a real-world scenario, ensure 'hello_world.ipynb' exists in your DAGs folder # or a location accessible by Airflow, with a 'parameters' tagged cell. # Example 'hello_world.ipynb': # # In a cell, add tag 'parameters' # msg = "Default message" # print(f"Hello, {msg}!") with DAG( dag_id="example_papermill_notebook", start_date=pendulum.datetime(2023, 1, 1, tz="UTC"), schedule=None, catchup=False, tags=["papermill", "example"], ) as dag: run_notebook = PapermillOperator( task_id="run_hello_world_notebook", input_nb="/tmp/hello_world.ipynb", # Replace with actual path or Airflow-accessible path output_nb="/tmp/out-{{ ds }}.ipynb", parameters={ "msgs": "Ran from Airflow at {{ ds }}!" }, )
airflow --version
Debug
Known issues
breakingProvider version 3.0.0 and above requires Apache Airflow 2.2+. Earlier versions of the provider (2.0.0) required Airflow 2.1.0+. Ensure your Airflow installation meets the minimum version requirement for the provider you are installing.
fix
Upgrade your Apache Airflow instance to at least version 2.2.0. For provider versions 2.0.0, upgrade to Airflow 2.1.0+.
affects: >=3.0.0
breakingPython 3.7 support was dropped in provider versions 3.2.1 and above. Ensure you are using a supported Python version.
fix
Upgrade your Python environment to 3.8 or newer. The latest provider versions support Python >=3.10.
affects: >=3.2.1
gotchaThe `PapermillOperator` executes notebooks locally within the Airflow worker's environment. You must ensure that the notebook's kernel (e.g., `ipykernel`) and any other dependencies required by your notebook code are installed in the Airflow worker's environment.
fix
Include `ipykernel` and any other specific Python packages needed by your notebooks in your Airflow environment's `requirements.txt` or equivalent.
affects: All
gotchaJupyter notebooks intended for use with `PapermillOperator` must have a cell explicitly tagged as 'parameters' if you intend to pass parameters from Airflow. If this tag is missing, parameters will be injected at the top of the notebook, which might not be the desired behavior.
fix
Add a cell in your Jupyter notebook and tag it as 'parameters' to define default values and control where injected parameters appear.
affects: All
gotchaA known bug with some `papermill` versions can cause 'No such file or directory' errors when writing grammar tables. This typically manifests as `Writing failed: [Errno 2] No such file or directory: '/home/astro/.cache/black/21.7b0/tmpzpsclowd'`.
fix
As a workaround, manually create the missing directory in your Airflow environment. For Astro projects, add `RUN mkdir -p /home/astro/.cache/black/21.7b0/` to your project's Dockerfile.
affects: Specific `papermill` versions (check GitHub issues for exact range)
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'airflow.providers.papermill'
The 'apache-airflow-providers-papermill' package is not installed.
fix
Install the package using pip: 'pip install apache-airflow-providers-papermill'.
ImportError: cannot import name 'PapermillOperator' from 'airflow.providers.papermill.operators.papermill'
The 'PapermillOperator' class is not available in the specified module path.
fix
Ensure you are importing 'PapermillOperator' correctly: 'from airflow.providers.papermill.operators.papermill import PapermillOperator'.
AttributeError: module 'airflow.providers.papermill' has no attribute 'PapermillOperator'
Attempting to access 'PapermillOperator' directly from the 'airflow.providers.papermill' module.
fix
Import 'PapermillOperator' from the correct submodule: 'from airflow.providers.papermill.operators.papermill import PapermillOperator'.
ModuleNotFoundError: No module named 'papermill'
The 'papermill' package, a dependency of 'apache-airflow-providers-papermill', is not installed.
fix
Install the 'papermill' package using pip: 'pip install papermill'.
ImportError: cannot import name 'PapermillOperator' from 'airflow.operators'
The 'PapermillOperator' is not located in the 'airflow.operators' module.
fix
Import 'PapermillOperator' from its correct location: 'from airflow.providers.papermill.operators.papermill import PapermillOperator'.
Upgrade
Version history
3.13.2latest on PyPI · released Aug 8, 2026
Audit
Dependencies
apache-airflowrequiredCore Airflow functionality
papermill[all]requiredRequired for Jupyter notebook parameterization and execution
scrapbook[all]requiredUsed for reading notebook outputs and scraps
ipykerneloptionalRequired to run the notebook kernel in the Airflow environment
Agent activity
51 hits · last 30 days
node
44
OpenAI (training)
2
Resources
apache-airflow-providers-papermill — pip install apache-airflow-providers-papermill · libregistry