nbmake

library1.5.5pypypi✓ verified 21d ago

nbmake is a pytest plugin designed for testing Jupyter notebooks. It executes notebooks programmatically, allowing for continuous integration and automated quality checks on documentation and research materials. Currently at version 1.5.5, the library maintains an active development cadence, frequently releasing updates to support new Python and Pytest versions and address user feedback.

testing

pip install nbmake pytest

github ↗package ↗

INSTALL

IMPORT

SIG · NBMAKE

nbmake

testingpythonv1.5.5

Install

8.9s avg

Import

—

Disk

121MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.5.5 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.000s · 135.2MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 8.9s · import 0.000s · 118MB

121MB installed

● package 121MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

nbmake

✓ nbmake is primarily used as a pytest plugin via the command line; direct Python imports of 'nbmake' are generally not required for basic usage.

Users typically interact with nbmake through the 'pytest' command with the '--nbmake' flag, rather than importing symbols into Python test files.

To quickly test a Jupyter notebook, install `nbmake` and `pytest`, then invoke pytest with the `--nbmake` flag on your notebook file(s). nbmake will execute the notebook from top to bottom, failing if any cell raises an unhandled exception or an assertion fails. Wildcards can be used to test multiple notebooks.

import os # Create a dummy notebook for testing notebook_content = """ { "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "x = 1\n", "y = 2\n", "assert x + y == 3" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print('Notebook executed successfully')" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 } """ with open("test_notebook.ipynb", "w") as f: f.write(notebook_content) # To run this, save it as a .py file and execute from your terminal: # python -c "$(cat your_script.py)" # (or simply run the file) # Then, in the same directory: # pytest --nbmake test_notebook.ipynb print("Created test_notebook.ipynb. Run 'pytest --nbmake test_notebook.ipynb' in your terminal.")

pytest --version

Debug

Known issues

breakingnbmake v1.5.0 dropped support for Python 3.7. Users on older Python versions will need to upgrade their Python environment or use an older nbmake version.

fix

Upgrade to Python 3.8+ (preferably 3.9+) or pin nbmake to a version <1.5.0.

affects: >=1.5.0

breakingnbmake v1.5.5 dropped support for Python 3.8, which reached end-of-life in October 2024. Users running Python 3.8 should upgrade their Python environment.

fix

Upgrade to Python 3.9 or newer. Python 3.12 is officially supported by nbmake 1.5.5.

affects: >=1.5.5

gotchaNotebooks can fail during testing due to missing Python package dependencies or an incorrect Jupyter kernel in the execution environment, particularly in CI setups. This results in `ModuleNotFoundError` or 'No such kernel' errors.

fix

Ensure all necessary Python packages are installed in the test environment (e.g., via `pip install -e .` for local packages or `pip install -r requirements.txt`). For kernel issues, install the required Jupyter kernel (`python -m ipykernel install --user --name mycustomkernel`) or use `--nbmake-kernel=mycustomkernel` to override. Use `--nbmake-find-import-errors` to specifically diagnose import issues.

affects: All

gotchaLong-running cells in notebooks can cause tests to time out. Default timeouts might be too strict for some computational tasks.

fix

Increase the per-cell timeout using `pytest --nbmake --nbmake-timeout=<seconds>` (e.g., `--nbmake-timeout=3000` for 50 minutes). Alternatively, mock out variables or skip complex cells using notebook metadata.

affects: All

gotchaSome notebook cells or entire notebooks are designed to raise exceptions. By default, `nbmake` will treat these as failures.

fix

For a cell expected to raise an exception, add `"tags": ["raises-exception"]` to its metadata. For an entire notebook, add `"execution": {"allow_errors": true}` to its top-level metadata. To skip a cell entirely, use `"tags": ["skip-execution"]`.

affects: All

gotchaRelative imports within notebooks (e.g., `from .. import my_module`) often fail when run by `pytest --nbmake` if the project structure isn't correctly handled by pytest or the package isn't installed in 'editable' mode.

fix

Install your project in editable mode (`pip install -e .`) so Python can find modules. Alternatively, ensure the notebook's directory is on the Python path during testing, or modify `sys.path` within the notebook itself, though this is generally less robust.

affects: All

Upgrade

Version history

1.5.5latest on PyPI · released Dec 23, 2024

Audit

Dependencies

pytestrequirednbmake is a plugin for the pytest testing framework.

nbclientrequiredUsed internally by nbmake for notebook execution.

ipykerneloptionalRequired for running notebooks with specific, non-default Jupyter kernels, especially in CI environments.

Agent activity

11 hits · last 30 days

node

Resources

githubhttps://github.com/treebeardtech/nbmake ↗

packagehttps://pypi.org/project/nbmake/ ↗