Registry / testing / nbval
library0.11.0pypypiunverified

nbval is a pytest plugin designed to validate Jupyter notebooks. It allows users to execute notebooks as part of their test suite and compare cell outputs against previously saved outputs, ensuring reproducibility and correctness. The current version is 0.11.0, and it has a sporadic but active release cadence, typically with minor releases every 6-12 months.

pip install nbval pytest
INSTALL
IMPORT
SIG · NBVAL
N
nbval
testingpythonv0.11.0
Install
Import
Disk
Pass rate
0/ 10
Env Coverage0 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v? · pip install
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
timeout
glibc
py 3.103.95 runs
timeout
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

nbval functionality
nbval is primarily used via the pytest CLI (e.g., `pytest --nbval`) and typically does not require direct Python imports for its core validation features. pytest automatically discovers and loads the plugin upon installation.
As a pytest plugin, direct imports from `nbval` are generally not needed for basic usage. Configuration is primarily done via pytest command-line options or `pytest.ini`.

This quickstart script demonstrates how to programmatically create a simple Jupyter notebook and then run `pytest` with the `nbval` plugin to validate its output. It shows the minimal setup required to get started with notebook testing using `nbval`.

import pytest import json import os # 1. Create a dummy Jupyter notebook file for testing notebook_content = { "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ {"name": "stdout", "output_type": "stream", "text": ["Hello from nbval!\n"]} ], "source": ["print('Hello from nbval!')"] } ], "metadata": { "kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7"} }, "nbformat": 4, "nbformat_minor": 5 } notebook_filename = "example.ipynb" with open(notebook_filename, "w") as f: json.dump(notebook_content, f, indent=4) print(f"Created {notebook_filename}. Running nbval tests...") # 2. Run pytest programmatically with nbval # The --nbval flag enables the plugin, and the notebook filename specifies the target. exit_code = pytest.main(["--nbval", notebook_filename]) # 3. Report results and clean up # os.remove(notebook_filename) # Uncomment to clean up the notebook file after run if exit_code == 0: print(f"\nnbval tests passed for {notebook_filename}!") else: print(f"\nnbval tests failed for {notebook_filename} with exit code {exit_code}.")
pytest --version
Debug
Known issues
breakingThe `--nbval-sanitize` command-line option was removed in version 0.10.0. This option was previously used to sanitize notebook outputs before comparison but was deprecated in earlier versions.
fix
Remove `--nbval-sanitize` from your pytest command-line arguments or `pytest.ini` configuration. If sanitation is still required, consider pre-processing notebooks or using `nbval`'s cell metadata options for more granular control over output comparison (e.g., `nbval-ignore_data_priority`).
affects: >=0.10.0
breakingSupport for Python 3.6 was dropped in version 0.9.0. Users on Python 3.6 will not be able to install or run `nbval` versions 0.9.0 or newer.
fix
Upgrade your Python environment to Python 3.7 or newer. The current minimum requirement is Python 3.7.
affects: >=0.9.0
gotchaBy default, `nbval` performs a strict diff on notebook outputs. This can lead to very verbose output in case of failures, especially with slight differences in timestamps, floating-point precision, or minor formatting changes. This behavior is controlled by the `nbval_diff` option.
fix
To get a simpler pass/fail indication without detailed diffs, you can set `nbval_diff=False` in your `pytest.ini` or use `--no-nbval-diff` on the command line. For more controlled output validation, explore options like `--nbval-lax` or specific cell metadata for ignoring certain outputs.
affects: All
gotchanbval's primary function is *output validation*, not just execution and failure on any runtime error. It compares the newly generated outputs against the stored outputs in the notebook. If the notebook is expected to raise an exception or has no outputs for a cell, this needs to be explicitly indicated using cell metadata (e.g., `raises-exception`, `skip-execution`).
fix
Ensure you understand how `nbval` handles outputs. For cells that are expected to fail, add `"raises-exception": true` to their cell metadata. For cells with no relevant output or to skip execution, use `"skip-execution": true` or `"skip-compare": true`. Use `--nbval-lax` for more permissive output comparison, or `--nbval-errors` to fail on any executed code error regardless of output comparison.
affects: All
Upgrade
Version history
0.11.0latest on PyPI · released Mar 4, 2024
Audit
Dependencies
pytestrequirednbval is a plugin for pytest and requires it to function.
Agent activity
61 hits · last 30 days
node
58
Resources
nbval — pip install nbval · libregistry