pytest-benchmark

library5.3.0pypypi✓ verified 27d ago

pytest-benchmark is a plugin for pytest that provides a `benchmark` fixture for benchmarking code. It automatically groups tests into rounds calibrated to the chosen timer, offering sensible defaults and automatic calibration for micro-benchmarks. As of version 5.2.3, it is actively maintained with regular major releases approximately once a year and several minor/patch releases.

testing

pip install pytest-benchmark

github ↗package ↗docs ↗changelog ↗

INSTALL

IMPORT

SIG · PYTEST-BENCHMARK

pytest-benchmark

testingpythonv5.3.0

Install

3.1s avg

Import

—

Disk

35MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v5.3.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.910 runs

installs and imports cleanly · install 0.0s · import 0.000s · 33.4MB

glibc

py 3.10–3.910 runs

installs and imports cleanly · install 3.1s · import 0.000s · 34MB

35MB installed

● package 35MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

benchmark

✓ def test_my_function(benchmark): # benchmark is a pytest fixture, no explicit import needed

The 'benchmark' fixture is automatically provided by pytest-benchmark to test functions; no direct import statement for 'pytest_benchmark' is typically required within test modules.

Define a test function that accepts the `benchmark` fixture. Pass the function you wish to benchmark as the first argument to `benchmark()`. You can also pass positional and keyword arguments to the benchmarked function. Running `pytest` will execute the benchmarks and display a summary table.

import time def something(duration=0.000001): """ Function that needs some serious benchmarking. """ time.sleep(duration) return 123 def test_my_stuff(benchmark): # benchmark something result = benchmark(something) assert result == 123 # To run: save the above as a Python file (e.g., test_benchmarks.py) and run `pytest` from your terminal. # Use `pytest --benchmark-autosave --benchmark-json=report.json` for advanced features.

Debug

Known issues

breakingDropped support for Python 3.8 in version 5.0.0. Projects using Python 3.8 or older must remain on an earlier version of pytest-benchmark.

fix

Upgrade to Python 3.9+ or pin pytest-benchmark to <5.0.0.

affects: >=5.0.0

breakingVersion 5.1.0 and later require pytest version 8.1 or newer due to internal hook handling changes. Older pytest versions will lead to errors.

fix

Upgrade pytest to version 8.1 or newer, or pin pytest-benchmark to <5.1.0.

affects: >=5.1.0

gotchaBenchmarking internal functions or wrapping code in a `@benchmark` decorator within the test function (e.g., `benchmark(lambda: my_func())`) can introduce unnecessary function call overhead, leading to less accurate micro-benchmarks. Pass the function reference directly instead.

fix

Prefer `benchmark(my_function, *args, **kwargs)` over `benchmark(lambda: my_function(*args, **kwargs))` for better accuracy, especially with very fast functions.

affects: <all>

gotchaHigh standard deviation in benchmark results (flaky benchmarks) often indicates external factors (e.g., other running services, VMs, CPU turbo boost) or non-deterministic test functions (I/O, external resources, side-effects). PyPy's GC and JIT can also add overhead, requiring sufficient warmup.

fix

Ensure a stable, isolated environment. Minimize I/O and external dependencies in benchmarked code. Use `--benchmark-warmup` and `--benchmark-warmup-iterations` for JIT-heavy interpreters like PyPy. Consider `median` or `IQR` over `mean`/`stddev` for outlier-prone results.

affects: <all>

gotchaIn 'pedantic' mode, if a `setup` function is provided to `benchmark.pedantic()`, you cannot simultaneously use the `args`, `kwargs`, or `iterations` parameters. The setup function is expected to return the arguments for the target function if needed.

fix

When using a `setup` function with `benchmark.pedantic`, have the `setup` function return the arguments tuple/dict, and do not pass `args`, `kwargs`, or `iterations` separately to `pedantic`.

affects: <all>

gotchaUsing the `benchmark` fixture more than once within a single test function is disallowed and will raise an exception (since v3.0.0). Each benchmarkable unit should be in its own test function.

fix

Create separate test functions for each distinct piece of code you want to benchmark. Use `pytest.mark.parametrize` for benchmarking the same function with different inputs.

affects: >=3.0.0

Upgrade

Version history

5.3.0latest on PyPI · released Aug 23, 2026

Audit

Dependencies

pytest>=8.1requiredCore testing framework dependency.

py-cpuinforequiredUsed for CPU information in benchmark reports.

pygaloptionalOptional, for generating histogram plots.

pygaljsoptionalOptional, for generating histogram plots.

elasticsearchoptionalOptional, for storing benchmark results in Elasticsearch.

aspectliboptionalOptional, for aspect mode functionality.