Install & Compatibility
Where this runs
tested against v0.2.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.252s · 18.5MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 1.9s · import 0.223s · 19MB
16MB installed
● package 16MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
Benchmark
✓ from asv_runner.benchmarks._base import Benchmark
While direct import is possible, most users interact with `asv` via the CLI. This import is for advanced customization or extending ASV's core benchmark types.
time_range
✓ from my_project.benchmarks import time_range
Benchmarks are typically defined in user-created files (e.g., `benchmarks.py`) within a project and then discovered and run by the `asv` command-line tool. Direct import of user benchmarks is part of regular Python module loading.
SkipNotImplemented
✓ from asv_runner.benchmark.mark import SkipNotImplemented
Used to programmatically skip benchmarks within their execution context, preferred over raising `NotImplementedError`.
ASV Runner provides the core logic for executing benchmarks. In a typical setup, users define benchmark functions or classes within Python files (e.g., `benchmarks.py`) which are then discovered and run by the `asv` command-line tool. This example demonstrates a simple benchmark suite that `asv` (using `asv-runner` internally) would execute.
# my_project/benchmarks.py
class TimeSuite:
def setup(self):
self.data = list(range(1_000_000))
def time_iterate_list(self):
for _ in self.data:
pass
def time_sum_list(self):
sum(self.data)
# To run this, you would typically use the `asv` command-line tool:
# 1. Initialize an ASV project: `asv quickstart` (creates asv.conf.json, etc.)
# 2. Place this file in your benchmark directory (e.g., 'benchmarks/')
# 3. Run benchmarks: `asv run`
Debug
Known issues
breakingWith the release of `asv` version 0.6.0, the project's functionality was split into the main `asv` package and `asv_runner`. `asv_runner` now strictly handles the core benchmark execution. Users or developers who previously relied on internal `asv` mechanisms for benchmark running prior to 0.6.0 may find their code no longer compatible if they attempt to interact directly with these components.fixMigrate any direct internal calls to `asv`'s benchmark execution logic to use the `asv_runner` API where appropriate, or preferably, use the `asv` command-line tool for orchestration. Review the `asv` and `asv_runner` documentation for updated interaction patterns.
affects: asv>=0.6.0
deprecatedRaising `NotImplementedError` directly from a benchmark function to skip it is deprecated. While it may still work in some contexts for backward compatibility, the recommended approach is to raise `asv_runner.benchmark.mark.SkipNotImplemented` or use the `skip_benchmark_if`/`skip_params_if` decorators.fixReplace `raise NotImplementedError` with `raise SkipNotImplemented()` (after importing it) or apply the `@skip_benchmark_if` or `@skip_params_if` decorators to your benchmark functions or classes.
affects: asv-runner>=0.1.0 (corresponds to asv>=0.6.0)
gotchaASV Runner (`asv-runner`) is primarily a low-level component designed to be used by the higher-level `asv` benchmarking tool. Most users should interact with the `asv` command-line interface (e.g., `asv run`, `asv quickstart`) or import from the main `asv` package for comprehensive benchmarking, environment management, and result analysis. Attempting to directly import and use `asv_runner` modules for high-level tasks will result in incomplete functionality and a lack of orchestration.fixFor full benchmarking workflows, install the `asv` package (`pip install asv`) and use its command-line tools. Only delve into `asv_runner` directly for extending ASV's core capabilities or for very specific, low-level interactions.
affects: All versions
gotchaASV Runner, when invoked by `asv`, automatically computes a 'version number' for each benchmark based on its source code. If the benchmark's code changes, this version number updates, causing `asv` to ignore previously collected results for that benchmark. This can lead to unexpected re-benchmarking if not understood.fixBe aware that modifying benchmark code will invalidate old results for that benchmark. If you need fine-grained control over benchmark versioning (e.g., to group different versions of a benchmark), you can explicitly set the `.version` attribute on your benchmark function or class.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'asv_runner'
The 'asv_runner' package is not installed in the Python environment.
fixInstall the package using pip: 'pip install asv_runner'.
ImportError: cannot import name 'Benchmark' from 'asv_runner'
The 'Benchmark' class is not available in the 'asv_runner' module.
fixEnsure you are importing the correct class or function from 'asv_runner'. Refer to the official documentation for the correct import statements.
TypeError: 'NoneType' object is not callable
Attempting to call a function or method that is None, possibly due to a failed import or incorrect assignment.
fixVerify that all necessary modules and functions are correctly imported and initialized before use.
AttributeError: module 'asv_runner' has no attribute 'run_benchmark'
The 'asv_runner' module does not have a 'run_benchmark' attribute, possibly due to a version mismatch or incorrect import.
fixCheck the module's documentation to confirm the available attributes and ensure you are using the correct version of 'asv_runner'.
ValueError: Invalid benchmark configuration
The benchmark configuration provided is invalid or missing required parameters.
fixReview the benchmark configuration settings and ensure all required parameters are correctly specified as per the documentation.
Upgrade
Version history
0.2.1latest on PyPI · released Feb 11, 2024
Audit
Dependencies
No dependency data recorded yet.