Install & Compatibility
Where this runs
tested against v0.9.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
✓ 49.6s
py 3.11
✕ build_error
✓ 47.73s
py 3.12
✕ build_error
✓ 43.38s
py 3.13
✕ build_error
✓ 44.18s
py 3.9
✕ build_error
✕ build_error
982MB installed
● package 982MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
MetricRecordInfo
✓ from aiperf.common.models import MetricRecordInfo
Used for parsing per-request metric records from benchmark output files (profile_export.jsonl).
Synthesizer
✓ from aiperf.dataset.synthesis import Synthesizer
Used for programmatic dataset synthesis, typically within custom plugin development.
random_generator as rng
✓ from aiperf.common import random_generator as rng
✗ import random
For reproducible data generation within plugins, always use `aiperf.common.random_generator.derive()` instead of Python's built-in `random` module, as `random`'s global state is fragile and can affect reproducibility.
This quickstart demonstrates running a basic performance benchmark against a locally running Ollama server. It profiles a specified model using a chat endpoint with streaming enabled, a specific tokenizer, and defines concurrency and request count.
python3 -m venv venv
source venv/bin/activate
pip install aiperf
# Assuming an Ollama server is running locally with a model like 'granite4:350m'
# (e.g., via docker run -d --name ollama -p 11434:11434 -v ollama-data:/root/.ollama ollama/ollama:latest && docker exec -it ollama ollama pull granite4:350m)
aiperf profile \
--model "granite4:350m" \
--streaming \
--endpoint-type chat \
--tokenizer ibm-granite/granite-4.0-micro \
--url http://localhost:11434 \
--concurrency 5 \
--request-count 10
aiperf --version
Debug
Known issues
gotchaOutput sequence length constraints (`--output-tokens-mean`) may not be guaranteed by the inference server unless `ignore_eos` and/or `min_tokens` are explicitly passed via `--extra-inputs` to a supporting server.fixConsult your inference server's documentation and use `--extra-inputs` to configure `ignore_eos` or `min_tokens` if precise output length control is critical for your benchmark.
affects: >=0.1.0
breakingLatency percentile metrics (P50, P90, P99) in AIPerf's current versions (prior to a potential future update) only consider successful requests. This can lead to misleading performance reports, especially in high error-rate scenarios, as failed requests are not factored into the percentiles.fixBe aware that reported latency percentiles only reflect successful requests. For a complete understanding, also examine the error rates and raw request data (`profile_export.jsonl`) to manually account for failed requests.
affects: >=0.1.0
gotchaVery high concurrency settings (typically >15,000) can lead to ephemeral port exhaustion on some systems, resulting in connection failures.fixIf encountering connection failures at high concurrency, reduce the `--concurrency` value. You may also need to adjust system-level ephemeral port limits (e.g., on Linux, by modifying `/proc/sys/net/ipv4/ip_local_port_range`).
affects: >=0.1.0
gotchaStartup errors caused by invalid configuration settings can cause AIPerf to hang indefinitely during initialization.fixIf AIPerf appears to freeze during startup, terminate the process (Ctrl+C) and thoroughly check all configuration settings and command-line arguments for syntax or logical errors.
affects: >=0.1.0
breakingWhen migrating from GenAI-Perf, the `payload` field in `inputs.json` has been renamed to `payloads` (plural) to better support multi-turn conversations. Additionally, a new `session_id` field has been added.fixUpdate your `inputs.json` files to use the `payloads` array and include `session_id` for each entry. Review the `Migrating from Genai-Perf` documentation for a detailed comparison.
affects: 0.1.0 - 0.7.0 (migration from GenAI-Perf)
Errors
Common errors & fixes
Connection refused / port exhaustion errors at high concurrency
The system has run out of available ephemeral ports to establish new outgoing connections, often due to extremely high concurrency settings (e.g., >15,000 requests).
fixReduce the `--concurrency` parameter in your AIPerf command. For persistent high-concurrency needs, consult your operating system's documentation on increasing ephemeral port limits.
AIPerf hangs indefinitely during startup/initialization
Invalid configuration settings or command-line arguments prevent the AIPerf system from initializing correctly, leading to a deadlock or infinite wait state.
fixTerminate the hung process (e.g., Ctrl+C) and carefully review all command-line arguments and configuration files for syntax errors, missing values, or logical inconsistencies.
Metrics for tokens-per-second or output sequence length appear incorrect when server tokenization differs from local tokenizer.
By default, AIPerf may use its local tokenizer to calculate token-based metrics. If the inference server uses a different tokenizer or tokenization strategy, these client-side calculations can be inaccurate.
fixEnable server token count support if your inference server provides it. This ensures AIPerf uses the server's own token counts for accurate throughput and output length metrics, reducing client-side overhead.
Upgrade
Version history
0.10.0latest on PyPI · released Jun 9, 2026
Audit
Dependencies
pythonrequiredRequires Python version 3.10 or newer.