perf-analyzer

library2.59.1pypypiunverified

Triton Performance Analyzer (perf_analyzer) is a command-line interface (CLI) tool designed to optimize the inference performance of models running on the NVIDIA Triton Inference Server. It measures key metrics such as throughput and latency by generating inference requests to your model and repeating measurements until stable values are achieved. The library is currently at version 2.59.1 and follows the release cadence of the broader Triton Inference Server project.

ai-ml observability devops

pip install perf-analyzer

package ↗homepage ↗

INSTALL

IMPORT

SIG · PERF-ANALYZER

perf-analyzer

ai-mlpythonv2.59.1

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

This quickstart demonstrates how to run `perf_analyzer` against a hypothetical model named 'my_model' on a running Triton Inference Server. It measures performance over a 5-second interval across varying concurrency levels (1, 3, 5, 7).

# Assuming Triton Inference Server is running at localhost:8000 with a model named 'my_model' # First, ensure Triton is running. Example (simplified): # docker pull nvcr.io/nvidia/tritonserver:24.02-py3 # docker run --gpus all --rm -it --net host nvcr.io/nvidia/tritonserver:24.02-py3 # (Inside container) tritonserver --model-repository /models & # Run perf_analyzer from a terminal where Triton is accessible perf_analyzer -m my_model --measurement-interval 5000 --concurrency-range 1:8:2

perf_analyzer --version

Debug

Known issues

deprecatedThe related tool `genai-perf` is being deprecated. Users should migrate to `AIPerf` for continued support and enhanced features in generative AI model benchmarking.

fix

For generative AI model benchmarking, consider using AIPerf instead of genai-perf. Consult NVIDIA Triton documentation for AIPerf migration guides.

affects: All versions

gotchaWhen installing `perf-analyzer` via `pip`, runtime dependencies (e.g., CUDA-related libraries for GPU support, `tritonclient` dependencies) are not automatically managed. Missing dependencies will cause errors during execution.

fix

Manually install any reported missing runtime dependencies. The recommended installation method is via the Triton SDK Docker container, which includes all necessary pre-built executables and dependencies.

affects: All versions

gotchaDirect C API mode within `perf_analyzer` has known limitations, including lack of support for asynchronous mode (`-a`), shared memory mode (`--shared-memory`), and request rate range mode.

fix

Avoid these specific options when using `--service-kind=triton_c_api`. Consider HTTP or gRPC endpoints for full feature support if these functionalities are critical.

affects: All versions

gotchaPerformance metrics, especially latency, can vary significantly between runs when not using shared memory. Using `--shared-memory=system` or `--shared-memory=cuda` can lead to more stable and representative results by reducing network overhead.

fix

For more stable and potentially lower-latency measurements, use the `--shared-memory=system` (for CPU shared memory) or `--shared-memory=cuda` (for GPU shared memory) options when benchmarking.

affects: All versions

gotchaRunning multiple `perf_analyzer` processes concurrently against a single Triton Inference Server instance can lead to unexpected behavior or issues.

fix

It is generally recommended to run a single `perf_analyzer` instance per Triton server or manage concurrent testing through a single `perf_analyzer` process's built-in concurrency options. If multi-model analysis is needed, consider `Triton Model Analyzer`.

affects: All versions

Errors

Common errors & fixes

Failed to maintain requested inference load. Worker thread(s) failed to generate concurrent requests. Thread [0] had error: Failed to increase the shared memory pool size for key 'triton_python_backend_shm_region_2' to X bytes. If you are running Triton inside docker, use '--shm-size' flag to control the shared memory region size. Error: No space left on device.

The Docker container running Triton Inference Server, which perf_analyzer is testing, does not have enough shared memory allocated, especially for models using shared memory for inference or Python backends that dynamically grow shared memory.

fix

Increase the shared memory allocated to the Docker container by using the `--shm-size` flag when starting the Triton Inference Server container. Example: `docker run --shm-size=1g ...`

error: failed to create concurrency manager: input <input_name> contains dynamic shape, provide shapes to send along with the request.

The model being analyzed has dynamic input shapes, but perf_analyzer was not explicitly told what specific shapes to use for the inference requests.

fix

Specify the concrete input shapes using the `--shape` flag in the perf_analyzer command. Example: `perf_analyzer -m my_model --shape input_1:1,224,224,3`

perf_analyzer: unrecognized option '--verbose-csv'

The perf_analyzer version being used does not support the specified command-line option, often due to a version mismatch between perf_analyzer and model_analyzer, or an older perf_analyzer version.

fix

Ensure perf_analyzer and model_analyzer (if used) are compatible versions, or check the documentation for your specific perf_analyzer version to verify supported options. If running in Docker, pull the latest SDK container.

Upgrade

Version history

2.59.1latest on PyPI · released Aug 13, 2025

Audit

Dependencies

No dependency data recorded yet.

Agent activity

16 hits · last 30 days

node

OpenAI (training)

Amazon

Resources

packagehttps://pypi.org/project/perf-analyzer/ ↗

homepagehttps://developer.nvidia.com/triton-inference-server ↗