Install & Compatibility
Where this runs
tested against v2.3.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
2/3 runs
py 3.11
✕ build_error
✓ 84.1s
py 3.12
✕ build_error
✓ 76.87s
py 3.13
✕ build_error
✓ 71.73s
py 3.9
✕ build_error
✕ timeout
5120MB installed
● package 5120MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
ORTModelForCausalLM
✓ from optimum.onnxruntime import ORTModelForCausalLM
✗ from optimum.onnxruntime import ORTModelForCausalLM
This quickstart demonstrates how to load an existing ONNX Runtime optimized model from the Hugging Face Hub and use it with a Hugging Face pipeline for accelerated text generation. It utilizes `ORTModelForCausalLM` and `optimum.onnxruntime.pipeline` for seamless integration and inference.
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForCausalLM, pipeline
import os
# Load an already optimized ONNX Runtime model from the Hugging Face Hub
model_id = "optimum/gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForCausalLM.from_pretrained(model_id)
# Create a pipeline using the ONNX Runtime optimized model
text_generator = pipeline("text-generation", model=model, tokenizer=tokenizer)
# Generate text
prompt = "My name is Philipp"
result = text_generator(prompt, max_new_tokens=10)
print(result)
Debug
Known issues
breakingBreaking Change in v2.0.0: ONNX integration (export and ONNX Runtime inference) was moved to a separate package, `optimum-onnx`. Users upgrading from v1.x must install `optimum-onnx` (e.g., `pip install "optimum-onnx[onnxruntime]"`) to retain ONNX functionality.fixInstall `optimum-onnx` alongside `optimum`. For ONNX Runtime support, use `pip install "optimum-onnx[onnxruntime]"`.
affects: >=2.0.0
deprecated`AutoGPTQ` functionality has been fully deprecated in v2.1.0 in favor of `GPTQModel`. Users should migrate to `GPTQModel` for quantization-aware training and inference with GPTQ.fixReplace imports and usage of `AutoGPTQ` with `GPTQModel`.
affects: >=2.1.0
deprecatedIn v2.0.0, support for TF Lite, BetterTransformer, and ONNX Runtime Training was deprecated and subsequently removed or moved to other specialized packages. TensorFlow model export was also removed.fixConsult `optimum` documentation for alternative approaches or specific sub-libraries if these functionalities are required.
affects: >=2.0.0
gotchaThe `export=True` argument in `ORTModelForCausalLM.from_pretrained` (and similar `ORTModel` classes) became optional and often inferred from v1.25.0 onwards. Using it explicitly might not be necessary or could lead to unexpected behavior if not handled correctly in newer versions.fixReview model loading calls; `export=True` is often no longer needed as the export process can be inferred or handled by `optimum-cli`.
affects: >=1.25.0
gotchaCompatibility issues can arise between specific `optimum` versions and newer `transformers` versions, particularly regarding internal module imports like `TF2_WEIGHTS_NAME` (fixed in v2.1.0). Ensure your `optimum` and `transformers` installations are compatible.fixUpgrade `optimum` to the latest version to ensure compatibility with recent `transformers` releases. Always check release notes for specific version requirements.
affects: <2.1.0 with newer Transformers
Upgrade
Version history
2.3.0latest on PyPI · released Aug 4, 2026
Audit
Dependencies
transformersrequiredCore dependency; Optimum extends and optimizes Hugging Face Transformers models.
optimum-onnxoptionalContains ONNX export and ONNX Runtime inference integrations, split from main `optimum` in v2.0.0.
onnxruntimeoptionalRuntime for executing ONNX models; typically installed via an extra like `optimum[onnxruntime]` or `optimum-onnx[onnxruntime]`.