Install & Compatibility
Where this runs
tested against v2.2.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
1/2 runs
py 3.11
✕ build_error
1/2 runs
py 3.12
✕ build_error
1/2 runs
py 3.13
✕ build_error
1/2 runs
py 3.9
✕ build_error
1/2 runs
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
LLM
✓ from ipex_llm import LLM
✗ from ipex_llm import LLM
This quickstart demonstrates how to load an LLM using the `ipex_llm.LLM` class for simple inference or `ipex_llm.transformers.AutoModel` and `AutoTokenizer` for more fine-grained control and compatibility with Hugging Face Transformers. Ensure your `model_name` or `model_id` points to a valid local path or Hugging Face model.
from ipex_llm import LLM
# Instantiate LLM model
model = LLM(
model_name='/path/to/your/model',
optimize_type='int4',
dtype='auto',
trust_remote_code=True
)
# Example for text generation
prompt = "What is the capital of France?"
output = model(prompt)
print(output)
# For AutoModel/AutoTokenizer
from ipex_llm.transformers import AutoModel, AutoTokenizer
model_id = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(
model_id,
load_in_4bit=True, # or load_in_low_bit, quantize=4 etc.
torch_dtype='auto'
)
input_ids = tokenizer.encode(prompt, return_tensors="pt")
output = model.generate(input_ids, max_new_tokens=32)
print(tokenizer.decode(output[0], skip_special_tokens=True))
ipex-llm --version
Debug
Known issues
breakingThe library was rebranded from `BigDL-LLM` to `ipex-llm`. This changes package names, import paths, and some CLI tools.fixUpdate your `pip install` commands from `bigdl-llm` to `ipex-llm`. Adjust import statements from `from bigdl.llm...` to `from ipex_llm...`.
affects: All versions prior to 2.x (BigDL-LLM) when migrating to 2.x (ipex-llm).
gotchaIPEX-LLM installations are hardware-specific. Users must install the correct extras for their target platform (`[cpu]` for Intel CPUs or `[xpu]` for Intel GPUs). Installing without the correct extra may lead to missing dependencies or suboptimal performance.fixAlways use `pip install ipex-llm[cpu]` or `pip install ipex-llm[xpu]` as appropriate for your system. Refer to the official documentation for detailed hardware requirements.
affects: All versions
gotchaIPEX-LLM's performance is highly dependent on specific PyTorch and underlying Intel oneAPI library versions. Incompatible versions can lead to errors or degraded performance.fixAlways check the official IPEX-LLM documentation and release notes for the recommended PyTorch and oneAPI versions for your IPEX-LLM release. Consider using the `--pre` flag during installation to get the latest compatible builds.
affects: All versions
deprecatedWhile still functional, the `ipex_llm.optimize_model` API is being superseded by the higher-level `ipex_llm.LLM` and `ipex_llm.transformers.AutoModel` APIs for model loading and quantization.fixFor new code, prefer using `ipex_llm.LLM` or `ipex_llm.transformers.AutoModel.from_pretrained()` with `load_in_4bit`/`load_in_low_bit` for easier integration and more streamlined workflows.
affects: 2.x onwards
Upgrade
Version history
2.2.0latest on PyPI · released Apr 7, 2025
Audit
Dependencies
torchrequiredIPEX-LLM is built on PyTorch and requires a compatible version.