Install & Compatibility
Where this runs
tested against v0.8.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
✓ 77.5s
7526MB installed
● package 7526MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
AutoLigerKernelForCausalLM
✓ from liger_kernel.transformers import AutoLigerKernelForCausalLM
✗ from liger_kernel.transformers import AutoLigerKernelForCausalLM
The simplest way to integrate Liger-Kernel is by using `AutoLigerKernelForCausalLM` to automatically patch a Hugging Face Causal LM. For training, it often integrates with `transformers.Trainer` or TRL trainers by setting a flag.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from liger_kernel.transformers import AutoLigerKernelForCausalLM
# Ensure a GPU is available
if not torch.cuda.is_available():
print("CUDA not available. Liger-Kernel requires a GPU.")
exit()
# 1. Load your Hugging Face model and tokenizer
model_name = "PY007/TinyLlama-1.1B-Chat-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16).cuda()
# 2. Patch the model using AutoLigerKernelForCausalLM
# This will automatically replace compatible layers with Liger Kernels
# No explicit assignment needed, it modifies the model in-place (monkey-patching)
_ = AutoLigerKernelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16).cuda()
print(f"Model type after Liger Kernel patching: {type(model)}")
# Example usage (inference/forward pass - for training, integrate into your training loop)
input_text = "Hello, my name is"
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
with torch.no_grad():
outputs = model(**inputs)
logits = outputs.logits
print("Model successfully patched and executed.")
# For full training integration, you would typically use a Hugging Face Trainer
# or TRL trainer and set use_liger_kernel=True in your training arguments.
Debug
Known issues
gotchaLiger-Kernel fundamentally relies on GPU hardware (NVIDIA, AMD, or Intel) and the Triton framework for its performance optimizations. It will not provide benefits on CPU-only setups and requires a compatible PyTorch and Triton installation.fixEnsure you have a supported GPU and a PyTorch installation (`torch >= 2.1.2`) that is compatible with Triton. Verify `torch.cuda.is_available()` returns True.
affects: All versions
gotchaWhile Liger-Kernel generally integrates well with `torch.compile`, there have been specific reports where using both together for certain models (e.g., Orpheus-TTS) led to significantly slower training, despite memory reductions. Benchmark your specific workload.fixThoroughly benchmark your training pipeline with and without `torch.compile` when Liger-Kernel is enabled. If slowdowns occur, consider running without `torch.compile` or investigating potential incompatibilities with the specific model architecture.
affects: All versions
gotchaOptimal performance gains (e.g., 20% throughput increase, 60% memory reduction) are typically observed under specific benchmark conditions, such as training LLaMA 3-8B with `bf16` precision, `AdamW` optimizer, `FSDP1` on multiple A100 GPUs, and large sequence lengths/batch sizes. Results may vary for different models, hardware, or training configurations.fixUnderstand that advertised performance benefits are scenario-dependent. While Liger-Kernel generally improves efficiency, conduct your own benchmarks to confirm improvements for your specific use case.
affects: All versions
gotchaWhen upgrading `transformers` library, especially around major version changes or specific model refactorings, ensure Liger-Kernel has corresponding support. Version 0.7.0 explicitly added full support for Transformers v5 and all versions >= 4.52.0.fixAlways check Liger-Kernel release notes for `transformers` compatibility. For `transformers` versions below 4.52.0, you might need an older Liger-Kernel release or face unexpected behavior. Upgrade to Liger-Kernel 0.7.0 or newer for full Transformers v5 compatibility.
affects: <0.7.0 with Transformers > v5, or Transformers <4.52.0
Upgrade
Version history
0.8.2latest on PyPI · released Aug 18, 2026
Audit
Dependencies
torchrequiredCore dependency for PyTorch integration and GPU operations.
transformersoptionalRequired for patching Hugging Face models, a common use case.
tritonrequiredUnderlying framework for kernel implementation.