Registry / ai-ml / lion-pytorch

lion-pytorch

JSON →
library0.2.4pypypi✓ verified 85d ago

lion-pytorch provides an efficient and high-performance implementation of the Lion optimizer for PyTorch. Based on the paper 'Symbolic Discovery of Optimization Algorithms', Lion often outperforms AdamW and other adaptive optimizers, especially in large-scale models, due to its sign-based update mechanism. The library is actively maintained, currently at version 0.2.4, and requires Python 3.9+.

pip install lion-pytorch
INSTALL
IMPORT
SIG · LION-PYTORCH
L
lion-pytorch
ai-mlpythonv0.2.4
Install
75.1s avg
Import
5409ms
Disk
4787MB
Pass rate
4/ 10
Env Coverage4 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.2.4 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
glibc
py 3.10
✕ build_error
✓ 84.95s
py 3.11
✕ build_error
✓ 71.65s
py 3.12
✕ build_error
✓ 71.55s
py 3.13
✕ build_error
✓ 72.25s
py 3.9
✕ build_error
✕ timeout
4787MB installed
● package 4787MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

Lion
from lion_pytorch import Lion

This quickstart demonstrates how to initialize a PyTorch model and the Lion optimizer, then perform a single forward and backward pass to update the model parameters. It highlights the typical workflow for integrating Lion into a PyTorch training loop.

import torch from torch import nn from lion_pytorch import Lion # 1. Define a simple PyTorch model class SimpleModel(nn.Module): def __init__(self): super().__init__() self.linear = nn.Linear(10, 2) self.relu = nn.ReLU() self.output = nn.Linear(2, 1) def forward(self, x): return self.output(self.relu(self.linear(x))) model = SimpleModel() # 2. Instantiate the Lion optimizer # Note: Lion often requires a smaller learning rate than AdamW (e.g., 1e-4) optimizer = Lion(model.parameters(), lr=1e-4, weight_decay=1e-2) # 3. Create dummy data and target inputs = torch.randn(32, 10) # 32 samples, 10 features targets = torch.randn(32, 1) # 32 samples, 1 target value # 4. Define a loss function criterion = nn.MSELoss() # 5. Training loop (one step for quickstart demonstration) optimizer.zero_grad() # Clear gradients from previous step outputs = model(inputs) # Forward pass loss = criterion(outputs, targets) # Calculate loss loss.backward() # Backward pass (compute gradients) optimizer.step() # Update model parameters print(f"Loss after one optimization step: {loss.item():.4f}")
Debug
Known issues
gotchaLion often requires a significantly lower learning rate (e.g., 1/3 to 1/10) compared to AdamW for optimal performance and stability. Directly applying learning rates common for AdamW may lead to issues like NaN losses or poor convergence.
fix
Experiment with learning rates in the range of 1e-5 to 1e-4 as a starting point, lower than typical AdamW values. Use a learning rate scheduler for better convergence.
affects: >=0.1.0
gotchaWhile powerful, Lion's optimal performance often requires re-tuning hyperparameters (especially `lr` and `betas`) specific to your task and model, rather than using it as a direct drop-in replacement with existing AdamW settings.
fix
Treat Lion as a new optimizer requiring independent hyperparameter search. Do not assume previous AdamW settings will transfer directly.
affects: >=0.1.0
gotchaUnlike optimizers like Adam, Lion uses only two `betas` values for momentum and update, and does *not* accept an `eps` (epsilon) parameter. Attempting to pass `eps` will result in a `TypeError`.
fix
Remove the `eps` argument from the `Lion` optimizer's constructor. Only provide `betas` if customizing beyond the default `(0.9, 0.99)`.
affects: >=0.1.0
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'lion_pytorch'
The `lion-pytorch` library is not installed in your current Python environment.
fix
Run `pip install lion-pytorch` to install the library.
TypeError: Lion.__init__() got an unexpected keyword argument 'eps'
You are attempting to pass an `eps` parameter to the `Lion` optimizer, which it does not support. This parameter is common in optimizers like Adam/AdamW.
fix
Remove the `eps` argument from the `Lion` optimizer's constructor.
TypeError: Lion.__init__() missing 1 required positional argument: 'params'
The `Lion` optimizer constructor requires an iterable of model parameters (e.g., `model.parameters()`) as its first argument.
fix
Ensure you pass `model.parameters()` to the optimizer: `optimizer = Lion(model.parameters(), lr=...)`.
Upgrade
Version history
0.2.4latest on PyPI · released Mar 4, 2026
Audit
Dependencies
torchrequiredRequired for PyTorch model and tensor operations.
Agent activity
7 hits · last 30 days
node
6
Amazon
1
Resources
lion-pytorch — pip install lion-pytorch · libregistry