Install & Compatibility
Where this runs
tested against v3.10.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ build_error
✓ 77.45s
py 3.11
✕ build_error
✓ 72.8s
py 3.12
✕ build_error
✓ 66.7s
py 3.13
✕ build_error
✓ 60.78s
py 3.9
✕ build_error
1/4 runs
4838MB installed
● package 4838MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
A2Grad
✓ from pytorch_optimizer import A2Grad
✗ from torch_optimizer import AdamP
ADOPT
✓ from pytorch_optimizer import ADOPT
APOLLO
✓ from pytorch_optimizer import APOLLO
This quickstart demonstrates how to initialize a model and apply an optimizer from the `torch_optimizer` library, using AdamP as an example. It includes a basic forward and backward pass to illustrate its use in a typical PyTorch training loop. You can directly import specific optimizers or use `load_optimizer` and `create_optimizer` for dynamic loading.
import torch
import torch.nn as nn
from torch_optimizer import AdamP
# Define a simple model
class SimpleModel(nn.Module):
def __init__(self):
super().__init__()
self.linear = nn.Linear(10, 1)
def forward(self, x):
return self.linear(x)
model = SimpleModel()
# Initialize AdamP optimizer with model parameters
# Replace with other optimizers like A2GradExp, AdaBelief, etc.
optimizer = AdamP(model.parameters(), lr=1e-3, betas=(0.9, 0.999), weight_decay=1e-2)
# Example of a training step
inputs = torch.randn(32, 10) # Batch of 32, 10 features
targets = torch.randn(32, 1) # Corresponding targets
optimizer.zero_grad()
outputs = model(inputs)
loss = torch.nn.functional.mse_loss(outputs, targets)
loss.backward()
optimizer.step()
print(f"Initial loss: {loss.item():.4f}")
Debug
Known issues
breakingThe RAdam optimizer was temporarily removed in version `0.2.0` due to its inclusion in PyTorch core, causing `ImportError` for existing users. It was subsequently re-added in version `0.3.0` ('Revert for Drop RAdam'). Users on `0.2.0` will not have RAdam available directly from `torch_optimizer`.fixUpgrade to `pytorch-optimizer>=0.3.0` to regain RAdam, or switch to `torch.optim.RAdam` for versions where it's available in PyTorch.
affects: 0.2.0
gotchaThe PyPI package is named `pytorch-optimizer`, but the correct Python module to import is `torch_optimizer`. Attempting to `import pytorch_optimizer` will result in an `ModuleNotFoundError`.fixAlways use `import torch_optimizer` or `from torch_optimizer import ...` for importing classes and functions.
affects: All versions
gotchaThe library author advises against selecting optimizers solely based on visualizations. Different optimizers have unique properties and may require specific learning rate schedules or tuning. It is recommended to start with built-in PyTorch optimizers like SGD or Adam to establish a baseline before experimenting with `pytorch-optimizer` variants.
Upgrade
Version history
3.10.1latest on PyPI · released May 23, 2026
Audit
Dependencies
torchrequiredCore dependency for PyTorch optimizers.
pythonrequiredRequired Python version.