prodigyopt

library1.1.2pypypiunverified

ProdigyOpt is an Adam-like optimizer for neural networks, designed for high performance and memory efficiency. It features adaptive learning rate estimation and implements decoupled weight decay. The current version is 1.1.2, and releases typically focus on minor bug fixes and performance enhancements.

ai-ml

pip install prodigyopt

github ↗package ↗

INSTALL

IMPORT

SIG · PRODIGYOPT

prodigyopt

ai-mlpythonv1.1.2

Install

1.5s avg

Import

—

Disk

16MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.1.2 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.910 runs

installs and imports cleanly · install 0.0s · import 0.000s · 17.8MB

glibc

py 3.10–3.910 runs

installs and imports cleanly · install 1.5s · import 0.000s · 18MB

16MB installed

● package 16MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

Prodigy

✓ from prodigyopt import Prodigy

✗ from prodigyopt import Prodigy

This quickstart demonstrates how to initialize the Prodigy optimizer with a PyTorch model and perform a single optimization step. It includes model definition, loss calculation, and the standard optimizer workflow.

import torch import torch.nn as nn from prodigyopt import Prodigy # 1. Define a simple model model = nn.Linear(10, 2) # 2. Initialize the optimizer with model parameters # decouple_wd=True is default, but explicitly shown for clarity optimizer = Prodigy(model.parameters(), lr=1e-3, decouple_wd=True) # 3. Define a loss function loss_fn = nn.MSELoss() # 4. Prepare dummy data inputs = torch.randn(5, 10) targets = torch.randn(5, 2) # 5. Perform a training step optimizer.zero_grad() # Clear gradients outputs = model(inputs) # Forward pass loss = loss_fn(outputs, targets) # Compute loss loss.backward() # Backward pass to compute gradients optimizer.step() # Update model parameters print(f"Loss after one step: {loss.item():.4f}")

Debug

Known issues

gotchaThe `slice_p` parameter (introduced in v1.1, default 1) can significantly impact memory usage for large models. Higher values (e.g., 4) process parameters in slices, reducing peak memory at the cost of a slight performance overhead.

fix

Consider increasing `slice_p` if you encounter OutOfMemory errors with large models. Experiment to find the optimal balance between memory and speed for your specific setup.

affects: >=1.1

gotchaThe `decouple_wd` parameter defaults to `True` in Prodigy. This applies weight decay in a decoupled manner, which is generally desired for AdamW-like optimizers but might behave differently from other optimizers if you expect coupled weight decay.

fix

If you require coupled weight decay or different regularization behavior, explicitly set `decouple_wd=False` during optimizer initialization: `Prodigy(..., decouple_wd=False)`.

affects: All versions

breakingVersions prior to `1.1.2` had known issues when used with PyTorch's FSDP (Fully Sharded Data Parallel), particularly when some parameters were frozen, leading to incorrect behavior or crashes.

fix

Upgrade to `prodigyopt>=1.1.2` for full compatibility and stability with FSDP setups. Ensure all parameters passed to the optimizer are unfrozen or handled correctly by FSDP.

affects: <1.1.2

Upgrade

Version history

1.1.2latest on PyPI · released Jan 16, 2025

Audit

Dependencies

No dependency data recorded yet.

Agent activity

14 hits · last 30 days

node

Resources

githubhttps://github.com/konstmish/prodigy ↗

packagehttps://pypi.org/project/prodigyopt/ ↗