rsl-rl-lib

library5.4.1pypypi✓ verified 87d ago

Fast and simple reinforcement learning algorithms (PPO) implemented in PyTorch, primarily designed for robotics applications like Isaac Lab. Current version 5.2.0, released April 2025. Active development with frequent releases.

ai-ml

pip install rsl-rl-lib

github ↗package ↗

INSTALL

IMPORT

SIG · RSL-RL-LIB

rsl-rl-lib

ai-mlpythonv5.4.1

Install

76.1s avg

Import

—

Disk

5094MB

Pass rate

4/ 10

Env Coverage4 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v5.4.1 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

✓ 87.4s

py 3.11

✕ build_error

✓ 80.8s

py 3.12

✕ build_error

✓ 71.3s

py 3.13

✕ build_error

✓ 64.8s

py 3.9

✕ build_error

✕ timeout

5094MB installed

● package 5094MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

Runner

✓ from rsl_rl.runners import OnPolicyRunner

✗ from rsl_rl.runner import OnPolicyRunner

Module is 'runners' (plural) since v5.0.0

PPO

✓ from rsl_rl.algorithms import PPO

✗ from rsl_rl.algos import PPO

Module is 'algorithms' (not 'algos') since v5.0.0

ActorCritic

✓ from rsl_rl.models import ActorCritic

Direct import; no common wrong import known

Basic training loop using OnPolicyRunner

import gym import torch from rsl_rl.runners import OnPolicyRunner from rsl_rl.algorithms import PPO from rsl_rl.modules import ActorCritic # Initialize environment (example using gym) env = gym.make('CartPole-v1') # Setup model and algorithm actor_critic = ActorCritic( num_actor_obs=env.observation_space.shape[0], num_critic_obs=env.observation_space.shape[0], num_actions=env.action_space.shape[0], ).to('cuda') algo = PPO(actor_critic=actor_critic, num_learning_epochs=5) # Create runner and train runner = OnPolicyRunner(env, algo, device='cuda') runner.learn(num_learning_iterations=100, init_at_random_ep_len=True) print('Training complete!')

Debug

Known issues

breakingv5.0.0 restructured library: modules 'runners' (not 'runner'), 'algorithms' (not 'algos'), 'models' (was 'actor_critic'). Old imports break.

fix

Update imports to new structure: from rsl_rl.runners import OnPolicyRunner; from rsl_rl.algorithms import PPO; from rsl_rl.models import ActorCritic

affects: >=5.0.0

breakingv5.0.0 introduced Batch class; positional argument order in RolloutStorage changed. Incorrect ordering can cause silent tensor switching.

fix

Use named arguments (e.g., obs=, actions=) instead of positional when calling RolloutStorage methods.

affects: >=5.0.0

deprecatedv5.0.0 deprecates old configuration format. The new config uses nested dicts instead of flat parameters for algorithm and runner settings.

fix

Use config dict with keys 'algorithm' and 'runner' as shown in docs.

affects: >=5.0.0

gotchatorch.compile mode 'default' may slow down training for small MLP networks; only beneficial for large CNNs.

fix

Disable compile for simple models: set model.cuda() and skip compile.

affects: >=5.1.0

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'rsl_rl.runner'

Module renamed from 'runner' to 'runners' in v5.0.0.

fix

Use import 'from rsl_rl.runners import OnPolicyRunner'

AttributeError: module 'rsl_rl' has no attribute 'algorithms'

Old import path for PPO algorithm.

fix

Use 'from rsl_rl.algorithms import PPO'

ValueError: The number of observations and actions do not match

Mismatch between environment observation/action spaces and ActorCritic dimensions.

fix

Ensure num_actor_obs, num_critic_obs, num_actions match the environment's observation and action spaces.

Upgrade

Version history

5.4.1latest on PyPI · released Jun 4, 2026

Audit

Dependencies

torchrequiredCore dependency for neural networks and tensor operations

Agent activity

8 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/leggedrobotics/rsl_rl ↗

packagehttps://pypi.org/project/rsl-rl-lib/ ↗