Install & Compatibility
Where this runs
tested against v1.6 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
build_error
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 4.1s · import 0.272s · 89MB
90MB installed
● package 90MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
Environment
✓ from dm_env import Environment
TimeStep
✓ from dm_env import TimeStep
StepType
✓ from dm_env import StepType
specs
✓ from dm_env import specs
ArraySpec
✓ from dm_env.specs import ArraySpec
BoundedArraySpec
✓ from dm_env.specs import BoundedArraySpec
This quickstart defines a simple counting environment using `dm-env`'s `Environment` abstract base class. It showcases how to define action, observation, reward, and discount specifications using `dm_env.specs`, implement `_reset` and `_step` methods, and interact with the environment through `reset()` and `step()` calls. The example also highlights the `TimeStep` object and its `step_type` attribute for managing episode progression.
import numpy as np
from dm_env import Environment, TimeStep, specs, StepType
class SimpleCountingEnv(Environment):
def __init__(self, max_count=5):
self._max_count = max_count
self._current_count = 0
self._reset_next_step = True
def discount_spec(self):
return specs.BoundedArray(shape=(), dtype=float, minimum=0.0, maximum=1.0, name='discount')
def observation_spec(self):
return specs.BoundedArray(shape=(), dtype=int, minimum=0, maximum=self._max_count, name='count')
def action_spec(self):
return specs.BoundedArray(shape=(), dtype=int, minimum=0, maximum=1, name='action') # 0: no-op, 1: increment
def reward_spec(self):
return specs.Array(shape=(), dtype=float, name='reward')
def _reset(self):
self._current_count = 0
self._reset_next_step = False
return TimeStep(step_type=StepType.FIRST,
reward=None,
discount=None,
observation=np.asarray(self._current_count, dtype=int))
def _step(self, action):
if self._reset_next_step:
return self._reset()
if action == 1:
self._current_count += 1
if self._current_count >= self._max_count:
self._reset_next_step = True
return TimeStep(step_type=StepType.LAST,
reward=np.asarray(1.0, dtype=float),
discount=np.asarray(0.0, dtype=float),
observation=np.asarray(self._current_count, dtype=int))
else:
return TimeStep(step_type=StepType.MID,
reward=np.asarray(0.0, dtype=float),
discount=np.asarray(1.0, dtype=float),
observation=np.asarray(self._current_count, dtype=int))
def reset(self):
return self._reset()
def step(self, action):
return self._step(action)
# --- Example Usage ---
env = SimpleCountingEnv()
timestep = env.reset()
print(f"Initial: {timestep.observation}")
while not timestep.last():
action = 1 # Always try to increment
timestep = env.step(action)
print(f"Step {env._current_count}: Obs={timestep.observation}, Reward={timestep.reward}, Type={timestep.step_type.name}")
# Demonstrating reset after LAST timestep
timestep = env.step(0) # Action is ignored here
print(f"After last, calling step (action ignored): {timestep.observation}")
Errors
Common errors & fixes
AttributeError: 'ArraySpec' object has no attribute 'sample'
Unlike some other RL frameworks (e.g., OpenAI Gym spaces), `dm-env.specs.ArraySpec` and `BoundedArraySpec` do not inherently provide a `.sample()` method for generating random values within their defined bounds. Users often expect this functionality to easily sample actions or observations.
fixManually implement sampling logic based on the `dtype`, `shape`, `minimum`, and `maximum` properties of the spec. For example, use `np.random.uniform(spec.minimum, spec.maximum, size=spec.shape)` for `BoundedArraySpec` or `np.zeros(spec.shape, dtype=spec.dtype)` as a placeholder.
MemoryError / Detected OOM-kill event(s)
While not directly a `dm-env` issue, larger RL setups using `dm-env` environments, especially with extensive replay buffers or complex observation spaces, can lead to high memory consumption and Out-Of-Memory (OOM) errors. This is particularly common in deep RL agent implementations.
fixMonitor memory usage of your agent and environment. Reduce replay buffer size, optimize observation/action data types (e.g., use `np.uint8` instead of `np.float32` if appropriate), or consider using techniques like experience replay compression.
Incompatibility with Gymnasium/Gym-style environments or agents
The `dm-env` interface is distinct from the popular Gymnasium (formerly OpenAI Gym) API. Directly using environments implemented with `dm-env` alongside agents designed for Gymnasium, or vice-versa, requires conversion or wrappers.
fixUse a compatibility wrapper library like `shimmy` (e.g., `shimmy.DmControlCompatibilityV0` for `dm-control` environments which use `dm-env` internally) to convert `dm-env` environments to the Gymnasium interface if you need to use Gymnasium-compatible agents or tools. Alternatively, implement a custom adapter or wrapper.
Upgrade
Version history
1.6latest on PyPI · released Dec 21, 2022
Audit
Dependencies
numpyrequiredCore data structures (observations, actions, specs) are based on NumPy arrays.