Gymbag¶
Gymbag is a Python 3 library for easy, efficient, single-file storage of OpenAI Gym reinforcement learning environment data. It stores observations, actions, and rewards in portable, compressed HDF5 files. You can easily playback the data for training or testing, or read it in for analysis.
Gymbag automatically appends to existing files, so you can easily accumulate all your training data in one place. You can also store multiple data sets from separate experiments or environments in the same file, along with descriptions to keep them straight. You can store arbitrary metadata with each step (in the info dict). Gymbag does not store rendered output (movies). It’s also easy to add other storage formats.
Recording¶
# Wrap env with HDF5 recorder, specifying filename
env = record_hdf5(gym.make('CartPole-v0'), 'cartpole.h5')
for episode in range(2):
env.reset()
done = False
while not done:
obs, reward, done, info = env.step(env.action_space.sample())
Reading¶
Reading the data back is just as easy:
for episode in HDF5Reader('cartpole.h5'):
for time, obs, action, reward, done, info in episode:
print(time, obs, action, reward, done, info)
1500617412.784789 [ 0.03698143 0.01418633 0.01207788 0.00391994] 0.0 nan False None
1500617412.78494 [ 0.03726516 -0.18110673 0.01215627 0.30038899] 0.0 1.0 False None
1500617412.784968 [ 0.03364302 0.01383986 0.01816405 0.01156456] 1.0 1.0 False None
1500617412.784994 [ 0.03391982 0.20869666 0.01839535 -0.27533251] 1.0 1.0 False None
1500617412.785022 [ 0.03809375 0.01331716 0.0128887 0.02309511] 0.0 1.0 False None
...
Playback¶
env = PlaybackEnv(HDF5Reader('cartpole.h5'))
while not env.played_out:
env.reset()
done = False
while not done:
obs, reward, done, info = env.step(env.action_space.sample())
print(obs, reward, done, info)
Performance¶
Typically wrapping an environment with Gymbag results in less than a 2X slowdown. Here is a comparison to gym_recording (which dumps data as uncompressed binary blobs):
| Environment | Time (s) | Space (MB) | |||
|---|---|---|---|---|---|
| unwrapped | gym_recording | gymbag.hdf5 | gym_recording | gymbag.hdf5 | |
| Cartpole-v0 | 0.38 (1X) | 0.63 (1.7X) | 0.64 (1.7X) | 1.6 (3.2X) | 0.5 (1X) |
| Breakout-v0 | 1.72 (1X) | 2.20 (1.3X) | 3.34 (1.9X) | 223 (74X) | 3 (1X) |
Extras¶
Also comes with tools for recording to memory, generating test data, comparing data between runs, converting saved data formats, and more.