custom gym environments are not compatible with current gym API #29
Description
I think you are depending on an older version of gym in some of the environments you made, a common error is
/usr/local/lib/python3.9/site-packages/gym/wrappers/time_limit.py in step(self, action)
48
49 """
---> 50 observation, reward, terminated, truncated, info = self.env.step(action)
51 self._elapsed_steps += 1
52
ValueError: not enough values to unpack (expected 5, got 4)
walk_env.py has step function return return (int(s), r, d, {"prob": p}) but the version of gym I'm on (0.26.0) seems to wrap your environment in some time limit and gym.make('RandomWalk-v0') returns something like this <TimeLimit<OrderEnforcing<PassiveEnvChecker<WalkEnv<RandomWalk-v0>>>>>.
Your dockerfile doesn't pin gym to any specific version so I don't know what the right version is.