A `KeyError` occurred when running the example code.

[JixiongRen]

Hi everyone, @samuelmyoussef @alxbilger @EulalieCoevoet @damienmarchal @ChristianDuriez

I encountered several issues while using SofaGym:

Issue 1:

I frequently get a KeyError when running example code with SofaGym. For instance, when running the following code (consistent with the README example):

import gym
import sofagym.envs

path = "data"

env = gym.make('trunk-v0')
env.seed(42)
observation = env.reset()

done = False
while not done:
    action = env.action_space.sample()  # this is where you would insert your policy
    observation, reward, done, info = env.step(action)
    env.render()
   
env.close()

I receive the following exception:

ub20@JixiongRenPC:~/SOFA_v23.06.00_Linux/bin/SofaGym$ /bin/python /home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/test.py
---------------------------------------
Checking SOFA_ROOT and SOFAPYTHON3_ROOT
Using environment variable SOFA_ROOT: /home/ub20/SOFA_v23.06.00_Linux
---------------------------------------
pygame 2.6.1 (SDL 2.28.4, Python 3.8.10)
Hello from the pygame community. https://www.pygame.org/contribute.html
[INFO]    [PluginManager] Loaded plugin: /home/ub20/SOFA_v23.06.00_Linux/lib/libSofa.Component.so
[ERROR]   [SofaRuntime] AttributeError: 'NoneType' object has no attribute 'get'
Traceback (most recent call last):
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/test.py", line 6, in <module>
    env = gym.make('trunk-v0')
  File "/home/ub20/.local/lib/python3.8/site-packages/gym/envs/registration.py", line 235, in make
    return registry.make(id, **kwargs)
  File "/home/ub20/.local/lib/python3.8/site-packages/gym/envs/registration.py", line 129, in make
    env = spec.make(**kwargs)
  File "/home/ub20/.local/lib/python3.8/site-packages/gym/envs/registration.py", line 90, in make
    env = cls(**_kwargs)
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/sofagym/envs/Trunk/TrunkEnv.py", line 53, in __init__
    super().__init__(config)
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/sofagym/AbstractEnv.py", line 134, in __init__
    self.config["dt"] = self.config.get('dt', 0.01)

Initially, I tried adding configurations to the gym.make() ， like this:

env = gym.make('trunk-v0', config={"scene": "Trunk",
                                "deterministic": True,
                                "source": [300, 0, 80],
                                "target": [0, 0, 80],
                                "goalList": [[40, 40, 100], [-10, 20, 80]],
                                "start_node": None,
                                "scale_factor": 5,
                                "timer_limit": 250,
                                "timeout": 50,
                                "display_size": (1600, 800),
                                "render": 1,
                                "save_data": False,
                                "save_image": False,
                                "save_path": path + "/Results" + "/Trunk",
                                "planning": False,
                                "discrete": True,
                                "seed": None,
                                "start_from_history": None,
                                "python_version": "python3",
                                "dt": 0.01,
                                "nb_actions": 2, 
                                "dim_state": 3,
                                "goalPos": [0, 0, 80],
                                "randomize_states": False,})

Most of these config options were copied from TrunkEnv.py. However, this did not resolve the issue, and I still encountered:

[ERROR]   [SofaRuntime] KeyError: 'goal'
Traceback (most recent call last):
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/test.py", line 31, in <module>
    observation = env.reset()
  File "/home/ub20/.local/lib/python3.8/site-packages/gym/wrappers/order_enforcing.py", line 16, in reset
    return self.env.reset(**kwargs)
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/sofagym/envs/Trunk/TrunkEnv.py", line 75, in reset
    super().reset()
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/sofagym/AbstractEnv.py", line 392, in reset
    self.init_simulation()
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/sofagym/AbstractEnv.py", line 507, in init_simulation
    if self.config["goal"]:

Clearly, this exception cannot be fixed by simply adding missing configurations.

---

Issue 2:

When attempting to run python3 test_env.py -e trunk-v0 -ep 100 -s 100, I get:

ub20@JixiongRenPC:~/SOFA_v23.06.00_Linux/bin/SofaGym$ python3 test_env.py -e trunk-v0 -ep 100 -s 100
---------------------------------------
Checking SOFA_ROOT and SOFAPYTHON3_ROOT
Using environment variable SOFA_ROOT: /home/ub20/SOFA_v23.06.00_Linux
---------------------------------------
pygame 2.6.1 (SDL 2.28.4, Python 3.8.10)
Hello from the pygame community. https://www.pygame.org/contribute.html
[INFO]    [PluginManager] Loaded plugin: /home/ub20/SOFA_v23.06.00_Linux/lib/libSofa.Component.so
[ERROR]   Environment name does not exist.
Available environments:
['bubblemotion-v0', 'cartstem-v0', 'cartstemcontact-v0', 'catchtheobject-v0', 'concentrictuberobot-v0', 'diamondrobot-v0', 'gripper-v0', 'maze-v0', 'multigaitrobot-v0', 'simple_maze-v0', 'stempendulum-v0', 'trunk-v0', 'trunkcup-v0', 'cartpole-v0', 'catheter_beam-v0']

I tried adding sofagym/ to the environment variables, but it did not work.

---

Issue 3:

When trying to train with python rl.py -e trunk-v0 -a PPO, I encounter:

ub20@JixiongRenPC:~/SOFA_v23.06.00_Linux/bin/SofaGym$ python rl.py -e trunk-v0 -a PPO
Traceback (most recent call last):
  File "rl.py", line 18, in <module>
    from agents.RLberryAgent import RLberryAgent
  File "/home/ub20/SOFA_v23.06.00_Linux/bin/SofaGym/agents/RLberryAgent.py", line 12, in <module>
    from rlberry.agents.experimental.torch import SACAgent
  File "/home/ub20/.local/lib/python3.8/site-packages/rlberry/__init__.py", line 12, in <module>
    configure_logging(level="INFO")
  File "/home/ub20/.local/lib/python3.8/site-packages/rlberry/utils/logging.py", line 112, in configure_logging
    gym.logger.set_level(
AttributeError: module 'gymnasium.logger' has no attribute 'set_level'

---

My environment:

• Python 3.8
• SOFA v23.06 (binary version with required plugins included)
• Configured exactly as per the README (except SPLIB plugin, as I was informed by SOFA that SPLIB is now integrated into STLIB)
  I have spent significant time setting up SofaGym, but none of the examples work, which is highly frustrating :-(. I believe I need your assistance.

Thank you!

[samuelmyoussef]

Hello,
Can you try to pull the latest commit and let me know if the issue persists. Also, make sure to have the compatible versions of gym, stable baselines and rlberry.

[JixiongRen]

Hello, Can you try to pull the latest commit and let me know if the issue persists. Also, make sure to have the compatible versions of gym, stable baselines and rlberry.

Hi, @samuelmyoussef

I pulled the latest repository, and both issue 1 and issue 2 mentioned before have been resolved. They are running very well without any issues. However, the problem still persists when executing the training code, that is, when calling python rl.py -e trunk-v0 -a PPO:

ub20@JixiongRenPC:~/SofaGym$ python rl.py -e trunk-v0 -a PPO
Traceback (most recent call last):
  File "rl.py", line 18, in <module>
    from agents.RLberryAgent import RLberryAgent
  File "/home/ub20/SofaGym/agents/RLberryAgent.py", line 12, in <module>
    from rlberry.agents.experimental.torch import SACAgent
  File "/home/ub20/.local/lib/python3.8/site-packages/rlberry/__init__.py", line 12, in <module>
    configure_logging(level="INFO")
  File "/home/ub20/.local/lib/python3.8/site-packages/rlberry/utils/logging.py", line 112, in configure_logging
    gym.logger.set_level(
AttributeError: module 'gymnasium.logger' has no attribute 'set_level'

I think there is no problem with the versions of gym, rlberry, and stable - baselines3 that I installed. To confirm, I conducted a check, and the results are as follows:

ub20@JixiongRenPC:~/SofaGym$ pip show gym stable-baselines3 rlberry
Name: gym
Version: 0.21.0
Summary: Gym: A universal API for reinforcement learning environments.
Home-page: https://github.com/openai/gym
Author: OpenAI
Author-email: jkterry@umd.edu
License: UNKNOWN
Location: /home/ub20/.local/lib/python3.8/site-packages
Requires: cloudpickle, numpy
Required-by: sofagym, stable-baselines3
---
Name: stable-baselines3
Version: 1.7.0
Summary: Pytorch version of Stable Baselines, implementations of reinforcement learning algorithms.
Home-page: https://github.com/DLR-RM/stable-baselines3
Author: Antonin Raffin
Author-email: antonin.raffin@dlr.de
License: MIT
Location: /home/ub20/.local/lib/python3.8/site-packages
Requires: cloudpickle, gym, importlib-metadata, matplotlib, numpy, pandas, torch
Required-by: 
---
Name: rlberry
Version: 0.5.0
Summary: An easy-to-use reinforcement learning library for research and education
Home-page: https://github.com/rlberry-py
Author: Omar Darwiche Domingues, Yannis Flet-Berliac, Edouard Leurent, Pierre Menard, Xuedong Shang
Author-email: 
License: MIT
Location: /home/ub20/.local/lib/python3.8/site-packages
Requires: dill, docopt, gymnasium, matplotlib, numpy, pandas, pygame-ce, pyyaml, scipy, seaborn, tqdm
Required-by: 

In fact, I've found that this anomaly is caused by import gymnasium in logging.py of rlberry. I tried changing import gymnasium as gym in the logging.py file to import gym. The previous anomaly didn't appear, but a new one did:

ub20@JixiongRenPC:~/SofaGym$ python rl.py -e trunk-v0 -a PPO
---------------------------------------
Checking SOFA_ROOT and SOFAPYTHON3_ROOT
Using environment variable SOFA_ROOT: /home/ub20/SOFA_v23.06.00_Linux
---------------------------------------
[INFO]    [PluginManager] Loaded plugin: /home/ub20/SOFA_v23.06.00_Linux/lib/libSofa.Component.so
[ERROR]   [SofaRuntime] AttributeError: module 'argparse' has no attribute 'BooleanOptionalAction'
Traceback (most recent call last):
  File "rl.py", line 82, in <module>
    action=argparse.BooleanOptionalAction, default=False)

This is because the Python 3.8 I'm using does not support argparse.BooleanOptionalAction. Tomorrow, I will recreate the environment using Python 3.9 and conduct the tests.

---

Anyway, I don't think it's appropriate to directly modify logging.py because it's not clear whether it will cause other exceptions. Do you have any solutions to this?

[samuelmyoussef]

If you're not planning on using algorithms from the rlberry library, you can comment this line

SofaGym/rl.py

Line 18 in f5ad4e4

from agents.RLberryAgent import RLberryAgent

[JixiongRen]

If you're not planning on using algorithms from the rlberry library, you can comment this line

SofaGym/rl.py

Line 18 in f5ad4e4

from agents.RLberryAgent import RLberryAgent

I'll give it a try.
However, so far I haven't been able to successfully run the SofaGym examples on Python 3.9 or higher versions. The reason is that the binary version of Sofa v23.06 is compiled based on Python 3.8, so the corresponding SofaPython3 plugin cannot run correctly on Python 3.9 or higher versions of Python. In fact, it wasn't until Sofa v23.12.01 that the supported Python version was upgraded from 3.8 to 3.10. I hope to use the pre - compiled Sofa binary files instead of compiling from the source code. Can you make some modifications to make it support Python 3.8? (For example, modify the syntax like argparse.BooleanOptionalAction that is not supported in 3.8.)
Thank you!

[samuelmyoussef]

You can use python 3.10 with Sofa v23.06. Just make sure your numpy version is < 2.0, v1.26.0 should work fine.

[JixiongRen]

I tried to install numpy 1.26.0, but I got the following exception:

ub22@JixiongRenPC:~$ python
Python 3.10.12 (main, Feb  4 2025, 14:57:36) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import Sofa
---------------------------------------
Checking SOFA_ROOT and SOFAPYTHON3_ROOT
Using environment variable SOFA_ROOT: /home/ub22/SOFA_v23.06.00_Linux
---------------------------------------
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ub22/SOFA_v23.06.00_Linux/plugins/SofaPython3/lib/python3/site-packages/Sofa/__init__.py", line 140, in <module>
    import Sofa.Helper
ModuleNotFoundError: No module named 'Sofa.Helper'
>>> numpy.__version__
'1.26.0'

The following are the relevant environment variables I configured:

# >>> SOFA PATH >>>
export SOFA_ROOT="$HOME/SOFA_v23.06.00_Linux"
export SOFAPYTHON3_ROOT="$HOME/SOFA_v23.06.00_Linux"

export PYTHONPATH="$HOME/SOFA_v23.06.00_Linux/plugins/SofaPython3/lib/python3/site-packages:$PYTHONPATH"
export PYTHONPATH="$HOME/SOFA_v23.06.00_Linux/plugins/STLIB/lib/python3/site-packages:$PYTHONPATH"
# <<< SOFA PATH <<<

[samuelmyoussef]

In order for it to work, you need to build from source and provide the path of the python version you want to use in cmake.

As an alternative if you are unable to build from source, you can replace the argparse.BooleanOptionalAction with this:

def boolean_flag(parser, name, default=False, help_text=""):
    """Adds --name and --no-name flags to an argparse parser."""
    group = parser.add_mutually_exclusive_group()
    group.add_argument(f"--{name}", dest=name, action="store_true", help=help_text)
    group.add_argument(f"--no-{name}", dest=name, action="store_false", help=f"Disable {help_text.lower()}")
    parser.set_defaults(**{name: default})

boolean_flag(parser, "test", default=False, help_text="Testing flag")

However, I recommend you try to build from source if you can cause you might get other errors due to the python version mismatch.

[JixiongRen]

Thank you. I've modified it according to your suggestion, and it can now run on Python 3.8. I've decided to use it as a temporary solution. Later, I'll try to compile SOFA from the source code.

In addition, I found some new problems when running the training instructions:

When running python rl.py -e bubblemotion-v0 -a REINFORCE ：

>>   ... Done.
[ERROR]   [SofaRuntime] NameError: name 'REINFORCE' is not defined
Traceback (most recent call last):
  File "rl.py", line 122, in <module>
    agent = Agent(env_name, algo_name, seed, results_dir, max_episode_steps, n_envs)
  File "/home/ub20/SOFA-Framework/SofaGym/agents/SB3Agent.py", line 181, in __init__
    self.algo = eval(self.algo_name)
  File "<string>", line 1, in <module>

I suspect it's because SB3 doesn't include REINFORCE, so I tried using RLberry. However, when I ran python rl.py -e bubblemotion-v0 -a REINFORCE -f RLberry, I got:

[ERROR]   [SofaRuntime] AttributeError: 'RLberryAgent' object has no attribute 'env'
Traceback (most recent call last):
  File "rl.py", line 122, in <module>
    agent = Agent(env_name, algo_name, seed, results_dir, max_episode_steps, n_envs)
  File "/home/ub20/SOFA-Framework/SofaGym/agents/RLberryAgent.py", line 27, in __init__
    self.env = self.env_wrap()
  File "/home/ub20/SOFA-Framework/SofaGym/agents/RLberryAgent.py", line 159, in env_wrap
    self.env_wrapped = Monitor(self.env)

Next, I tried different parameters, such as python rl.py -e bubblemotion-v0 -a DQN -f RLberry, but it also threw an AttributeError exception.
My rlberry version is 0.5.0.
I'm not sure if this problem is caused by my modification of logging.py. Please try to reproduce this problem.

[samuelmyoussef]

By default, when you run the rl.py training script, the framework used is SB3 if you don't specify the framework argument -f. So, since the REINFORCE algorithm is not part of SB3 as you said, it causes the first error you had.

For rlberry, you will not be able to use it since the change I suggested here #66 (comment) disables it. I recommend keeping it like this for now since rlberry hasn't been fully tested with the new changes we made in SofaGym. You can use the algorithms in SB3 and for REINFORCE alternatives you can use any of the policy-gradient methods like PPO or A2C.

[JixiongRen]

Okay, I understand.
Today I tried to compile Sofa under Python 3.10. Most of the built - in environments in SofaGym can run normally. However, for the environments bubblemotion-v0 and catchtheobject-v0, an exception occurred when running the following code:

import gym
import sofagym.envs
import Sofa

env = gym.make('bubblemotion-v0')
env.seed(42)
observation = env.reset()

done = False
while not done:
    action = env.action_space.sample()  # this is where you would insert your policy
    observation, reward, done, info = env.step(action)
    env.render()

env.close()

>>  Init bubblemotion_config...
>>  Init bubblemotion...
[ERROR]   [SofaRuntime] ValueError: Object type SurfacePressureConstraint<Vec3d> was not created
The object 'SurfacePressureConstraint' is not in the factory.
But the following exits:
                      : SurfacePressureForceField (72% match)
                      : TaitSurfacePressureForceField (66% match)

Traceback (most recent call last):
  File "/home/ub22/SOFA-Framework/SofaGym/test.py", line 5, in <module>
    env = gym.make('bubblemotion-v0')
  File "/home/ub22/.local/lib/python3.10/site-packages/gym/envs/registration.py", line 235, in make
    return registry.make(id, **kwargs)
  File "/home/ub22/.local/lib/python3.10/site-packages/gym/envs/registration.py", line 129, in make
    env = spec.make(**kwargs)
  File "/home/ub22/.local/lib/python3.10/site-packages/gym/envs/registration.py", line 90, in make
    env = cls(**_kwargs)
  File "/home/ub22/SOFA-Framework/SofaGym/sofagym/envs/BubbleMotion/BubbleMotionEnv.py", line 83, in __init__
    self.env.init_root()
  File "/home/ub22/SOFA-Framework/SofaGym/sofagym/AbstractEnv.py", line 205, in init_root
    self.init_simulation()
  File "/home/ub22/SOFA-Framework/SofaGym/sofagym/AbstractEnv.py", line 482, in init_simulation
    self.create_scene(self.root, self.config, mode=mode)
  File "/home/ub22/SOFA-Framework/SofaGym/sofagym/envs/BubbleMotion/BubbleMotionScene.py", line 67, in createScene
    bubblemotion.onEnd(rootNode)
  File "/home/ub22/SOFA-Framework/SofaGym/sofagym/envs/BubbleMotion/BubbleMotion.py", line 43, in onEnd
    cavity = self._createCavity(self.board, i, self.max_pressure)
  File "/home/ub22/SOFA-Framework/SofaGym/sofagym/envs/BubbleMotion/BubbleMotion.py", line 87, in _createCavity
    cavity.addObject('SurfacePressureConstraint', template='Vec3', triangles='@topo.triangles',

I tried to find a solution on the issue page of sofa, but I didn't find one. I guess this was caused by a problem during the compilation process because I tested the same code in a Python 3.8 environment with a binary version without any exceptions.

[samuelmyoussef]

SurfacePressureConstraint is from the SoftRobots plugin. Did you include it in the build?
Try to check if the scene runs correctly using runSofa

[JixiongRen]

In the figure below, I have listed all the projects in CMake that are related to the plugin SoftRobots.

Tried to start the scene in Sofa Executing
./runSofa ~/SOFA-Framework/SofaGym/sofagym/envs/BubbleMotion/BubbleMotionScene.py did not cause any exceptions.

[samuelmyoussef]

I am not exactly sure, could you try to add this

export PYTHONPATH=$SOFA_ROOT/lib/python3/site-packages:$PYTHONPATH

[JixiongRen]

Thanks, But there is only a cmake folder in the $SOFA_ROOT/lib/ path, and there is no python3.