Isaac Gym Environments for Legged Robots

This repository provides the environment used to train ANYmal (and other robots) to walk on rough terrain using NVIDIA's Isaac Gym. It includes all components needed for sim-to-real transfer: actuator network, friction & mass randomization, noisy observations and random pushes during training.

Maintainer: Nikita Rudin
Affiliation: Robotic Systems Lab, ETH Zurich
Contact: rudinn@ethz.ch

---

🔔 Announcement (09.01.2024)

With the shift from Isaac Gym to Isaac Sim at NVIDIA, we have migrated all the environments from this work to Isaac Lab. Following this migration, this repository will receive limited updates and support. We encourage all users to migrate to the new framework for their applications.

Information about this work's locomotion-related tasks in Isaac Lab is available here.

---

Useful Links

Project website: https://leggedrobotics.github.io/legged_gym/
Paper: https://arxiv.org/abs/2109.11978

1. Installation

Virtual Environment

1. Create a new python virtual env with python 3.6, 3.7 or 3.8 (3.8 recommended)

2. Install pytorch 1.13.1 with cuda 11.7:

   conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia

If you are using ubuntu 22 or later, you may need to config the LD_LIBRARY_PATH because the default python version is no longer 3.8 . Below is the turorial to config the LD_LIBRARY_PATH:

cd $CONDA_PREFIX
mkdir -p ./etc/conda/activate.d
mkdir -p ./etc/conda/deactivate.d
touch ./etc/conda/activate.d/env_vars.sh
touch ./etc/conda/deactivate.d/env_vars.sh

sudo nano ./etc/conda/activate.d/env_vars.sh

# add export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib

sudo nano ./etc/conda/deactivate.d/env_vars.sh

## add
ORIGINAL_LD_LIBRARY_PATH=$LD_LIBRARY_PATH
DIRECTORY_TO_REMOVE="$CONDA_PREFIX/lib"

NEW_LD_LIBRARY_PATH=$(echo $LD_LIBRARY_PATH | tr ':' '\n' | grep -v "$DIRECTORY_TO_REMOVE" | tr '\n' ':')
export LD_LIBRARY_PATH=$NEW_LD_LIBRARY_PATH

Check the LD_LIBRARY_PATH by running echo $LD_LIBRARY_PATH in the terminal. If the path is correct, you can see the path of the conda environment. After deactivating conda environment, the LD_LIBRARY_PATH will be reset to the original path.

Install Isaac Gym

• Download and install Isaac Gym Preview 4 from https://developer.nvidia.com/isaac-gym
• Install
  • cd isaacgym/python
  • pip install -e .
• Try running an example
  • cd examples && python 1080_balls_of_solitude.py
  • If cannot run success, try sudo prime-select on-demand
• For troubleshooting check docs isaacgym/docs/index.html

Install rsl_rl (PPO implementation)

• Clone https://github.com/leggedrobotics/rsl_rl
• cd rsl_rl && git checkout v1.0.2 && pip install -e .

Install legged_gym

• Navigate to the folder legged_gym
• pip install -e .

2. CODE STRUCTURE

1. Each environment is defined by an env file (legged_robot.py) and a config file (legged_robot_config.py). The config file contains two classes: one containing all the environment parameters (LeggedRobotCfg) and one for the training parameters (LeggedRobotCfgPPo).
2. Both env and config classes use inheritance.
3. Each non-zero reward scale specified in cfg will add a function with a corresponding name to the list of elements which will be summed to get the total reward.
4. Tasks must be registered using task_registry.register(name, EnvClass, EnvConfig, TrainConfig). This is done in envs/__init__.py, but can also be done from outside of this repository.

3. Usage

3.1 Check Environment

In the test environment, you can check if the pd controller is working correctly or whether the initial states are appropriate.

python legged_gym/tests/test_env.py --task=a1_flat

3.1 Train:

python legged_gym/scripts/train.py --task=a1_flat --headless

• To run on CPU add following arguments: --sim_device=cpu, --rl_device=cpu (sim on CPU and rl on GPU is possible).
• To run headless (no rendering) add --headless.

Important: To improve performance, once the training starts press v to stop the rendering. You can then enable it later to check the progress.

• The trained policy is saved in issacgym_anymal/logs/<experiment_name>/<date_time>_<run_name>/model_<iteration>.pt. Where <experiment_name> and <run_name> are defined in the train config.
• The following command line arguments override the values set in the config files:
• --task TASK: Task name.
• --resume: Resume training from a checkpoint
• --experiment_name EXPERIMENT_NAME: Name of the experiment to run or load.
• --run_name RUN_NAME: Name of the run.
• --load_run LOAD_RUN: Name of the run to load when resume=True. If -1: will load the last run.
• --checkpoint CHECKPOINT: Saved model checkpoint number. If -1: will load the last checkpoint.
• --num_envs NUM_ENVS: Number of environments to create.
• --seed SEED: Random seed.
• --max_iterations MAX_ITERATIONS: Maximum number of training iterations.

3.2. Play a trained policy:

python legged_gym/scripts/play.py --task=a1_flat

• By default, the loaded policy is the last model of the last run of the experiment folder.
• Other runs/model iteration can be selected by setting load_run and checkpoint in the train config.

3.3 Adding a new environment

The base environment legged_robot implements a rough terrain locomotion task. The corresponding cfg does not specify a robot asset (URDF/ MJCF) and has no reward scales.

1. Add a new folder to envs/ with '<your_env>_config.py, which inherit from an existing environment cfgs
2. If adding a new robot:
   • Add the corresponding assets to resources/.
   • In cfg set the asset path, define body names, default_joint_positions and PD gains. Specify the desired train_cfg and the name of the environment (python class).
   • In train_cfg set experiment_name and run_name
3. (If needed) implement your environment in <your_env>.py, inherit from an existing environment, overwrite the desired functions and/or add your reward functions.
4. Register your env in isaacgym_anymal/envs/__init__.py.
5. Modify/Tune other parameters in your cfg, cfg_train as needed. To remove a reward set its scale to zero. Do not modify parameters of other envs!

Troubleshooting

1. If you get the following error: ImportError: libpython3.8m.so.1.0: cannot open shared object file: No such file or directory, do: sudo apt install libpython3.8. It is also possible that you need to do export LD_LIBRARY_PATH=/path/to/libpython/directory / export LD_LIBRARY_PATH=/path/to/conda/envs/your_env/lib(for conda user. Replace /path/to/ to the corresponding path.).

Known Issues

1. The contact forces reported by net_contact_force_tensor are unreliable when simulating on GPU with a triangle mesh terrain. A workaround is to use force sensors, but the force are propagated through the sensors of consecutive bodies resulting in an undesirable behaviour. However, for a legged robot it is possible to add sensors to the feet/end effector only and get the expected results. When using the force sensors make sure to exclude gravity from the reported forces with sensor_options.enable_forward_dynamics_forces. Example:

    sensor_pose = gymapi.Transform()
    for name in feet_names:
        sensor_options = gymapi.ForceSensorProperties()
        sensor_options.enable_forward_dynamics_forces = False # for example gravity
        sensor_options.enable_constraint_solver_forces = True # for example contacts
        sensor_options.use_world_frame = True # report forces in world frame (easier to get vertical components)
        index = self.gym.find_asset_rigid_body_index(robot_asset, name)
        self.gym.create_asset_force_sensor(robot_asset, index, sensor_pose, sensor_options)
    (...)

    sensor_tensor = self.gym.acquire_force_sensor_tensor(self.sim)
    self.gym.refresh_force_sensor_tensor(self.sim)
    force_sensor_readings = gymtorch.wrap_tensor(sensor_tensor)
    self.sensor_forces = force_sensor_readings.view(self.num_envs, 4, 6)[..., :3]
    (...)

    self.gym.refresh_force_sensor_tensor(self.sim)
    contact = self.sensor_forces[:, :, 2] > 1.