Animatable avatars for MuJoCo models in Unity

[Balint-H]

To be able to apply animations in the Unity plugin, or exploit all the nice rigging methods implemented in the engine for kinematic controllers, a model needs to be imported in an FBX format. To have a 1:1 equivalence with the MuJoCo model, it is important that they have matching local positions of body parts. To achieve this you can remove all Mj components from your humanoid model, and can then use the FBX exporter package in unity to create an animatable rig, which you can reimport.

During the import process, if its a humanoid rig, you can select it as such. In this case you will probably have to edit the skeleton, and reassign a few of the transforms to the correct body-parts. Once done, you can track the individual transforms with MjMocapBodies to get their information to the simulation (or to connect regular bodies to them with constraints).

Here's an example I've been working with for the humanoid model from (Merel et al. 2017):
humanoid_CMU.zip

If someone has a recommendation on how to combine the individual meshes into a single weighted mesh for neater material assignment that doesn't involve manually recreating the skeleton and painting the vertex weights, let me know. For the time being I've been using an editor script to apply a material to all meshrenderers of the animated model.

It's also possible to do it with the traditional MuJoCo humanoid (e.g. Tassa et al. 2012), but for humanoid rigs in Unity it is important to have the character in a T-pose. To achieve this, you can modify the local transform of body segments, but don't offset the Geom objects themselves.