This repository contains the files required to simulate, train, and deploy a reinforcement learning–based compliant controller for the manipulation of objects with varying masses using a dual-arm FR3 robot system.
The controller is trained in simulation and later transferred to the real robot platform.
The research associated with this project is described in detail in the following paper:
(Add paper link here)
The main folders of this repository are described below.
This section contains the MuJoCo XML models of the dual-arm FR3 robot used in the simulations.
The model parameters armature, friction loss, and damping were optimized for each joint in order to obtain a simulation that closely matches the real robot dynamics in MuJoCo.
The initial robot model was obtained from the URDF released by Franka Emika:
https://github.com/frankarobotics/franka_description
This model can be directly simulated in MuJoCo.
It was generated by converting the original URDF into a MuJoCo XML model and subsequently optimizing the following parameters for each joint:
- armature
- friction loss
- damping
The end-effector used in this model corresponds to the default gripper provided with the FR3 robot arms.
This model uses the same optimized dynamic parameters as the franka_fr3_dual model.
The main difference is the end-effector configuration. Instead of the default gripper, this model uses the custom end-effector used in the experiments, which was designed to manipulate objects with different masses.
This section contains the files used during the reinforcement learning training stage.
It includes:
- trained agents
- the simulation environment
- the training scripts
This folder contains trained policies obtained using the Soft Actor-Critic (SAC) algorithm.
The models differ depending on the reward function terms used during training.
This Jupyter notebook defines the simulation environment used to train the reinforcement learning agent.
The environment follows the interface of Gymnasium:
This notebook allows the visualization of intermediate policies during training.
Running this notebook with a separate kernel makes it possible to monitor the agent's behavior while training is still running.
This notebook contains the training pipeline used to train the reinforcement learning controller.
Training is performed using Stable-Baselines3:
https://stable-baselines3.readthedocs.io/
This section contains the final implementation used to perform the experiments on the real dual-arm FR3 robot.
The code controls both robot arms simultaneously, therefore the control computer must have access to both robots.
For simplicity, it is recommended to:
- connect both FR3 robots to a network switch
- connect the switch to the control computer
The robot control is implemented using the official C++ library libfranka:
https://github.com/frankarobotics/libfranka
The policy trained in simulation is exported and executed in C++ using ONNX Runtime: