Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions docs/getting_started_controllers.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Robots that we tested are:

- **FR3** (real and simulated): https://github.com/danielsanjosepro/pixi_franka_ros2
- **FER/Panda** (real): https://github.com/lvjonok/pixi_panda_ros2
- **UR5** (real): https://github.com/danielsanjosepro/crisp_ur_demo
- **IIWA** (simulated): https://github.com/danielsanjosepro/pixi_iiwa_ros2

### Grippers
Expand Down Expand Up @@ -54,12 +55,13 @@ Some other manipulators that could be added to this list is [Duatic](https://git

### Available Demos

| Robots | Franka Robotics FR3 | FR Dual FR3 | IIWA 14 | Kinova Gen3 |
| :--- | :---: | :---: | :---: | :---: |
| MuJoCo simulated Hardware | ✅ | ✅ | ✅ | ✅ |
| Real Hardware | ✅ | ✅ | ❔[^1] | ❔[^1] |
| Robots | Franka Robotics FR3 | FR Dual FR3 | IIWA 14 | Kinova Gen3 | UR5 |
| :--- | :---: | :---: | :---: | :---: | :---: |
| MuJoCo simulated Hardware | ✅ | ✅ | ✅ | ✅ | ✅ [^2]|
| Real Hardware | ✅ | ✅ | ❔[^1] | ❔[^1] | ✅ [^2] |

[^1]: Untested, but effort interface available.
[^2]: Available at: https://github.com/danielsanjosepro/crisp_ur_demo

We also have some examples with cameras.

Expand Down
90 changes: 78 additions & 12 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,21 @@ hide:
<img src="media/crisp_logo_dark.webp#only-dark" alt="CRISP Controllers Logo"/>
<img src="media/crisp_logo_light.webp#only-light" alt="CRISP Controllers Logo"/>



# CRISP - **C**ompliant **R**OS2 Controllers for Learn**i**ng-Ba**s**ed Manipulation **P**olicies

*Authors: [Daniel San Jose Pro](https://danielsanjosepro.github.io)[^1], [Oliver Hausdörfer](https://oliver.hausdoerfer.de/)[^1], [Ralf Römer](https://ralfroemer99.github.io)[^1], Maximilian Dösch[^1], [Martin Schuck](https://amacati.github.io/) [^1] and Angela Schoellig [^1]*.
[^1]: The authors are with Technical University of Munich, Germany; TUM School of Computation, Information and Technology, Department of Computer Engineering, Learning Systems and Robotics Lab; Munich Institute of Robotics and Machine Intelligence.

> A collection of real-time, C++ controllers for compliant torque-based control for manipulators compatible with `ros2_control`. Developed for deploying high-level learning-based policies (VLA, Diffusion, ...) and teleoperation on your manipulator. It is compatible with any manipulator offering an effort interface.
You want to deploy your learning-based policy to your manipulator, or collect teleoperation data? `CRISP` provides the tools for that.

- `CRISP` provides the `ros2_control` low-level controllers (compliant, real-time, C++, torque-based) and simple python interfaces `CRISP_PY` to interface with them. It is compatible with any manipulator offering an joint-level torque effort interface.

- `CRISP` provides also a Gymnasium environment `CRISP_GYM` to deploy learning-based `LeRobot` policies and collect data in `LeRobotDataset` format.

_If you use this work, please cite it using the [bibtex](#citing) below._

Check the [controllers (CRISP controllers) :simple-github:](https://github.com/utiasDSL/crisp_controllers) , robot [demos (CRISP controllers demos) :simple-github:](https://github.com/utiasDSL/crisp_controllers_demos), a simple [python interface (CRISP_PY) :simple-github:](https://github.com/utiasDSL/crisp_py), and a [Gymnasium wrapper (CRISP_GYM) :simple-github:](https://github.com/utiasDSL/crisp_gym) for real-world experiments.
Check the [controllers (CRISP controllers) :simple-github:](https://github.com/utiasDSL/crisp_controllers), the simple [python interface (CRISP_PY) :simple-github:](https://github.com/utiasDSL/crisp_py), and a [Gymnasium wrapper (CRISP_GYM) :simple-github:](https://github.com/utiasDSL/crisp_gym) for real-world experiments.


!!! info "Aloha gripper for Manipulators"
Check out [aloha4franka](https://tum-lsy.github.io/aloha4franka/) for the gripper used in the videos.
Expand All @@ -32,31 +35,31 @@ Check the [controllers (CRISP controllers) :simple-github:](https://github.com/u
<div class="carousel-container">
<div class="carousel">
<div class="carousel-slide active">
<video src="media/videos_concatenaded.mp4" playsinline muted loop autoplay alt="Robot Collection"></video>
<video src="../media/videos_concatenaded.mp4" playsinline muted loop autoplay alt="Robot Collection"></video>
<div class="carousel-caption">End-to-end policies in action performing different contact-rich tasks.</div>
</div>
<div class="carousel-slide">
<video src="media/wrist_cam.mp4" playsinline muted loop autoplay alt="Data visualized"></video>
<video src="../media/wrist_cam.mp4" playsinline muted loop autoplay alt="Data visualized"></video>
<div class="carousel-caption">Recorded Trajectories in LeRobot format for different tasks using CRISP.</div>
</div>
<div class="carousel-slide">
<video src="media/viser_visualization.mp4" playsinline muted loop autoplay alt="Viser"></video>
<video src="../media/viser_visualization.mp4" playsinline muted loop autoplay alt="Viser"></video>
<div class="carousel-caption">Provided example with 1x video of the tracking capabilities of the controller.</div>
</div>
<div class="carousel-slide">
<video src="media/pap_demo.mp4" playsinline muted loop autoplay alt="Teleoperation PaP"></video>
<video src="../media/pap_demo.mp4" playsinline muted loop autoplay alt="Teleoperation PaP"></video>
<div class="carousel-caption">Leader-follower teleoperation for a pick and place task.</div>
</div>
<div class="carousel-slide">
<img src="media/franka.gif" alt="Franka Robot Demo">
<img src="../media/franka.gif" alt="Franka Robot Demo">
<div class="carousel-caption">Controllers: Franka Robot - Following a moving target while base joint follows a sine curve</div>
</div>
<div class="carousel-slide">
<img src="media/kinova.gif" alt="Kinova Robot Demo">
<img src="../media/kinova.gif" alt="Kinova Robot Demo">
<div class="carousel-caption">Controllers: Kinova Robot - Simulated robot with continuous joints and nullspace control</div>
</div>
<div class="carousel-slide">
<img src="media/iiwa.gif" alt="IIWA Robot Demo">
<img src="../media/iiwa.gif" alt="IIWA Robot Demo">
<div class="carousel-caption">Controllers: IIWA Robot - Impedance control in simulation</div>
</div>
</div>
Expand Down Expand Up @@ -98,10 +101,73 @@ While `ROS2` frameworks like `MoveIt` offer comprehensive motion planning capabi
We present a set of lightweight, torque-based Cartesian and joint-space controllers implemented in C++ for `ros2_control`, compatible with any robot exposing an effort interface—a common standard among modern manipulators.
Our controllers incorporate friction compensation, joint limit avoidance, and error clipping, and have been validated on the Franka Robotics FR3 on hardware, and on various platforms in simulation.

Designed for fast integration and real-time control, our implementation lowers the barrier to deploying learning-based algorithms on `ROS2`-compatible platforms.
We provide tooling to collect data in `LeRobotDataset` format using teleoperation and deploy learning-based policies with minimal effort using `CRISP_PY` and `CRISP_GYM`.

**Why the name "CRISP"**? "CRISP" reflects our design philosophy behind the package: a concise, to-the-point implementation for easy deployment and integration in other software stacks.

## Real World Deployments

Our controllers have been tested in real hardware on the following robots:
<div class="robot-grid">
<div class="robot-card">
<a href="https://github.com/danielsanjosepro/pixi_franka_ros2">
<img src="media/fr3_visual.png" alt="FR3">
<div class="robot-name">FR3</div>
</a>
</div>
<div class="robot-card">
<a href="https://github.com/lvjonok/pixi_panda_ros2">
<img src="media/panda_visual.png" alt="Panda">
<div class="robot-name">Panda</div>
</a>
</div>

<div class="robot-card">
<a href="https://github.com/danielsanjosepro/crisp_ur_demo">
<img src="media/ur10_visual.png" alt="UR5">
<div class="robot-name">UR5</div>
</a>
</div>
<!-- <div class="robot-card greyscale"> -->
<!-- <img src="media/dynaarm_visual.png" alt="UR10"> -->
<!-- <div class="robot-name">DynaArm</div> -->
<!-- </div> -->
</div>

<!---->
<!-- ... and in simulation on: -->
<!-- <div class="robot-grid"> -->
<!-- <div class="robot-card"> -->
<!-- <img src="media/iiwa14_visual.png" alt="IIWA"> -->
<!-- <div class="robot-name">IIWA14</div> -->
<!-- </div> -->
<!-- <div class="robot-card greyscale"> -->
<!-- <img src="media/kinova_visual.png" alt=""> -->
<!-- <div class="robot-name">Kinova</div> -->
<!-- </div> -->
<!-- </div> -->
<!---->

Many thanks community contributions:

- Lev Kozlov [@lvjonok](https://github.com/lvjonok) for testing and providing interfaces for the Panda/FER.
- Vincenzo Orlando [@VinsOrl09](https://github.com/lvjonok) for testing and providing interfaces for the UR robots.

Check the robots in action:

<div class="video-container">
<div class="video-item">
<video src="../media/crisp_ur5_test.mp4" playsinline muted loop autoplay alt="Data visualized"></video>
<div class="hover-text">UR5 Robot using a CRISP controller to track a circular trajectory while being disturbed.</div>
</div>
<div class="video-item">
<video src="../media/crisp_fr3_test.mp4" playsinline muted loop autoplay alt="Data visualized"></video>
<div class="hover-text">End-To-End policy stacking Lego bricks with the CRISP controllers.</div>
</div>
</div>



## Features
- 🐍 **Python interface** to move your ROS2 robot around without having to think about topics, spinning, and more ROS2 concepts but without losing the powerful ROS2 API. Check [CRISP_PY](https://github.com/utiasDSL/crisp_py) for more information and examples.
- 🔁 **Gymnasium environment** with utilities to deploy learning-based policies and record trajectories in LeRobotFormat. Check [CRISP_GYM](https://github.com/utiasDSL/crisp_gym).
Expand Down
Binary file added docs/media/CRISP_logo_with_robot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/crisp_fr3_test.mp4
Binary file not shown.
Binary file added docs/media/crisp_ur5_test.mp4
Binary file not shown.
Binary file added docs/media/dynaarm_visual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/fr3_visual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/iiwa14_visual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/panda_visual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/robotiq_visual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/ur10_visual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/ur5_test.mp4
Binary file not shown.
78 changes: 77 additions & 1 deletion docs/stylesheets/extra.css
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,38 @@
justify-content: center;
border-radius: 12px;
box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1);
margin: 1rem 0.2rem;
}

.video-item video {
width: 100%;
height: auto;
border-radius: 12px;
background-color: rgba(0, 0, 0, 0.0);
transition: filter 0.3s ease;
}

.video-item:hover video {
filter: grayscale(100%) brightness(0.5);
}

.video-item .hover-text {
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
color: white;
font-size: 0.8rem;
font-weight: 300;
text-align: center;
padding: 0.1rem;
opacity: 0;
transition: opacity 0.3s ease;
pointer-events: none;
}

.video-item:hover .hover-text {
opacity: 1;
}

.highlight .vc {
Expand Down Expand Up @@ -229,7 +254,7 @@
}

.carousel-caption {
font-size: 0.85rem;
font-size: 0.25rem;
padding: 0.8rem 1rem;
}

Expand All @@ -241,3 +266,54 @@
right: 0.5rem;
}
}

/* Robot Grid Styles */
.robot-grid {
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 1rem;
margin: 1rem 0;
}

.robot-card {
position: relative;
overflow: hidden;
border-radius: 8px;
border: 1px solid #ddd;
}

.robot-card img {
width: 100%;
height: auto;
display: block;
transition: transform 0.3s ease, filter 0.3s ease;
}

.robot-card:hover img {
transform: scale(1.2);
}

.robot-card.greyscale img {
filter: grayscale(100%);
}

.robot-card.greyscale:hover img {
filter: grayscale(0%);
}

.robot-card .robot-name {
position: absolute;
bottom: 0;
left: 0;
right: 0;
background: rgba(0, 0, 0, 0.7);
color: white;
text-align: center;
padding: 0.5rem;
opacity: 0;
transition: opacity 0.3s ease;
}

.robot-card:hover .robot-name {
opacity: 1;
}