Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
# *EmbodiedGen*: Towards a Generative 3D World Engine for Embodied Intelligence

[![🌐 Project Page](https://img.shields.io/badge/🌐-Project_Page-blue)](https://horizonrobotics.github.io/robot_lab/embodied_gen/index.html)
[![📖 Documentation](https://img.shields.io/badge/📖-Documentation-blue)](https://horizonrobotics.github.io/EmbodiedGen/)
[![📄 arXiv](https://img.shields.io/badge/📄-arXiv-b31b1b)](https://arxiv.org/abs/2506.10600)
[![🎥 Video](https://img.shields.io/badge/🎥-Video-red)](https://www.youtube.com/watch?v=rG4odybuJRk)
[![中文介绍](https://img.shields.io/badge/中文介绍-07C160?logo=wechat&logoColor=white)](https://mp.weixin.qq.com/s/HH1cPBhK2xcDbyCK4BBTbw)
<!-- [![🌐 Project Page](https://img.shields.io/badge/🌐-Project_Page-blue)](https://horizonrobotics.github.io/robot_lab/embodied_gen/index.html) -->
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-EmbodiedGen_Asset_Gallery-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Gallery-Explorer)
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Image_to_3D_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Image-to-3D)
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Text_to_3D_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Text-to-3D)
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Texture_Gen_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Texture-Gen)
[![中文介绍](https://img.shields.io/badge/中文介绍-07C160?logo=wechat&logoColor=white)](https://mp.weixin.qq.com/s/HH1cPBhK2xcDbyCK4BBTbw)


[![🤗 Hugging Face](https://img.shields.io/badge/🤗-EmbodiedGen_Asset_Gallery-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Gallery-Explorer)


> ***EmbodiedGen*** is a generative engine to create diverse and interactive 3D worlds composed of high-quality 3D assets(mesh & 3DGS) with plausible physics, leveraging generative AI to address the challenges of generalization in embodied intelligence related research.
> It composed of six key modules: `Image-to-3D`, `Text-to-3D`, `Texture Generation`, `Articulated Object Generation`, `Scene Generation` and `Layout Generation`.
Expand All @@ -20,6 +18,8 @@
---

## ✨ Table of Contents of EmbodiedGen
[![📖 Documentation](https://img.shields.io/badge/📖-Documentation-blue)](https://horizonrobotics.github.io/EmbodiedGen/) Follow the documentation to get started!

- [🖼️ Image-to-3D](#image-to-3d)
- [📝 Text-to-3D](#text-to-3d)
- [🎨 Texture Generation](#texture-generation)
Expand All @@ -30,6 +30,8 @@

## 🚀 Quick Start

[![📖 Documentation](https://img.shields.io/badge/📖-Documentation-blue)](https://horizonrobotics.github.io/EmbodiedGen/)

### ✅ Setup Environment
```sh
git clone https://github.com/HorizonRobotics/EmbodiedGen.git
Expand Down Expand Up @@ -67,7 +69,7 @@ You can choose between two backends for the GPT agent:

### 📸 Directly use EmbodiedGen All-Simulators-Ready Assets

Explore EmbodiedGen generated assets in [![🤗 Hugging Face](https://img.shields.io/badge/🤗-EmbodiedGen_Asset_Gallery-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Gallery-Explorer) that are ready for simulation across any simulators (SAPIEN, Isaac Sim, MuJoCo, PyBullet, Genesis, Isaac Gym etc.). Details in chapter [any-simulators](#any-simulators).
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-EmbodiedGen_Asset_Gallery-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Gallery-Explorer) Explore EmbodiedGen generated assets that are ready for simulation across any simulators (SAPIEN, Isaac Sim, MuJoCo, PyBullet, Genesis, Isaac Gym etc.). Details in chapter [any-simulators](#any-simulators).

---

Expand Down
7 changes: 4 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,15 @@ hide:

# 👋 Welcome to EmbodiedGen

[![🌐 Project Page](https://img.shields.io/badge/🌐-Project_Page-blue)](https://horizonrobotics.github.io/robot_lab/embodied_gen/index.html)
[![📖 Documentation](https://img.shields.io/badge/📖-Documentation-blue)](https://horizonrobotics.github.io/EmbodiedGen/)
[![📄 arXiv](https://img.shields.io/badge/📄-arXiv-b31b1b)](https://arxiv.org/abs/2506.10600)
[![🎥 Video](https://img.shields.io/badge/🎥-Video-red)](https://www.youtube.com/watch?v=rG4odybuJRk)
[![中文介绍](https://img.shields.io/badge/中文介绍-07C160?logo=wechat&logoColor=white)](https://mp.weixin.qq.com/s/HH1cPBhK2xcDbyCK4BBTbw)
<!-- [![🌐 Project Page](https://img.shields.io/badge/🌐-Project_Page-blue)](https://horizonrobotics.github.io/robot_lab/embodied_gen/index.html) -->
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-EmbodiedGen_Asset_Gallery-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Gallery-Explorer)
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Image_to_3D_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Image-to-3D)
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Text_to_3D_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Text-to-3D)
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Texture_Gen_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Texture-Gen)
[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Asset_Gallery-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Gallery-Explorer)
[![中文介绍](https://img.shields.io/badge/中文介绍-07C160?logo=wechat&logoColor=white)](https://mp.weixin.qq.com/s/HH1cPBhK2xcDbyCK4BBTbw)

*EmbodiedGen*: Towards a Generative 3D World Engine for Embodied Intelligence.

Expand Down
10 changes: 10 additions & 0 deletions docs/services/image_to_3d.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@
noSwipingClass: 'swiper-no-swiping',
watchSlidesProgress: true,
});
const modelViewers = document.querySelectorAll('model-viewer');
let loadedCount = 0;
modelViewers.forEach(mv => {
mv.addEventListener('load', () => {
loadedCount++;
if (loadedCount === modelViewers.length) {
swiper.update();
}
});
});
});
</script>

Expand Down
10 changes: 10 additions & 0 deletions docs/services/text_to_3d.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@
noSwipingClass: 'swiper-no-swiping',
watchSlidesProgress: true,
});
const modelViewers = document.querySelectorAll('model-viewer');
let loadedCount = 0;
modelViewers.forEach(mv => {
mv.addEventListener('load', () => {
loadedCount++;
if (loadedCount === modelViewers.length) {
swiper.update();
}
});
});
});
</script>

Expand Down
11 changes: 11 additions & 0 deletions docs/services/texture_edit.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,20 @@
noSwipingClass: 'swiper-no-swiping',
watchSlidesProgress: true,
});
const modelViewers = document.querySelectorAll('model-viewer');
let loadedCount = 0;
modelViewers.forEach(mv => {
mv.addEventListener('load', () => {
loadedCount++;
if (loadedCount === modelViewers.length) {
swiper.update();
}
});
});
});
</script>


# 🎨 Texture Generation Service

[![🤗 Hugging Face](https://img.shields.io/badge/🤗-Texture_Gen_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Texture-Gen)
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/any_simulators.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,6 @@ dst_asset_path = cvt_embodiedgen_asset_to_anysim(
)
```

<img src="assets/simulators_collision.jpg" alt="simulators_collision" width="800">
<img src="/EmbodiedGen/assets/simulators_collision.jpg" alt="simulators_collision" width="800">

Collision and visualization mesh across simulators, showing consistent geometry and material fidelity.
2 changes: 1 addition & 1 deletion docs/tutorials/digital_twin.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Real-to-Sim Digital Twin Creation

<img src="assets/real2sim_mujoco.gif" alt="real2sim_mujoco" width="600">
<img src="/EmbodiedGen/assets/real2sim_mujoco.gif" alt="real2sim_mujoco" width="600">
4 changes: 2 additions & 2 deletions docs/tutorials/gym_env.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ python embodied_gen/scripts/parallel_sim.py \
```

<div style="display: flex; justify-content: center; align-items: center; gap: 16px; margin: 16px 0;">
<img src="assets/parallel_sim.gif" alt="parallel_sim1"
<img src="/EmbodiedGen/assets/parallel_sim.gif" alt="parallel_sim1"
style="width: 330px; max-width: 100%; border-radius: 12px; display: block;">
<img src="assets/parallel_sim2.gif" alt="parallel_sim2"
<img src="/EmbodiedGen/assets/parallel_sim2.gif" alt="parallel_sim2"
style="width: 330px; max-width: 100%; border-radius: 12px; display: block;">
</div>

10 changes: 10 additions & 0 deletions docs/tutorials/image_to_3d.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@
noSwipingClass: 'swiper-no-swiping',
watchSlidesProgress: true,
});
const modelViewers = document.querySelectorAll('model-viewer');
let loadedCount = 0;
modelViewers.forEach(mv => {
mv.addEventListener('load', () => {
loadedCount++;
if (loadedCount === modelViewers.length) {
swiper.update();
}
});
});
});
</script>

Expand Down
29 changes: 20 additions & 9 deletions docs/tutorials/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,20 @@
noSwipingClass: 'swiper-no-swiping',
watchSlidesProgress: true,
});
const modelViewers = document.querySelectorAll('model-viewer');
let loadedCount = 0;
modelViewers.forEach(mv => {
mv.addEventListener('load', () => {
loadedCount++;
if (loadedCount === modelViewers.length) {
swiper.update();
}
});
});
});
</script>


# Tutorials & Interface Usage

Welcome to the tutorials for `EmbodiedGen`. `EmbodiedGen` is a powerful toolset for generating 3D assets, textures, scenes, and interactive layouts ready for simulators and digital twin environments.
Expand Down Expand Up @@ -150,7 +161,7 @@ Generate **high-quality textures** for 3D meshes using **text prompts**, support

Generate **physically consistent and visually coherent 3D environments** from text prompts. Typically used as **background** 3DGS scenes in simulators for efficient and photo-realistic rendering.

<img src="assets/scene3d.gif" style="width: 500px; max-width: 100%; border-radius: 12px; display: block; margin: 16px auto;">
<img src="/EmbodiedGen/scene3d.gif" style="width: 500px; max-width: 100%; border-radius: 12px; display: block; margin: 16px auto;">

---

Expand All @@ -159,10 +170,10 @@ Generate **physically consistent and visually coherent 3D environments** from te
Generate diverse, physically realistic, and scalable **interactive 3D scenes** from natural language task descriptions, while also modeling the robot and manipulable objects.

<div align="center" style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 16px; justify-items: center; margin: 20px 0;">
<img src="assets/layout1.gif" alt="layout1" style="width: 400px; border-radius: 12px; display: block;">
<img src="assets/layout2.gif" alt="layout2" style="width: 400px; border-radius: 12px; display: block;">
<img src="assets/layout3.gif" alt="layout3" style="width: 400px; border-radius: 12px; display: block;">
<img src="assets/Iscene_demo2.gif" alt="layout4" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/layout1.gif" alt="layout1" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/layout2.gif" alt="layout2" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/layout3.gif" alt="layout3" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/Iscene_demo2.gif" alt="layout4" style="width: 400px; border-radius: 12px; display: block;">
</div>


Expand All @@ -173,9 +184,9 @@ Generate diverse, physically realistic, and scalable **interactive 3D scenes** f
Generate multiple **parallel simulation environments** with `gym.make` and record sensor and trajectory data.

<div style="display: flex; justify-content: center; align-items: center; gap: 16px; margin: 16px 0;">
<img src="assets/parallel_sim.gif" alt="parallel_sim1"
<img src="/EmbodiedGen/parallel_sim.gif" alt="parallel_sim1"
style="width: 330px; max-width: 100%; border-radius: 12px; display: block;">
<img src="assets/parallel_sim2.gif" alt="parallel_sim2"
<img src="/EmbodiedGen/parallel_sim2.gif" alt="parallel_sim2"
style="width: 330px; max-width: 100%; border-radius: 12px; display: block;">
</div>

Expand All @@ -187,11 +198,11 @@ Generate multiple **parallel simulation environments** with `gym.make` and recor
Seamlessly use EmbodiedGen-generated assets in major simulators like **IsaacSim**, **MuJoCo**, **Genesis**, **PyBullet**, **IsaacGym**, and **SAPIEN**, featuring **accurate physical collisions** and **consistent visual appearance**.

<div align="center">
<img src="assets/simulators_collision.jpg" alt="simulators_collision" style="width: 400px; max-width: 100%; border-radius: 12px; display: block; margin: 16px 0;">
<img src="/EmbodiedGen/simulators_collision.jpg" alt="simulators_collision" style="width: 400px; max-width: 100%; border-radius: 12px; display: block; margin: 16px 0;">
</div>

## [🔧 Real-to-Sim Digital Twin Creation](digital_twin.md)

<div align="center">
<img src="assets/real2sim_mujoco.gif" alt="real2sim_mujoco" style="width: 400px; max-width: 100%; border-radius: 12px; display: block; margin: 16px 0;">
<img src="/EmbodiedGen/real2sim_mujoco.gif" alt="real2sim_mujoco" style="width: 400px; max-width: 100%; border-radius: 12px; display: block; margin: 16px 0;">
</div>
12 changes: 6 additions & 6 deletions docs/tutorials/layout_gen.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
Layout Generation enables the generation of diverse, physically realistic, and scalable **interactive 3D scenes** directly from natural language task descriptions, while also modeling the robot's pose and relationships with manipulable objects. Target objects are randomly placed within the robot's reachable range, making the scenes readily usable for downstream simulation and reinforcement learning tasks in any mainstream simulator.

<div align="center" style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 16px; justify-items: center; margin: 20px 0;">
<img src="assets/layout1.gif" alt="layout1" style="width: 400px; border-radius: 12px; display: block;">
<img src="assets/layout2.gif" alt="layout2" style="width: 400px; border-radius: 12px; display: block;">
<img src="assets/layout3.gif" alt="layout3" style="width: 400px; border-radius: 12px; display: block;">
<img src="assets/Iscene_demo2.gif" alt="layout4" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/assets/layout1.gif" alt="layout1" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/assets/layout2.gif" alt="layout2" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/assets/layout3.gif" alt="layout3" style="width: 400px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/assets/Iscene_demo2.gif" alt="layout4" style="width: 400px; border-radius: 12px; display: block;">
</div>

!!! note "Model Requirement"
Expand Down Expand Up @@ -44,8 +44,8 @@ layout-cli \

You will get the following results:
<div align="center" style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 16px; justify-items: center; margin: 20px 0;">
<img src="assets/Iscene_demo1.gif" alt="Iscene_demo1" style="width: 320px; border-radius: 12px; display: block;">
<img src="assets/Iscene_demo2.gif" alt="Iscene_demo2" style="width: 320px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/assets/Iscene_demo1.gif" alt="Iscene_demo1" style="width: 240px; border-radius: 12px; display: block;">
<img src="/EmbodiedGen/assets/Iscene_demo2.gif" alt="Iscene_demo2" style="width: 480px; border-radius: 12px; display: block;">
</div>


Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/scene_gen.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generate **physically consistent and visually coherent 3D environments** from te

---

<img src="assets/scene3d.gif" style="width: 600px; border-radius: 12px; display: block; margin: 16px auto;">
<img src="/EmbodiedGen/assets/scene3d.gif" style="width: 600px; border-radius: 12px; display: block; margin: 16px auto;">

---

Expand Down
10 changes: 10 additions & 0 deletions docs/tutorials/text_to_3d.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@
noSwipingClass: 'swiper-no-swiping',
watchSlidesProgress: true,
});
const modelViewers = document.querySelectorAll('model-viewer');
let loadedCount = 0;
modelViewers.forEach(mv => {
mv.addEventListener('load', () => {
loadedCount++;
if (loadedCount === modelViewers.length) {
swiper.update();
}
});
});
});
</script>

Expand Down
11 changes: 11 additions & 0 deletions docs/tutorials/texture_edit.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,20 @@
noSwipingClass: 'swiper-no-swiping',
watchSlidesProgress: true,
});
const modelViewers = document.querySelectorAll('model-viewer');
let loadedCount = 0;
modelViewers.forEach(mv => {
mv.addEventListener('load', () => {
loadedCount++;
if (loadedCount === modelViewers.length) {
swiper.update();
}
});
});
});
</script>


# 🎨 Texture Generation: Create Visually Rich Textures for 3D Meshes

Generate **high-quality textures** for 3D meshes using **text prompts**, supporting both **Chinese and English**. This allows you to enhance the visual appearance of existing 3D assets for simulation, visualization, or digital twin applications.
Expand Down