code release

Author: lif314

.gitignore

@@ -26,6 +26,8 @@ share/python-wheels/
 *.egg
 MANIFEST
 
+logs/
+
 # PyInstaller
 #  Usually these files are written by a python script from a template
 #  before PyInstaller builds the exe, so as to inject date/other infos into it.

LICENSE

@@ -1,11 +1,22 @@
 <p align="center">
   <h1 align="center">
-    GS$^3$LAM: Gaussian Semantic Splatting SLAM
+    GS<sup>3</sup>LAM: Gaussian Semantic Splatting SLAM
     <br>
     [ACM MM 2024]
   </h1>
+  <p align="center">
+  <a href="https://github.com/lif314"><strong>Linfei Li</strong></a>
+  ·
+  <a href="https://scholar.google.com/citations?user=8VOk_S4AAAAJ&hl=en"><strong>Lin Zhang*</strong></a>
+  ·
+  <a href="https://scholar.google.com/citations?user=rrkp_usAAAAJ&hl=en"><strong>Zhong Wang</strong></a>
+  ·
+  <a href="https://scholar.google.com/citations?user=A0N_mS0AAAAJ&hl=en"><strong>Ying Shen</strong></a>
+</p>
 
-  <h3 align="center"><a href="https://github.com/lif314/GS3LAM">🌐Project page (Comming soon)</a> | <a href="https://github.com/lif314/GS3LAM">📝Paper (Comming soon)</a></h3>
+  <h3 align="center"><a href="https://github.com/lif314/GS3LAM">🌐Project page (comming soon)</a> 
+  | <a href="https://dl.acm.org/doi/10.1145/3664647.3680739">📝Paper(ACM DL)</a>
+  </h3>
   <div align="center"></div>
 </p>
 
@@ -15,7 +26,7 @@
   </a>
 </p>
 
-<p align="center">
+<!-- <p align="center">
   <a href="">
     <img src="./assets/splatting_rendering.gif" alt="splatting" width="100%">
   </a>
@@ -28,8 +39,175 @@
   <a href="">
     <img src="./assets/r0_pointslam_ours.gif" alt="r0_pointslam_ours" width="100%">
   </a>
-</p>
+</p> -->
+
+<!-- TABLE OF CONTENTS -->
+<details open="open" style='padding: 10px; border-radius:5px 30px 30px 5px; border-style: solid; border-width: 1px;'>
+  <summary>Table of Contents</summary>
+  <ol>
+    <li>
+      <a href="#installation">Installation</a>
+    </li>
+    <li>
+      <a href="#datasets">Datasets</a>
+    </li>
+    <li>
+      <a href="#benchmarking">Benchmarking</a>
+    </li>
+    <li>
+      <a href="#visualizer">Visualizer</a>
+    </li>
+    <li>
+      <a href="#acknowledgement">Acknowledgement</a>
+    </li>
+    <li>
+      <a href="#citation">Citation</a>
+    </li>
+  </ol>
+</details>
+
+## Installation
+
+The simplest way to install all dependences is to use [anaconda](https://www.anaconda.com/) and [pip](https://pypi.org/project/pip/) in the following steps: 
+
+```bash
+conda create -n gs3lam python==3.10
+conda activate gs3lam
+conda install -c "nvidia/label/cuda-11.7.0" cuda-toolkit
+pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
+pip install -r requirements.txt
+
+
+# install Gaussian Rasterization
+pip install submodules/gaussian-semantic-rasterization
+```
+
+## Datasets
+
+DATAROOT is `./data` by default. Please change the `basedir` path in the scene-specific config files if datasets are stored somewhere else on your machine.
+
+### Replica
+
+The original Replica dataset does not contain semantic labels. We obtained semantic labels from [vMAP](https://github.com/kxhit/vMAP). You can download our generated semantic Replica dataset from [here](https://huggingface.co/datasets/3David14/GS3LAM-Replica), then place the data into the `./data/Replica` folder.
+
+> Note, if you directly use the Replica dataset provided by vMAP, please modify the [Replica Dataloader](./src/datasets/replica.py) and the  [png_depth_scale](./configs/camera/replica.yaml) parameter in config files.
+
+### TUM-RGBD
+
+<!-- 由于TUM-RGBD没有真值语义标签，该数据集并不是我们的评估数据集。不过，为了测试我们方法的有效性，我们使用DEVA生成伪语义标签，您可以从这下载带有语义标签的TUM-RGBD。不幸的是，现有语义分割模型难以保证长序列数据的帧间语义一致性，因此我们只在fr1序列上进行了测试。 -->
+The TUM-RGBD dataset does not have ground truth semantic labels, so it is not our evaluation dataset. However, in order to evalute the effectiveness of GS3LAM, we use pseudo-semantic labels generated by [DEVA](https://github.com/hkchengrex/Tracking-Anything-with-DEVA), which you can download from [here](https://huggingface.co/datasets/3David14/TUM-DEVA). Unfortunately, existing semantic segmentation models struggle to maintain inter-frame semantic consistency in long sequence data, so we only tested on the `freiburg1_desk` sequence.
+
+### ScanNet
+
+Please follow the data downloading procedure on the [ScanNet](http://www.scan-net.org/) website, and extract color/depth frames from the `.sens` file using this [code](https://github.com/ScanNet/ScanNet/blob/master/SensReader/python/reader.py).
+
+<details>
+  <summary>[Directory structure of ScanNet (click to expand)]</summary>
+
+```
+  DATAROOT
+  └── scannet
+        └── scene0000_00
+            └── frames
+                ├── color
+                │   ├── 0.jpg
+                │   ├── 1.jpg
+                │   └── ...
+                ├── depth
+                │   ├── 0.png
+                │   ├── 1.png
+                │   └── ...
+                ├── label-filt
+                │   ├── 0.png
+                │   ├── 1.png
+                │   └── ...
+                ├── intrinsic
+                └── pose
+                    ├── 0.txt
+                    ├── 1.txt
+                    └── ...
+```
+</details>
+
+
+We use the following sequences: 
+```
+scene0000_00
+scene0059_00
+scene0106_00
+scene0169_00
+scene0181_00
+scene0207_00
+```
+
+## Benchmarking
+### TUM-RGBD
+
+To run GS3LAM on the `freiburg1_desk` scene, run the following command:
+
+```bash
+python run.py configs/Tum/tum_fr1.py
+```
+
+### Replica
+
+To run GS3LAM on the `office0` scene, run the following command:
+
+```bash
+python run.py configs/Replica/office0.py
+```
+
+To run GS3LAM on all Replica scenes, run the following command:
+
+```bash
+bash scripts/eval_full_replica.sh
+```
+
+### ScanNet
+
+To run GS3LAM on the `scene0059_00` scene, run the following command:
+
+```bash
+python run.py configs/Scannet/scene0059_00.py
+```
+
+To run GS3LAM on all ScanNet scenes, run the following command:
+
+```bash
+bash scripts/eval_full_scannet.bash
+```
+
+## Visualizer
+
+```
+TBD
+```
+
+## Acknowledgement
+We thank the authors of the following repositories for their open-source code:
+
+- [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting)
+- [SplaTAM](https://github.com/spla-tam)
+- [Gaussian-SLAM](https://github.com/VladimirYugay/Gaussian-SLAM)
+- [vMAP](https://github.com/kxhit/vMAP)
+- [Point-SLAM](https://github.com/eriksandstroem/Point-SLAM)
+- [Gaussian Grouping](https://github.com/lkeab/gaussian-grouping)
+
+## Citation
 
-# TODO
+If you find our paper and code useful for your research, please use the following BibTeX entry.
 
-- [ ] Code release.
+```bibtex
+@inproceedings{li2024gs3lam,
+      author = {Li, Linfei and Zhang, Lin and Wang, Zhong and Shen, Ying},
+      title = {GS3LAM: Gaussian Semantic Splatting SLAM},
+      year = {2024},
+      publisher = {Association for Computing Machinery},
+      address = {New York, NY, USA},
+      booktitle = {Proceedings of the 32nd ACM International Conference on Multimedia},
+      pages = {3019–3027},
+      numpages = {9},
+      location = {Melbourne VIC, Australia},
+      series = {MM '24}
+}
+```