HUSH: Holistic Panoramic 3D Scene Understanding using Spherical Harmonics

Jongsung Lee · Harin Park · Byeong-Uk Lee · Kyungdon Joo

This is official PyTorch implementation of "HUSH: Holistic Panoramic 3D Scene Understanding using Spherical Harmonics" (CVPR 2025)

Contents

1. Installation
2. Dataset Preparation
3. Train and Test
4. Acknowledgements
5. BibTeX

Installation

Our code is absed on CUDA 11.1 and PyTorch 1.10.1.

a. Download the source code:

git clone https://github.com/vision3d-lab/HUSH.git
cd HUSH

b. Create the conda environment and install required modules:

conda create -n hush python=3.8 -y
conda activate hush
pip install -r requirements.txt

c. Install the Deformation Attention:

• Here, we follow the instructions described at idisc.

cd models/ops
bash ./mask.sh

Dataset Preparation

Following the prior works, we used three benchmark datasets: Stanford2D3D, Matterport3D, and Structured3D.

a. Stanford2D3D Dataset

    We follow the data organization noted at Stanford2D3D.

b. Matterport3D Dataset

    We used the processed stitched skybox Matterport3D dataset.

Please refer the official repository and this issue for this step.

c. Structured3D Dataset

    We follow the data organization noted at Structured3D.

d. Layout Estimation (optional)

    For layout estimation, we need to pre-process the Matterport3D dataset to generate aligned panoramas.
    Official repository: MatterportLayout.

If you have problems during this process, this issue will be helpful.

Train and Test

• Train on the Matterport3D and Structured3D are also done similarly with train on the SF2D3D dataset.

python train_sf2d3d.py

• Test could be easily done by implementing validation functions and metric.py file.

Acknowledgements

This work is built on several great research works, thanks a lot to all the authors for sharing their works.

• AdaBins [CVPR 2021]
• iDisc [CVPR 2023]
• HRDFuse [CVPR 2023]
• torch-harmonics

BibTeX

@inproceedings{lee2025hush,
  title={HUSH: Holistic Panoramic 3D Scene Understanding using Spherical Harmonics},
  author={Lee, Jongsung and Park, Harin and Lee, Byeong-Uk and Joo, Kyungdon},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  pages={16599--16608},
  year={2025}
}