[CVPR2025] VoxelSplat: Dynamic Gaussian Splatting as an Effective Loss for Occupancy and Flow Prediction
Ziyue Zhu · Shenlong Wang · Jin Xie · Jiang-jiang Liu · Jingdong Wang · Jian Yang
VoxelSplat is a novel regularization framework that leverages dynamic 3D Gaussian Splatting to improve the prediction of occupancy and scene flow.
Recent advances in camera-based occupancy prediction aim to jointly estimate 3D semantics and scene flow. We propose VoxelSplat, a novel regularization framework that leverages 3D Gaussian Splatting to improve learning in two key ways:
(i) 2D-Projected Semantic Supervision: During training, sparse semantic Gaussians decoded from 3D features are projected onto the 2D camera view, enabling camera-visible supervision to guide 3D semantic learning.
(ii) Enhanced Scene Flow Learning: Motion is modeled by propagating Gaussians with predicted scene flow, allowing enhanced flow learning from adjacent-frame labels.
VoxelSplat integrates easily into existing occupancy models, improving both semantic and motion predictions without increasing inference time.
- [2025/7]: Code and pre-trained weights are released.
- [2025/3]: Paper is accepted on CVPR 2025.
| Backbone | Config | Image Size | Epochs | Train Pretrain | Memory | RayIoU | mAVE | checkpoints |
|---|---|---|---|---|---|---|---|---|
| R50 | FB-Occ (Baseline) | 256 x 704 | 48 | ImageNet | 17 G | 33.57 | 0.504 | [model] |
| R50 | voxelsplat-r50 | 256 x 704 | 48 | ImageNet | 19 G | 37.14 | 0.312 | [model] |
| EVA-VIT | voxelsplat-eva | 640 x 1600 | 24 | ImageNet | 28 G | - | - | [model] |
| Intern-XL | voxelsplat-intern | 640 x 1600 | 24 | Nus-Det | 39 G | - | - | [model] |
If you find our paper and code useful for your research, please consider citing:
@inproceedings{zhu2025voxelsplat,
title={Voxelsplat: Dynamic gaussian splatting as an effective loss for occupancy and flow prediction},
author={Zhu, Ziyue and Wang, Shenlong and Xie, Jin and Liu, Jiang-jiang and Wang, Jingdong and Yang, Jian},
booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
pages={6761--6771},
year={2025}
}