Skip to content

Latest commit

 

History

History
114 lines (74 loc) · 4.27 KB

README.md

File metadata and controls

114 lines (74 loc) · 4.27 KB

icon Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving

Custom badge Custom badge GitHub license

Yu Yang1,*, Jianbiao Mei1,*, Yukai Ma1, Siliang Du2,†, Wenqing Chen2, Yijie Qian1, Yuxiang Feng1, Yong Liu1,†
1 Zhejiang University 2 Huawei Technologies
* Equal Contribution † Corresponding Authors

📢 News

  • [2024-12-10] Our Drive-OccWorld is accepted by AAAI 2025 (Oral)!

  • [2024-10-14] We released our project website here.

  • [2024-08-26] The paper can be accessed at arxiv.

🎯 Abstract

pipeline

4D Occupancy Forecasting and Planning via World Model. Drive-OccWorld takes observations and trajectories as input, incorporating flexible action conditions for action-controllable generation. By leveraging world knowledge and the generative capacity of the world model, we further integrate it with a planner for continuous forecasting and planning.

📝 Getting Started

🎥 Demo of 4D Occupancy and Flow Forecasting

Drive-OccWorld understands how the world evolves by accurately modeling the dynamics of movable objects and the future states of the static environment.

Scene 1 (Lane Change)

Local GIF

Scene 2 (Pedestrian Crossing)

Local GIF

Scene 3 (Vehicle Following)

Local GIF

🚗 Demo of Continuous Forecasting and Planning (E2E Planning)

Drive-OccWorld plans trajectories through forecasting future occupancy state and selecting optimal trajectory based on a comprehensive occupancy-based cost function.

Scene 1 (Turn Left to Avoid Stopped Vehicle)

Local GIF

Scene 2 (Slowing Down to Wait for Crossing Pedestrians)

Local GIF

Scene 3 (Turn Right to Avoid Stopped Vehicle)

Local GIF

Acknowledgments

We utilized the following repos during development:

Thanks for their Awesome open-sourced work!

🔖 Citation

If you find our project useful, please kindly cite us via:

@article{yang2024driving,
  title={Driving in the occupancy world: Vision-centric 4d occupancy forecasting and planning via world models for autonomous driving},
  author={Yang, Yu and Mei, Jianbiao and Ma, Yukai and Du, Siliang and Chen, Wenqing and Qian, Yijie and Feng, Yuxiang and Liu, Yong},
  journal={arXiv preprint arXiv:2408.14197},
  year={2024}
}