State Space Models for Sparse Geometric Data

Figure 1: A. An event stream, B. The STREAM module converts relative differences in coordinates into the $\Delta_k$ scale of the SSM, C. A STREAM module integrates pairwise spatial relationships based on an exponentially oscillating kernel.

This repository contains code for our paper State Space Models for Sparse Geometric Data. It is an extension of our previous work to include the Mamba model, which allows our system to have a larger state size and input-dependent parameters. We call our model STREAM (Spatio-Temporal Recursive Encoding of Asynchronous Modalities). This work was accepted to the 2nd Workshop on Neuromorphic Vision at ICCV 2025.

Reproducing our results

We provide code to reproduce the neuromorphic data results in the neuromorphic/ directory and point-cloud results in the point-clouds/ directory. Please refer to the respective README.md files in these directories for instructions on how to run the code.

Citation

Please use the following when citing our work:

@misc{schöne2024streamuniversalstatespacemodel,
      title={STREAM: A Universal State-Space Model for Sparse Geometric Data}, 
      author={Mark Schöne and Yash Bhisikar and Karan Bania and Khaleelulla Khan Nazeer and Christian Mayr and Anand Subramoney and David Kappel},
      year={2024},
      eprint={2411.12603},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2411.12603}, 
}