mmdetection3D - Run LIDAR inference and generate confusion amtrix on NuScenes dataset

[ranai-srivastav]

I am attempting to create a confusion matrix by running LIDAR inference on the NuScenes Dataset. What's the best way of doing so?

Things I have tried

• I was successful in running MMDetection3D on LIDAR data but could not decipher what units the output results are in, or how to compare those to the annotations in NuScenes.
• What is the best way for doing so? tools/test.py does not work for me. I have an issue open in MMDetection3D but nobody has reponded on there.
• I have tried running the BEVFusion Demo but that does not work as well

[OvenDes]

Hey, I am dealing with a similar problem right now. Did you ever figure out how to generate a confusion matrix with the NuScenes dataset?

[ranai-srivastav]

I did not. I ended making a script by myself that works purely for my use case. I will warn you the code was written in haste and it's not well documented. Feel free to try it out and if you wish to make changes and do a PR.

https://github.com/IowaState-AutonomousSystemsLab/opencv-image-saver/blob/main/src/confusion_matrix.py