Isaac PointNet 🚀

Overview

Isaac PointNet is an open-source project that provides a structured approach to 3D pointcloud calssfication and cuboid detection, by capturing data on simulation theough an automated labeling process, with pytorch and on issac sim:

🔎 why on sim ?

The approach aims to shed light on affordable prototyping. 3D LiDARs are extremely expensive, but with this approach, you can create a LiDAR object detection model without even purchasing one.

🚧 Project Limits

1. While the LabelImg process is much easier, it only works on static scenes.

2. scene creation in itself is also very time consuming.

3. The labeling task takes more time when more instances are present.

4. No class implementation at the moment, only instances.

⚙️ Features

Scene & PCD Data (through ISSAC)

📡 Capturing point cloud data in Isaac Sim using Physics LiDAR.

Labeled Data:

🏷️ Auto-labeling objects lable instances by adding them in groups on issac.

Model Architecture

PointNet Model Architecture

🏗️ Training a Model based on a PointNet Architecture for object detection and segmentation.

Prediction vs Processed Prediction

**☁️ processing point cloud** processing predction to enahnce object accuracy.

---

Data Format

PCD v0.7 Format

Point cloud files are saved using the PCD v0.7 format. An example header is as follows:

.PCD v0.7 - Point Cloud Data file format
FIELDS x y z instance
SIZE 4 4 4 4
TYPE F F F I
COUNT 1 1 1 1
WIDTH N
HEIGHT 1
VIEWPOINT 0 0 0 1 0 0 0
POINTS N
DATA ascii

• x, y, z: The 3D coordinates for each point.
• instance: An integer representing the object instance label (e.g., background is labeled as 0).

---

Pretrained Example weights from Hugging Face 🤗

How to Pull the Example Model

A pretrained PointNet-based model is available on Hugging Face to help jump-start experimentation. You can download it using either of the following methods:

Option 1: Clone the Repository

git clone https://huggingface.co/Hamze-Hammami/ISSAC-PointNet

Option 2: Download from Hugging Face Model Hub

Download the Model Here

Source: Hugging Face Hub Documentation

---

auto-Labeling, Training, Testing, and Visualization Scripts

📝 auto-Labeling PointCloud data through issac

This script is an experimental implementation for capturing and labeling cones in a simulated Isaac Sim environment. The script utilizes a simulated 3D LiDAR sensor to extract and label point cloud data.

Running via Script Editor (Isaac Sim) To run the script directly within Isaac Sim, follow these steps:

1. Open Isaac Sim and load your scene.
2. Navigate to Window → Script Editor.
3. open the script scripts/ISSAC_LABEL_PCD.py, Copy and paste the script into the editor and run it
4. Click Run to execute the script.

currently script captures static scenes, finding a way to capture live scenes would speed up operations

🏋️ Training the PointNet Model

The training script can be used to Train a PointNet-based model on labeled point cloud data. The current setup has been tested on a limited dataset of five PCD files.

Run the training script with:

python scripts/train.py

🧪 Testing the Model on a Labeled PCD File

To evaluate the trained model and compare the predicted cuboids with the actual labeled cuboids, run:

python scripts/test.py

🎨 Visualizing Point Cloud Data

An experimental script for visualizing labeled point cloud data is provided. This script supports toggling between predicted, merged, and corrected bounding boxes:

python scripts/visualise_pcd.py

---

📖 Citation

community Contributions and improvements are welcome!

however, if you use outside of the repo towards your research, please cite it as:

@misc{hamzehammami2025isaacpointnet,
  author = {Hamze Hammami},
  title = {Isaac Point Net},
  year = {2025},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Hamze-Hammami/Issac-Point-Net}},
  note = {Accessed: 2025-02-07}
}

Author & Contact Links ✍️

Developed by me Hamze Hammami Check out my other Work :D

Hugging Face
ORCID
LinkedIn

Contributors 👥

project author discontinued working on project, community contribution are welcome

🌟 Want to contribute? Open a pull request and your name will be added here!