Release 3 #44

vamsikalagaturu · 2025-12-17T17:15:01Z

No description provided.

Removed RGBD and color/depth enabling options from launch command.

Updated input mode handling and point cloud topic subscriptions. Improved empty spot finding logic and added error handling for plane segmentation.

Implemented a scoring system which would favour spots far from the objects, but also close to the center of the table

…able segmentation

…isualization

Change direction of arrow

Make sure the placed object would fit into the table

Updated README to focus on table segmentation and removed sections related to object detection and pose estimation.

Removed unused console scripts for OpenCV nodes.

Added instructions for running object segmentation and updated table segmentation command.

Added instructions for running object and table segmentation, including parameters and example commands.

Added instructions for running table height predictor and floor detection, along with results and methodology.

Predicts the Height of varying heights of Table

Added Action Server for Perception

Added instructions for launching perception and running action vision manager.

Copilot

Pull request overview

This is a major "Release 3" that adds significant new functionality to the perception and robotics system, including table height prediction, table segmentation for object placement, action server infrastructure, and tool detection capabilities.

Key Changes

New table_height_predictor package: Implements table height estimation using YOLO-World and SAM models with RANSAC-based floor detection
Enhanced perception capabilities: Adds table segmentation, object segmentation with sphere approximation, and action-based vision pipeline management
Action server infrastructure: Introduces custom ROS 2 action interfaces and vision manager for coordinated perception tasks
Tool detection training: Includes training scripts and demo code for YOLOv8 model fine-tuning
Documentation: Adds comprehensive feature readmes for new capabilities

Reviewed changes

Copilot reviewed 49 out of 128 changed files in this pull request and generated 25 comments.

Show a summary per file

File	Description
src/table_height_predictor/table_height_predictor/table_height_node.py	Implements table height estimation using YOLO-World, SAM, and RANSAC for floor detection
src/table_height_predictor/table_height_predictor/floor_detector_node.py	Provides floor plane segmentation from point clouds using Open3D RANSAC
src/perception/perception/table_segmentation.py	Finds empty spots on tables for object placement using plane segmentation and collision checking
src/perception/perception/segment_object.py	Segments objects and approximates them as spheres for collision avoidance
src/perception/perception/action_yolo_object_detection.py	YOLO detection node with lifecycle management via service toggle
src/perception/perception/action_pose_pca.py	Pose estimation node with PCA-based orientation and lifecycle management
src/perception/perception/action_vision_manager.py	Action server that coordinates YOLO and pose estimation nodes with temporal averaging
src/my_robot_interfaces/action/RunVision.action	Custom action definition for vision pipeline execution
src/perception/setup.py	Updates console script entry points for new perception nodes
Feature_Readme/*.md	Documentation for table height, table segmentation, action server, and tool detection features

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-17T17:24:52Z

Feature_Readme/Table_Height_Readme/README.md

@@ -0,0 +1,22 @@
+### Running Table Height
+```code
+ros2 run table_height_predictor table_heigt


Typo in command: 'table_heigt' should be 'table_height'

Copilot · 2025-12-17T17:24:53Z

src/perception/perception/action_yolo_object_detection.py

+from ultralytics import YOLO
+from ultralytics.engine.results import Boxes
+import torch
+import cv2


Import of 'cv2' is not used.

Copilot · 2025-12-17T17:24:53Z

Asjad/perception/perception/mock_depth_publisher.py

@@ -0,0 +1,56 @@
+import rclpy
+from rclpy.node import Node
+from sensor_msgs.msg import Image, PointCloud2, PointField


Import of 'PointField' is not used.

Copilot · 2025-12-17T17:24:54Z

Asjad/perception/perception/mock_depth_publisher.py

+from sensor_msgs.msg import Image, PointCloud2, PointField
+from cv_bridge import CvBridge
+import numpy as np
+import struct


Import of 'struct' is not used.

Copilot · 2025-12-17T17:24:54Z

Asjad/perception/perception/pose_pca.py

+import struct
+import sensor_msgs_py.point_cloud2 as pc2
+from transforms3d.quaternions import mat2quat
+from tf2_ros import TransformBroadcaster


Import of 'TransformBroadcaster' is not used.

Copilot · 2025-12-17T17:24:56Z

src/perception/perception/action_yolo_object_detection.py

+    node = YoloDetectorNode()
+    try:
+        rclpy.spin(node)
+    except KeyboardInterrupt:


'except' clause does nothing but pass and there is no explanatory comment.

Copilot · 2025-12-17T17:24:56Z

src/table_height_predictor/table_height_predictor/floor_detector_node.py

+    node = FloorDetectorNode()
+    try:
+        rclpy.spin(node)
+    except KeyboardInterrupt:


'except' clause does nothing but pass and there is no explanatory comment.

Copilot · 2025-12-17T17:24:56Z

src/table_height_predictor/table_height_predictor/table_height_node.py

+                # Sanity Check: Floor must be visibly lower (higher Y value) than table
+                if floor_y_at_table < t_y + 0.1: 
+                    floor_y_at_table = None # Invalid, floor appeared above table
+            except:


'except' clause does nothing but pass and there is no explanatory comment.

Copilot · 2025-12-17T17:24:56Z

src/table_height_predictor/table_height_predictor/table_height_node.py

+    node = TableHeightNode()
+    try:
+        rclpy.spin(node)
+    except KeyboardInterrupt: pass


'except' clause does nothing but pass and there is no explanatory comment.

Copilot · 2025-12-17T17:24:57Z

Asjad/perception/perception/opencv_camera_feed.py

+        if self.use_synthetic:
+            frame = self.synthetic_frame.copy()
+            # Add some noise or movement?
+            pass


Unnecessary 'pass' statement.

Mohsin-Ali-Mirza and others added 30 commits November 3, 2025 14:28

Update RealSense camera launch options

6b16a2d

Removed RGBD and color/depth enabling options from launch command.

Initial table segmentation file

8d789de

Refactor table segmentation logic and parameters

aa84c0b

Updated input mode handling and point cloud topic subscriptions. Improved empty spot finding logic and added error handling for plane segmentation.

Change find empty spot logic

49da4d2

Implemented a scoring system which would favour spots far from the objects, but also close to the center of the table

Fix formatting of table segmentation node entry in setup.py

a80be2a

Add orientation calculation for RViz, and change pose publishing in t…

5ff70bf

…able segmentation

Seperate arrow pose publishing from real pose publishing for better v…

0213d4d

…isualization

Publish a Marker for arrow rather than a Pose for better visualization

63e8f40

Change direction of arrow

Implement object filtering above the table

ed1c31a

Fixed comment

cd1d0d5

Improve empty spot detection logic by flattening objects

8ff0cb6

Add depth filtering to ignore the floor

d8334a8

Publish the safety radius of the object (sphere)

5b79abb

Add publishers and subscribers for the object radius

f12905f

Make sure the placed object would fit into the table

update readme for table segmentation

b15b048

removed input mode cropped

48db781

Fix z coordinate of the object on the table, add rviz config

1ba1570

Revise README for table segmentation instructions

37a0a91

Updated README to focus on table segmentation and removed sections related to object detection and pose estimation.

Cleaning repo

cd6ac9e

Update requirements for Python 3.12 compatibility

7ff10b9

No Longer Needed

f4026e3

No Longer Needed

29011dd

Remove OpenCV console scripts from setup.py

747222b

Removed unused console scripts for OpenCV nodes.

Cleaning the repo for table seg

3ae9c05

Added Segmentation of Custom Objects (Not Automatic)

8725cdb

Added custom object segmentation

43bf1d5

Update README with object segmentation instructions

a5a3af9

Added instructions for running object segmentation and updated table segmentation command.

Added example results for placing object

fcec07c

Merge branch 'feat/table-segmentation' into dev

1d677e6

Create ignore.html

8802fd2

Mohsin-Ali-Mirza and others added 20 commits December 17, 2025 13:57

Add README for Object and Table Segmentation

c5a291d

Added instructions for running object and table segmentation, including parameters and example commands.

Add README for Table Height Predictor features

e0b5847

Added instructions for running table height predictor and floor detection, along with results and methodology.

Create README.md

b351543

Create README.md

754ef9a

Add Release_3 section to README

e5c7f4f

Update README with Release_3 feature links

12bbc86

Added Table Height

451e967

Merge pull request #42 from Robots4Sustainability/feat/table-height

3e915e1

Predicts the Height of varying heights of Table

Added Action Server for Percption

9ab5bed

Merge pull request #43 from Robots4Sustainability/feat/action-server

8e1fba4

Added Action Server for Perception

Update README.md

25d6afc

Create ignore.html

f03dd97

Added Asjad's code

81cd79c

Add empy, lark-parser, and setuptools to requirements

ded2c63

Update README.md

e9140ff

Update README with terminal commands for perception

bcc5f36

Added instructions for launching perception and running action vision manager.

Add catkin_pkg dependency for Python 3.12

b783cac

Add table segmentation and object segmentation nodes

aee30d1

Update launch command in README for action server

cbc4c70

Create action_launch.py

17861bb

vamsikalagaturu requested a review from Copilot December 17, 2025 17:17

Copilot started reviewing on behalf of vamsikalagaturu December 17, 2025 17:18 View session

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Release 3 #44

Release 3 #44

Uh oh!

vamsikalagaturu commented Dec 17, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Copilot AI Dec 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Release 3 #44

Are you sure you want to change the base?

Release 3 #44

Uh oh!

Conversation

vamsikalagaturu commented Dec 17, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants