feat: update human-pose and age-gender experiments #650

jkbmrz · 2025-04-17T16:35:19Z

Purpose

Updating the experiments with the new host nodes (GatherData) and helper classes/functions (AnnotationHelper, generate_script_content).

Specification

human-pose:
- utilize GatherData node,
- utilize the depthai_nodes' generate_script_content function (instead of a custom implementation),
- update AnnotationNode to use AnnotationHelper.
age-gender:
- utilize GatherData node (instead of a custom implementation of the sync node in DetectionsAgeGenderSync),
- utilize the depthai_nodes' generate_script_content function (instead of a custom implementation in ProcessDetections node),
- update AnnotationNode to use AnnotationHelper,
- allow FPS setting.

Dependencies & Potential Impact

None

Deployment Plan

None

Testing & Validation

Running the experiments on RVC4 and RVC2 (human-pose only as the age-gender does not support it).

jkbmrz · 2025-04-23T11:41:41Z

Points of discussion:

Should we extend the AnnotationHelper with a method that automatically draws objects with coordinates relative to a bbox? For example, if one provides a bbox and keypoints with coordinatesrelative to that bbox, the AnnotationHelper could automatically adapt the keypoint coordinates to absolute values. This would save some lines of code in human-pose/utils/annotation_node.py and probably also elsewhere;
The depthai_nodes' generate_script_content function could also be utilized in some other experiments. We can either add it in this or a separate PR. I've identified the he following experiments:
- pose-estimation/animal-pose,
- object-tracking/deepsort-tracking,
- 3D-detection/objectron.

klemen1999

generate_script_content function could also be utilized in some other experiments

Essentially this could be used in every 2stage exepriment no?

klemen1999 · 2025-04-23T12:14:54Z

neural-networks/pose-estimation/human-pose/main.py

+    pose_nn.out.link(gather_data_node.input_data)
+    detections_filter.out.link(gather_data_node.input_reference)
+
+    skeleton_edges = (


I belive this information should be now already part of ImgDetectionExtended message

The detection model here is YOLOv6 so we are dealing with dai.ImgDetections. Do you propose we transform to ImgDetectionsExtended and merge it together with Keypoints into a single message?

Oh my bad, then this is ok as is I think

klemen1999 · 2025-04-23T12:19:26Z

neural-networks/facial-detection/age-gender/main.py

-    age_gender_node.getOutput(0).link(sync_node.age_input)
-    age_gender_node.getOutput(1).link(sync_node.gender_input)
+    # gather age-gender info
+    rec_gathered = pipeline.create(GatherData).build(fps, wait_count_fn=return_one)


What is the purpose of this gather? Generally the outputs of the same NN should already be syned between eachother no?

The purpose is to join the two recognitions (age and gender) into a single message that can then be synced with the detections message. Do you maybe see a better way of how this could be done?

Hm I'm thinking how should we set an example for best practice in this case. Maybe creating just GatheredData type manually would be better? Because you only want to "package" up all the things into one message and adding GatherData node just for this seems a bit overkill, adds overhead. But I'm not sure, maybe this is infact the cleanest solution. Thoughts?
Maybe @dominik737 if you have some thoughts?

Yeah I agree it is not the cleanest solution. Ideally, GatherData would sync input_data from multiple streams. But that might be an overkill for a single node.

I would advise to not use GatherData for this, as it's primary purpose should be to gather data on a non-trivial condition e.g. NNData based off len(ImgDetections.detections).

The ways I would go about this are:

Using dai.node.Sync, which is made to sync 1:1 messages based on timestamp. The output message is of type dai.MessageGroup.

Implement a node that can merge inputs into a dai.MessageGroup without syncing. The merging could be done based on a lambda as in GatherData node.

Make a custom type for this particular experiment and return it from a custom node that will merge the messages.

Have outputs property be part of the ParsingNeuralNetwork, where the multiple head outputs are outputted into a single dai.MessageGroup.

I personally like the option (4) the most. I think there will hardly be use beside this use-case of merging multiple head outputs into group. Else if such cases are expected I would opt for (2).

I like 4th option. Easy to implement on dai-nodes side and quite useful in such cases (at least 5 multihead models in the ZOO) + can work with GatherData.

I like option 4 as well, let's add it on dai-nodes side yeah

I agree. Starting a PR for it HERE.

jkbmrz added 5 commits April 17, 2025 17:20

feat: human-pose use GatherData and AnnotationHelper

fbfb4b1

feat: human-pose update fps setting

db12316

feat: human pose improve readability

8e371a2

feat: human-pose remove custom script generator

c6dfb79

feat: human-pose set default fps to None

85ecc92

This was referenced Apr 18, 2025

Gen3 - AgeGender & HumanPose: Adding TwoStageSync Node #641

Closed

feat: generate_script_content update luxonis/depthai-nodes#206

Merged

jkbmrz added 4 commits April 23, 2025 13:02

feat: age-gender update fps setting

9bc4e8e

feat: age-gender use GatherData node

d5dc61c

feat: age-gender use AnnotationHelper

bfbcf8c

fix: pre-commit

a834650

jkbmrz marked this pull request as ready for review April 23, 2025 12:06

klemen1999 reviewed Apr 23, 2025

View reviewed changes

jkbmrz added 4 commits April 23, 2025 15:46

fix: human-pose use default colors

c2eb33f

refactor: human-pose improve readability

cbd3ecb

refactor: age-gender improve readability

ea19f26

feat: age-gender use generic script generator

067de68

jkbmrz mentioned this pull request Apr 25, 2025

feat: ParsingNeuralNetwork add property to send out outputs of multiple heads in a single message luxonis/depthai-nodes#208

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: update human-pose and age-gender experiments #650

feat: update human-pose and age-gender experiments #650

jkbmrz commented Apr 17, 2025 •

edited

Loading

jkbmrz commented Apr 23, 2025 •

edited

Loading

klemen1999 left a comment

klemen1999 Apr 23, 2025

jkbmrz Apr 23, 2025

klemen1999 Apr 23, 2025

klemen1999 Apr 23, 2025

jkbmrz Apr 23, 2025

klemen1999 Apr 23, 2025 •

edited

Loading

jkbmrz Apr 23, 2025

dominik737 Apr 23, 2025

kkeroo Apr 24, 2025

klemen1999 Apr 24, 2025

jkbmrz Apr 24, 2025

feat: update human-pose and age-gender experiments #650

Are you sure you want to change the base?

feat: update human-pose and age-gender experiments #650

Conversation

jkbmrz commented Apr 17, 2025 • edited Loading

Purpose

Specification

Dependencies & Potential Impact

Deployment Plan

Testing & Validation

jkbmrz commented Apr 23, 2025 • edited Loading

klemen1999 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

klemen1999 Apr 23, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkbmrz commented Apr 17, 2025 •

edited

Loading

jkbmrz commented Apr 23, 2025 •

edited

Loading

klemen1999 Apr 23, 2025 •

edited

Loading