Orientation issue -> Landscape

[maheshgoqii]

first of all thank you, for your effort.

issue:

• when phone rotation goes to portrait to landscape camera view getting black but skeletion is working fine in ios

[tusharthinksys]

@maheshgoqii We will look into this in relation to the support modes.

[maheshgoqii]

@maheshgoqii We will look into this in relation to the support modes.

I would like to propose a few enhancements and also need some clarification:

1. Pose-detection skeleton overlay
   I would like to add support for drawing the pose-detection skeleton skeleton img.
   This will help visually validate the detected keypoints.

2. Exercise-matching logic
   If we want to identify or validate a specific exercise, what would be the recommended way to implement this?
   Should we compare the keypoints at the native level, or is there another suggested approach?

3. Application Crashed
   I also encountered an application crash while testing the current implementation.
   The crash occurs when the RNMediapipe component is opened and then we navigate back from it.
   It would be helpful to understand the root cause or any known limitations that might be triggering this behavior.

I’m also planning to work on these enhancements and will try to contribute through a pull request.

[tusharthinksys]

@maheshgoqii

Thanks for your interest in contributing.

Here’s some clarity on each of your points:

1. Pose-detection skeleton overlay:
   Feel free to proceed — a PR for this would be great.

2. Exercise-matching / pose-analysis logic:
   From our side, the recommended approach is:
   Perform exercise/pose analysis at the JavaScript layer using the body landmarks.

The MediaPipe landmark data already provides all joint positions (33 points for pose).
Most exercise-matching logic can be implemented by:

a. Calculating angles between joints.
b. Checking positional relationships.
c. Detecting repetitions through temporal changes.

3. React Native callback for analysis:
   We already expose the "onLandmark" callback, which provides full landmark data to React Native in realtime.
   This callback is specifically intended for developers to run their own analysis logic (including exercise-matching as mentioned above).

[maheshgoqii]

• Hey @tusharthinksys

I found a bug that occurs when current screen unmount (in my case go back)
Bug Screenshot

• Does we need to close the component? while navigating?

[tusharthinksys]

You should release the component when the screen unmounts.
The camera and MediaPipe pipeline both use heavy system resources, so keeping them active after navigating away can cause issues.

When the user returns to the screen, you can safely re-initialize the component again.

[ErMapsh]

You should release the component when the screen unmounts. The camera and MediaPipe pipeline both use heavy system resources, so keeping them active after navigating away can cause issues.

When the user returns to the screen, you can safely re-initialize the component again.

We need call stop() method in Cleanup function, I'm right? that doesn't exist right now, but have to create

[mantu-bit]

You should release the component when the screen unmounts. The camera and MediaPipe pipeline both use heavy system resources, so keeping them active after navigating away can cause issues.

When the user returns to the screen, you can safely re-initialize the component again.

I don't think this is unmounting issue, It is triggered in react fabric and most probably it is because this package do not support new arch. I have also faced same issue while writing pose detection library. You need to write new arch support using turbo modules

[tusharthinksys]

You should release the component when the screen unmounts. The camera and MediaPipe pipeline both use heavy system resources, so keeping them active after navigating away can cause issues.
When the user returns to the screen, you can safely re-initialize the component again.

I don't think this is unmounting issue, It is triggered in react fabric and most probably it is because this package do not support new arch. I have also faced same issue while writing pose detection library. You need to write new arch support using turbo modules

If this is confirmed as a New Architecture issue, We'll prioritize adding Turbo Module support.

[mantu-bit]

You should release the component when the screen unmounts. The camera and MediaPipe pipeline both use heavy system resources, so keeping them active after navigating away can cause issues.
When the user returns to the screen, you can safely re-initialize the component again.

I don't think this is unmounting issue, It is triggered in react fabric and most probably it is because this package do not support new arch. I have also faced same issue while writing pose detection library. You need to write new arch support using turbo modules

If this is confirmed as a New Architecture issue, We'll prioritize adding Turbo Module support.

I wrote the posedetection implementation in turbo modules and i am still facing the same issues when i use navigation.replace or reset. I am still figuring out the issue. For reference here is my package https://github.com/EndLess728/react-native-mediapipe-posedetection

Edit - It just got fixed when upgraded my expo project from 53 to 54.