WBC not working on real G1

[BankDM]

We are not able to let the WBC run on the real robot.
Everything seems to work in simulation.

When we set the flag to real, the hands of the robot calibrate and the robot goes into debug mode but afterwards the robot doe not move. We dont get any errors, from the terminal it looks like the policy is streaming values to the robot, but the robot doesnt move.

Has anyone experienced behaviour like this and how did you solve it?

[limshoonkit]

Hi, try without the hands args, if you dont have the dex3 hand it would not work

[BankDM]

Hi, we have the dex3 hands. And they also calibrate themselves successfully after running the control loop. But after that the robot just does not move

[limshoonkit]

1. Is the G1 in dev mode?
2. Does the G1 go in idle mode where both hands is holding up?
3. You have to activate the policy, on keyboard it is ']' key

[BankDM]

1. G1 is in dev mode
2. No It does not go into idle not by pressing i or l (after hitting l it states that Teleoperation will start/is started but nothing will happen afterwards)
3. Yes we tried that. Simulation is also working fine just the real robot is not.

Unitree xr_teleoperate is working, however.

[limshoonkit]

Is your network interface still on lo (loopback), if you use ethernet cable, you need specify it, eg enpso1.

Any other debug outputs, what is the command you run?

[BankDM]

Where can I set the network interface?

We tried the flags --interface real and --interface (our network interface)

But that didn't change anything.

We tried the deployment script and the control loop scripts with the same results using the commands from the readme. No additional debug output.

[limshoonkit]

https://github.com/NVlabs/GR00T-WholeBodyControl/blob/3966393ed7a37612af16c9b77e7aaed91231c407/gr00t_wbc/control/main/teleop/configs/configs.py#L90C1-L91C1

that should work out of the box
if running the control loop does not work https://github.com/NVlabs/GR00T-WholeBodyControl/blob/main/gr00t_wbc/control/main/teleop/run_g1_control_loop.py

try this in the docker and check whether the lower body is moving
python scripts/deploy_g1.py
--interface real
--no-with_hands
--no-enable_upper_body_operation

[BankDM]

Unfortuantely none of the tree worked. This is the output we get:

python scripts/deploy_g1.py
--interface real
--no-with_hands
--no-enable_upper_body_operation

So no visible errors. The sim that opens up in parallel is running just fine
Network is confiugred correctly:

I mean it is also moving and calibrating its hands in the beginning. But nothing happens afterwards.
What version of the unitree sdk and unitree python sdk were you using?

[limshoonkit]

Your use policy action is false. You need to enable it.

Press the ']' button on the keyboard to activate it and 'o' button to deactivate it

[limshoonkit]

Your use policy action is false. You need to enable it.

Press the ']' button on the keyboard to activate it and 'o' button to deactivate it

[BankDM]

Your use policy action is false. You need to enable it.

Press the ']' button on the keyboard to activate it and 'o' button to deactivate it

I did that as well (in all variations of pressing l, i and ] in different orders on startup), but nothing happens. It shows that the policy is enabaled, it works in simulation but not on the robot itself. I also never got it to do the start position. In simulation everything works, we are able to control the robot normally.

The robot just does not move after the hands calibrated. It stays in the damp state inside of the debug mode.

[limshoonkit]

This is definitely strange, it just works fine on ours. We are using the unitree g1 29 dof edu version. Did not change the default jetpack or sdk that came with it.

Even for the gr00t wbc, we are using the docker that came with the unitree python sdk.

Probably can try creating a simple test script whether you can send the low level angle commands and start there.

[BankDM]

This fixes the issue: #11

However the control has quite a bit of lag, which is also varying. we have a 4090 so compute shodnt be the issue. Have you experienced something like that?

[limshoonkit]

Great that you are able to get it working.

Your total loop is about 20ms which is the base 50hz control cycle same as ours.

Given you are using ethernet connection, you should not have any noticable latency.

We are using pico and a rtx4090 as well.
Pico to the rtx4090 is manageable as well.

The issue we are facing now is that the lower body walking policy is not quite stable when strafing and moving backwards.

[BankDM]

Ufortuantely it is much longer and unreliable i.e. sometimes its quite fast, someties the control inputs take a second to be done on the G1. The PC is connected over ethernet, the Headset is connected via wifi to the PC.

Are you working with NVIDIA/for NVIDIA and do you know, when the code for SONIC will be released?