[Feature]: Debug shapes with nyx renderer ?

[jgleyze]

What feature or enhancement are you proposing?

Hello,

Quick question, how would I easily use the debug arrows/shapes with the nyx renderer. I am currently training locomotion policies and use them to point out the command. For my "good looking" evaluation videos I would like to use the latest nyx but I have a hard time with transfering the arrows from the rasterizer to nyx (I do understand that from the nature of the debug mesh nyx don't see them).

I could use an asset for the arrow mesh also but that would over complicate the code.

If it's not possible I think that could be a nice add to genesis :)

Motivation

Having basic shapes available for the nyx renderer would help a lot in showing tasks/commands.

What is the expected outcome of the implementation work?

Having basic visual shapes usable with the nyx renderer

[duburcqa]

Debug markers are currently not supported.

[jgleyze]

Oh ok thanks,
Then I am having a hard time showing them here an example code where I use draw_debug_arrow and export one frame but the arrow is not showing :

import os

import cv2
import genesis as gs
import gs_nyx.nyx_py_renderer as npr
from gs_nyx_plugin.nyx_camera_options import NyxCameraOptions


gs.init()

scene = gs.Scene(show_viewer=False)
scene.add_entity(gs.morphs.Plane())

cam = scene.add_sensor(
    NyxCameraOptions(
        res=(512, 512),
        pos=(2.6, -3.2, 2.0),
        lookat=(0.0, 0.0, 0.35),
        fov=45,
        spp=8,
        denoise=False,
        render_mode=npr.ERenderMode.Forward,
        lights=[{"type": "directional", "dir": (-0.4, -0.4, -1.0), "intensity": 4.0}],
    )
)

scene.build(n_envs=1)
scene.step()
scene.draw_debug_arrow(
    pos=(-0.35, 0.0, 0.18),
    vec=(0.9, 0.25, 0.35),
    radius=0.05,
    color=(1.0, 0.0, 0.0, 1.0),
)

rgb = cam.read().rgb[0].detach().cpu().numpy()

cv2.imwrite("nyx_debug_arrow.png", cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR))

I am running genesis 1.1.0 gs-nyx 0.1.2 and gs-nyx-plugin 0.1.3. What am I missing ?

Thanks, have a nice day !

[duburcqa]

@jgleyze sorry I messed up when updating my message the first time and remove the negation. To be clear: it is NOT supported, but it will be in the future!

[jgleyze]

Ahah ok no problem !

[AnisB]

As @duburcqa said, no support for these shape for now, but it is on our roadmap.