Best practice to customize teacher/actor/critic networks

[alaurenzi]

Dear developers,
What is the best practice to have RSLRL runners load a custom network (module)?

From a quick look to the code, it looks like RSLRL relies on a class_name attribute inside the policy config dataclass, which is then passed to python's built-in eval:

rsl_rl/rsl_rl/runners/distillation_runner.py

Line 156 in c49573e

student_teacher_class = eval(self.policy_cfg.pop("class_name"))

This however relies on the runner having pre-imported the required python module

rsl_rl/rsl_rl/runners/on_policy_runner.py

Line 19 in c49573e

from rsl_rl.modules import ActorCritic, ActorCriticRecurrent, resolve_rnd_config, resolve_symmetry_config

I believe this make it more difficult for users to inject their custom modules, but maybe I've missed something.

We're now experimenting with swapping class_name with class_type (in the same spirit of IsaacLab's config dataclasses), which returns a handle to a class that the user can freely set at configuration time.

student_teacher_class = self.policy_cfg["class_type"]  # or something similar

Thanks for your help!

Arturo

@lucarossini-iit

[ClemensSchwarke]

Hi @alaurenzi, thanks for bringing this up! I think adding a custom model to the imports isn't too much work but if there is a better way we could definitely think about changing it. Could you create a PR once you have a version you are happy with?

Cheers,
Clemens

[alaurenzi]

Hi Clemens,
Yes, to add an import is definitely not much trouble, for us it was more avoiding to fork your repository - and just use the pre-installed one (dependency of isaac lab).

Now we're resorting to what I consider a bit "hack"-ish (but maybe it's just the way Python is meant to be used) - that is to monkey-patch the on_policy_runner module) from our code, e.g.

from rsl_rl.runners import on_policy_runner
on_policy_runner.MyActorCritic = MyActorCritic

so that is becomes available inside the eval call.

[ClemensSchwarke]

Ah now I get it, yeah that is not ideal. I will put this on my list and will try to find a solution. We usually directly modify the library so it was never an issue.

Thanks!

[ClemensSchwarke]

Hi @alaurenzi,
there is an open PR #145 to fix this issue. Hopefully that helps :)

[alaurenzi]

Hi @alaurenzi, there is an open PR #145 to fix this issue. Hopefully that helps :)

Great, thanks! This definitely simplifies integration of custom modules!