feature(rjy): add crowd md env new, and multi-head policy #230

nighood · 2024-06-07T08:15:01Z

New Environment: CrowdSim
- Description: The CrowdSim environment is a grid world simulation where robots navigate through an environment populated with humans. The primary task for the robots is to minimize the average age of information (AoI) of the humans by moving to their locations and collecting data. Key features of the environment include:
  - Dynamic Interaction: Humans generate data at a constant rate, and robots must manage their limited energy supply while moving to collect this data.
  - Modes:
    - Easy Mode: Robots can only collect data from humans within a certain range, and collecting data resets the AoI of a human to zero.
    - Hard Mode: Robots can collect data from humans even when not within range, and collecting data does not reset the total AoI.
  - Initialization: The environment starts with a dataset of human locations and timestamps. Robots aim to minimize the average AoI by efficiently collecting data.
  - Completion Criteria: The environment is considered solved when the average AoI is minimized to a certain threshold or the time limit is reached.
  - Additional Features: Methods for resetting, closing, and stepping through the environment, seeding for reproducibility, saving replay videos, and generating random actions. Detailed properties for accessing observation space, action space, and reward space.
Multi-Head Policy Version for MuZero, EfficientZero, and Sampled EfficientZero
- Modification: Introduced multi-head policy versions for the MuZero, EfficientZero, and Sampled EfficientZero algorithms.

lzero/agent/efficientzero.py

fix(rjy): fix crowd env scale/add entropy info

…nto rjy-crowd-md-env-new

lzero/agent/sampled_efficientzero.py

puyuan1996 · 2024-06-25T09:24:30Z

lzero/mcts/utils.py

        if observation_array.ndim == 3:
            # Flatten the last two dimensions
            observation_array = observation_array.reshape(batch_size, -1)
        else:
            raise ValueError("For 'mlp' model_type, the observation must have 3 dimensions [B, S, O]")

+    elif model_type == 'rgcn':
+        if observation_array.ndim == 4:
+            # TODO(rjy): strage process


strage process是什么意思？

解释一下'rgcn'下面各种情况的含义吧

puyuan1996 · 2024-06-25T09:25:29Z

lzero/model/common.py

-            activation: Optional[nn.Module] = nn.ReLU(inplace=True),
-            last_linear_layer_init_zero: bool = True,
-            norm_type: Optional[str] = 'BN',
+        self,


bash format.sh一下

puyuan1996 · 2024-06-25T09:26:27Z

lzero/model/common.py

+        output_support_size: int = 601,
+        last_linear_layer_init_zero: bool = True,
+        activation: Optional[nn.Module] = nn.ReLU(inplace=True),
+        norm_type: Optional[str] = 'BN',


这些缩进还是换成原来的格式哈

puyuan1996 · 2024-06-25T09:27:25Z

lzero/model/common_gcn.py

+    """
+    Overview:
+        Relational graph convolutional network layer.
+    """


给一下这里代码实现的参考链接

gcn的实验我们有测试吗？还是只是测试了md的情况？gcn相对md的优点是？目前我们合到main里面的只放完整测试过的吧

zoo/CrowdSim/envs/CrowdSim_env.py

puyuan1996 · 2024-06-25T09:34:50Z

zoo/CrowdSim/envs/CrowdSim_env.py

+@ENV_REGISTRY.register('crowdsim_lightzero')
+class CrowdSimEnv(BaseEnv):
+
+    def __init__(self, cfg: dict = {}) -> None:


增加overview注释

zoo/CrowdSim/envs/Crowdsim/env/model/agent.py

puyuan1996 · 2024-06-25T09:37:41Z

zoo/CrowdSim/envs/crowdsim_lightzero_env.py

+@ENV_REGISTRY.register('crowdsim_lightzero')
+class CrowdSimEnv(BaseEnv):
+
+    def __init__(self, cfg: dict = {}) -> None:


增加overview注释，将之前的文档中英文版本放在这里的envs/路径下面哈

zoo/CrowdSim/envs/test_crowdsim_lightzero_env.py

puyuan1996 · 2024-06-25T09:42:54Z

lzero/model/muzero_model_md.py

+
+
+@MODEL_REGISTRY.register('MuZeroModelMD')
+class MuZeroModelMD(nn.Module):


所有增加的文件都需要继承自已有的文件，以避免冗余代码哈，只重写修改过的method。例如这里需要继承自MuZeroModel。相应的注释也需要更新一下。

puyuan1996 · 2025-02-14T07:04:04Z

zoo/crowd_sim/envs/CrowdSim/crowd_sim_base_config.py

+            "nlon": 200,
+            "nlat": 120,
+            "human_num": 59,
+            "dataset_dir": 'crowd_sim/dataset/purdue/59 users.csv', # TODO


这里3种大学的不同数量users.csv是什么含义？是作为环境设置的一部分吗？需要上传上去环境才能运行的吧？

这个环境的代码是在哪篇论文和code上修改的呢

puyuan1996 · 2025-02-14T07:04:46Z

lzero/model/common_gcn.py

+    """
+    Overview:
+        Relational graph convolutional network layer.
+    """


gcn的实验我们有测试吗？还是只是测试了md的情况？gcn相对md的优点是？目前我们合到main里面的只放完整测试过的吧

puyuan1996 · 2025-02-14T07:08:26Z

zoo/crowd_sim/envs/crowdsim_lightzero_env.py

+
+@ENV_REGISTRY.register('crowdsim_lightzero')
+class CrowdSimEnv(BaseEnv):
+


这个和crowdsim_env的区别是？应该只保留一个就好吧

…rowd-md-env-new

puyuan1996 · 2025-02-14T07:58:56Z

zoo/crowd_sim/envs/CrowdSim/crowd_sim_base_config.py

+    print(CL)
+
+
+# Maximum Coupling Loss (110dB is recommended)


这个是什么含义？

nighood and others added 16 commits June 8, 2023 21:48

env(rjy): add crowdsim env

c27ae92

config(rjy): add mz/ez config for crowdsim

c6acd7d

Merge branch 'main' into rjy-crowd-2

0d235ab

env(rjy): add crowdsim env

ad0cd02

feature(rjy): add RGCN for represent net

14542a1

feature(rjy): add obs/action env mode. fix rgcn pipeline.

dc4a774

feature(rjy): add multi-head policy(combine logits)

c99db40

feature(rjy): modify new env with transmitted data

61831f1

feature(rjy): add rough vis of crowdsim

9599faa

polish(rjy): fix new env info in collecter

15d9a44

feature(rjy): add sez mlp_multi-head

3c8804d

feature(rjy): set the environment to two modes

c6723a0

Merge branch 'rjy-crowd-md-com-sez' into rjy-crowd-md-env-new

e100fe4

feature(rjy): add ez multi-head model

c4e9d58

Merge branch 'rjy-crowd-md-com-ez' into rjy-crowd-md-env-new

f677af1

polish(rjy): add v_trans in config

cb044af

puyuan1996 added environment New or improved environment config New or improved configuration labels Jun 7, 2024

fix(rjy): fix env bug

715b5b8

puyuan1996 reviewed Jun 12, 2024

View reviewed changes

lzero/agent/efficientzero.py Show resolved Hide resolved

puyuan1996 mentioned this pull request Jun 12, 2024

feature(rjy): add crowdsim env and related configs #208

Closed

nighood and others added 4 commits June 14, 2024 15:43

feature(rjy): add entropy info/set margin

fecf5d3

Merge pull request #1 from nighood/rjy-crowd-md-env-scale

c4da015

fix(rjy): fix crowd env scale/add entropy info

polish(rjy): polish code according to comments

63d37a8

Merge branch 'rjy-crowd-md-env-new' of github.com:nighood/LightZero i…

745f0a8

…nto rjy-crowd-md-env-new

puyuan1996 requested changes Jun 25, 2024

View reviewed changes

puyuan1996 reviewed Jun 25, 2024

View reviewed changes

polish(pu): reformat zoo/crowd_sim/

f41a41b

puyuan1996 requested changes Feb 14, 2025

View reviewed changes

Merge tag 'main' of https://github.com/opendilab/LightZero into rjy-c…

0a7af42

…rowd-md-env-new

Merge branch 'main' into rjy-crowd-md-env-new

62c38a2

puyuan1996 reviewed Feb 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature(rjy): add crowd md env new, and multi-head policy #230

feature(rjy): add crowd md env new, and multi-head policy #230

nighood commented Jun 7, 2024 •

edited

Loading

puyuan1996 Jun 25, 2024

puyuan1996 Feb 14, 2025

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Feb 14, 2025

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Jun 25, 2024

puyuan1996 Feb 14, 2025

puyuan1996 Feb 14, 2025

puyuan1996 Feb 14, 2025

puyuan1996 Feb 14, 2025

puyuan1996 Feb 14, 2025



		@MODEL_REGISTRY.register('MuZeroModelMD')
		class MuZeroModelMD(nn.Module):


		@ENV_REGISTRY.register('crowdsim_lightzero')
		class CrowdSimEnv(BaseEnv):

feature(rjy): add crowd md env new, and multi-head policy #230

Are you sure you want to change the base?

feature(rjy): add crowd md env new, and multi-head policy #230

Conversation

nighood commented Jun 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nighood commented Jun 7, 2024 •

edited

Loading