Skip to content

Logging #260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open

Logging #260

wants to merge 2 commits into from

Conversation

dapatil211
Copy link
Collaborator

Adds console logging to Hive. See attached output.

$ python -m hive.runners.single_agent_loop -p gym/dqn.yml
[INFO 2022-03-24 19:47:58,550 torch_utils.py:197] Registered PyTorch optimizers.
[INFO 2022-03-24 19:47:58,550 torch_utils.py:224] Registered PyTorch losses.
[INFO 2022-03-24 19:47:58,553 utils.py:142] Registered PyTorch initialization functions.
[INFO 2022-03-24 19:47:58,554 __init__.py:18] Registered function approximators.
[INFO 2022-03-24 19:47:58,557 __init__.py:18] Registered replays.
[INFO 2022-03-24 19:47:59,129 schedule.py:204] Registered schedules.
[INFO 2022-03-24 19:47:59,129 loggers.py:454] Registered loggers.
[INFO 2022-03-24 19:47:59,130 __init__.py:21] Registered agents.
[INFO 2022-03-24 19:47:59,633 __init__.py:46] Registered environments.
wandb: Currently logged in as: dapatil211 (use `wandb login --relogin` to force relogin)
wandb: wandb version 0.12.11 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.12.9
wandb: Syncing run gym-dqn
wandb:  View project at https://wandb.ai/dapatil211/Hive
wandb:  View run at https://wandb.ai/dapatil211/Hive/runs/323x6337
wandb: Run data is saved locally in /home/darshan/research/RLHive/wandb/run-20220324_194800-323x6337
wandb: Run `wandb offline` to turn off syncing.
[INFO 2022-03-24 19:48:07,404 single_agent_loop.py:214] Using config: 
{'agent': {'kwargs': {'act_dim': 2,
                      'batch_size': 128,
                      'device': 'cuda',
                      'discount_rate': 0.99,
                      'epsilon_schedule': {'kwargs': {'value': 0.1},
                                           'name': 'ConstantSchedule'},
                      'log_frequency': 100,
                      'logger': {},
                      'loss_fn': {'kwargs': {}, 'name': 'SmoothL1Loss'},
                      'min_replay_history': 500,
                      'obs_dim': (4,),
                      'optimizer_fn': {'kwargs': {}, 'name': 'Adam'},
                      'replay_buffer': {'kwargs': {'capacity': 1000,
                                                   'gamma': 0.99,
                                                   'observation_dtype': 'np.float64',
                                                   'observation_shape': (4,),
                                                   'stack_size': 1},
                                        'name': 'CircularReplayBuffer'},
                      'representation_net': {'kwargs': {'hidden_units': [256,
                                                                         256]},
                                             'name': 'MLPNetwork'},
                      'reward_clip': 1,
                      'target_net_update_schedule': {'kwargs': {'off_value': False,
                                                                'on_value': True,
                                                                'period': 100},
                                                     'name': 'PeriodicSchedule'},
                      'test_epsilon': 0.001,
                      'update_period_schedule': {'kwargs': {'off_value': False,
                                                            'on_value': True,
                                                            'period': 1},
                                                 'name': 'PeriodicSchedule'}},
           'name': 'DQNAgent'},
 'environment': {'kwargs': {'env_name': 'CartPole-v0'}, 'name': 'GymEnv'},
 'loggers': {'kwargs': {'logger_list': [{'kwargs': {}, 'name': 'ChompLogger'},
                                        {'kwargs': {'name': 'gym-dqn',
                                                    'project': 'Hive',
                                                    'resume': 'allow',
                                                    'start_method': 'fork'},
                                         'name': 'WandbLogger'}]},
             'name': 'CompositeLogger'},
 'max_steps_per_episode': 1000,
 'run_name': 'gym-dqn',
 'save_dir': 'experiment',
 'saving_schedule': {'kwargs': {'off_value': False,
                                'on_value': True,
                                'period': 10000},
                     'name': 'PeriodicSchedule'},
 'stack_size': 1,
 'test_episodes': 10,
 'test_frequency': 5000,
 'train_steps': 50000}
[INFO 2022-03-24 19:48:07,405 single_agent_loop.py:217] Created runner. Starting run!
[INFO 2022-03-24 19:48:07,405 base.py:130] Starting train loop
[INFO 2022-03-24 19:48:19,039 base.py:143] 5123/50000 training steps completed.
[INFO 2022-03-24 19:48:19,039 base.py:147] Running testing.
[INFO 2022-03-24 19:48:19,639 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:48:31,254 base.py:143] 10009/50000 training steps completed.
[INFO 2022-03-24 19:48:31,254 base.py:147] Running testing.
[INFO 2022-03-24 19:48:31,708 base.py:152] Testing results: {'0_episode_length': 217.4, '0_reward': 217.4, 'full_episode_length': 217.4}
[INFO 2022-03-24 19:48:31,708 base.py:156] Saving run.
[INFO 2022-03-24 19:48:31,709 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:48:42,861 base.py:143] 15013/50000 training steps completed.
[INFO 2022-03-24 19:48:42,861 base.py:147] Running testing.
[INFO 2022-03-24 19:48:43,242 base.py:152] Testing results: {'0_episode_length': 163.9, '0_reward': 163.9, 'full_episode_length': 163.9}
[INFO 2022-03-24 19:48:54,719 base.py:143] 20073/50000 training steps completed.
[INFO 2022-03-24 19:48:54,719 base.py:147] Running testing.
[INFO 2022-03-24 19:48:55,200 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:48:55,200 base.py:156] Saving run.
[INFO 2022-03-24 19:48:55,201 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:07,722 base.py:143] 25073/50000 training steps completed.
[INFO 2022-03-24 19:49:07,722 base.py:147] Running testing.
[INFO 2022-03-24 19:49:08,202 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:19,683 base.py:143] 30121/50000 training steps completed.
[INFO 2022-03-24 19:49:19,683 base.py:147] Running testing.
[INFO 2022-03-24 19:49:20,162 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:20,162 base.py:156] Saving run.
[INFO 2022-03-24 19:49:20,163 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:31,406 base.py:143] 35177/50000 training steps completed.
[INFO 2022-03-24 19:49:31,406 base.py:147] Running testing.
[INFO 2022-03-24 19:49:31,870 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:43,221 base.py:143] 40126/50000 training steps completed.
[INFO 2022-03-24 19:49:43,221 base.py:147] Running testing.
[INFO 2022-03-24 19:49:43,713 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:43,713 base.py:156] Saving run.
[INFO 2022-03-24 19:49:43,714 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:55,012 base.py:143] 45101/50000 training steps completed.
[INFO 2022-03-24 19:49:55,012 base.py:147] Running testing.
[INFO 2022-03-24 19:49:55,491 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:50:06,749 base.py:143] 50164/50000 training steps completed.
[INFO 2022-03-24 19:50:06,749 base.py:147] Running testing.
[INFO 2022-03-24 19:50:07,232 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:50:07,232 base.py:156] Saving run.
[INFO 2022-03-24 19:50:07,233 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:50:07,736 experiment.py:82] Saving the experiment at experiment/gym-dqn/current

wandb: Waiting for W&B process to finish, PID 456977... (success).
wandb:                                                                                
wandb: Run history:
wandb:                   0/epsilon █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
wandb:                0/train_loss ▁▁▁▁▁▁▁▃▂▁▁▃▃▁▁▃▄▄▁▁▄▁▁▁▁▁▄▄█▁▁▁▂▄▃▁▁▁▅█
wandb:                0/train_qval ▁▁▁▂▃▃▄▄▅▅▅▆▆▆▆▇▇▇▇▇▇▇██████████▇███████
wandb:                      0_step ▁▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▆▇▇▇▇▇███
wandb:       test/0_episode_length ██▁████████
wandb:               test/0_reward ██▁████████
wandb:    test/full_episode_length ██▁████████
wandb:                   test_step ▁▁▁▁▁▂▂▂▂▃▃▃▃▃▃▃▃▄▄▄▅▅▅▅▆▆▆▆▆▆▆▆▇▇▇▇████
wandb:      train/0_episode_length ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:              train/0_reward ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:   train/full_episode_length ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:                  train_step ▁▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▆▇▇▇▇▇███
wandb: 
wandb: Run summary:
wandb:                   0/epsilon 0.1
wandb:                0/train_loss 0.78445
wandb:                0/train_qval 101.70953
wandb:                      0_step 50164
wandb:       test/0_episode_length 220.0
wandb:               test/0_reward 220.0
wandb:    test/full_episode_length 220.0
wandb:                   test_step 11
wandb:      train/0_episode_length 200
wandb:              train/0_reward 200.0
wandb:   train/full_episode_length 200
wandb:                  train_step 50164
wandb: 
wandb: Synced 7 W&B file(s), 0 media file(s), 0 artifact file(s) and 1 other file(s)
wandb: Synced gym-dqn: https://wandb.ai/dapatil211/Hive/runs/323x6337
wandb: Find logs at: ./wandb/run-20220324_194800-323x6337/logs/debug.log
wandb:

@sriyash421
Copy link

sriyash421 commented May 19, 2022 •
edited
Loading

Adds console logging to Hive. See attached output.

$ python -m hive.runners.single_agent_loop -p gym/dqn.yml
[INFO 2022-03-24 19:47:58,550 torch_utils.py:197] Registered PyTorch optimizers.
[INFO 2022-03-24 19:47:58,550 torch_utils.py:224] Registered PyTorch losses.
[INFO 2022-03-24 19:47:58,553 utils.py:142] Registered PyTorch initialization functions.
[INFO 2022-03-24 19:47:58,554 __init__.py:18] Registered function approximators.
[INFO 2022-03-24 19:47:58,557 __init__.py:18] Registered replays.
[INFO 2022-03-24 19:47:59,129 schedule.py:204] Registered schedules.
[INFO 2022-03-24 19:47:59,129 loggers.py:454] Registered loggers.
[INFO 2022-03-24 19:47:59,130 __init__.py:21] Registered agents.
[INFO 2022-03-24 19:47:59,633 __init__.py:46] Registered environments.
wandb: Currently logged in as: dapatil211 (use `wandb login --relogin` to force relogin)
wandb: wandb version 0.12.11 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.12.9
wandb: Syncing run gym-dqn
wandb:  View project at https://wandb.ai/dapatil211/Hive
wandb:  View run at https://wandb.ai/dapatil211/Hive/runs/323x6337
wandb: Run data is saved locally in /home/darshan/research/RLHive/wandb/run-20220324_194800-323x6337
wandb: Run `wandb offline` to turn off syncing.
[INFO 2022-03-24 19:48:07,404 single_agent_loop.py:214] Using config: 
{'agent': {'kwargs': {'act_dim': 2,
                      'batch_size': 128,
                      'device': 'cuda',
                      'discount_rate': 0.99,
                      'epsilon_schedule': {'kwargs': {'value': 0.1},
                                           'name': 'ConstantSchedule'},
                      'log_frequency': 100,
                      'logger': {},
                      'loss_fn': {'kwargs': {}, 'name': 'SmoothL1Loss'},
                      'min_replay_history': 500,
                      'obs_dim': (4,),
                      'optimizer_fn': {'kwargs': {}, 'name': 'Adam'},
                      'replay_buffer': {'kwargs': {'capacity': 1000,
                                                   'gamma': 0.99,
                                                   'observation_dtype': 'np.float64',
                                                   'observation_shape': (4,),
                                                   'stack_size': 1},
                                        'name': 'CircularReplayBuffer'},
                      'representation_net': {'kwargs': {'hidden_units': [256,
                                                                         256]},
                                             'name': 'MLPNetwork'},
                      'reward_clip': 1,
                      'target_net_update_schedule': {'kwargs': {'off_value': False,
                                                                'on_value': True,
                                                                'period': 100},
                                                     'name': 'PeriodicSchedule'},
                      'test_epsilon': 0.001,
                      'update_period_schedule': {'kwargs': {'off_value': False,
                                                            'on_value': True,
                                                            'period': 1},
                                                 'name': 'PeriodicSchedule'}},
           'name': 'DQNAgent'},
 'environment': {'kwargs': {'env_name': 'CartPole-v0'}, 'name': 'GymEnv'},
 'loggers': {'kwargs': {'logger_list': [{'kwargs': {}, 'name': 'ChompLogger'},
                                        {'kwargs': {'name': 'gym-dqn',
                                                    'project': 'Hive',
                                                    'resume': 'allow',
                                                    'start_method': 'fork'},
                                         'name': 'WandbLogger'}]},
             'name': 'CompositeLogger'},
 'max_steps_per_episode': 1000,
 'run_name': 'gym-dqn',
 'save_dir': 'experiment',
 'saving_schedule': {'kwargs': {'off_value': False,
                                'on_value': True,
                                'period': 10000},
                     'name': 'PeriodicSchedule'},
 'stack_size': 1,
 'test_episodes': 10,
 'test_frequency': 5000,
 'train_steps': 50000}
[INFO 2022-03-24 19:48:07,405 single_agent_loop.py:217] Created runner. Starting run!
[INFO 2022-03-24 19:48:07,405 base.py:130] Starting train loop
[INFO 2022-03-24 19:48:19,039 base.py:143] 5123/50000 training steps completed.
[INFO 2022-03-24 19:48:19,039 base.py:147] Running testing.
[INFO 2022-03-24 19:48:19,639 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:48:31,254 base.py:143] 10009/50000 training steps completed.
[INFO 2022-03-24 19:48:31,254 base.py:147] Running testing.
[INFO 2022-03-24 19:48:31,708 base.py:152] Testing results: {'0_episode_length': 217.4, '0_reward': 217.4, 'full_episode_length': 217.4}
[INFO 2022-03-24 19:48:31,708 base.py:156] Saving run.
[INFO 2022-03-24 19:48:31,709 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:48:42,861 base.py:143] 15013/50000 training steps completed.
[INFO 2022-03-24 19:48:42,861 base.py:147] Running testing.
[INFO 2022-03-24 19:48:43,242 base.py:152] Testing results: {'0_episode_length': 163.9, '0_reward': 163.9, 'full_episode_length': 163.9}
[INFO 2022-03-24 19:48:54,719 base.py:143] 20073/50000 training steps completed.
[INFO 2022-03-24 19:48:54,719 base.py:147] Running testing.
[INFO 2022-03-24 19:48:55,200 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:48:55,200 base.py:156] Saving run.
[INFO 2022-03-24 19:48:55,201 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:07,722 base.py:143] 25073/50000 training steps completed.
[INFO 2022-03-24 19:49:07,722 base.py:147] Running testing.
[INFO 2022-03-24 19:49:08,202 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:19,683 base.py:143] 30121/50000 training steps completed.
[INFO 2022-03-24 19:49:19,683 base.py:147] Running testing.
[INFO 2022-03-24 19:49:20,162 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:20,162 base.py:156] Saving run.
[INFO 2022-03-24 19:49:20,163 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:31,406 base.py:143] 35177/50000 training steps completed.
[INFO 2022-03-24 19:49:31,406 base.py:147] Running testing.
[INFO 2022-03-24 19:49:31,870 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:43,221 base.py:143] 40126/50000 training steps completed.
[INFO 2022-03-24 19:49:43,221 base.py:147] Running testing.
[INFO 2022-03-24 19:49:43,713 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:43,713 base.py:156] Saving run.
[INFO 2022-03-24 19:49:43,714 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:55,012 base.py:143] 45101/50000 training steps completed.
[INFO 2022-03-24 19:49:55,012 base.py:147] Running testing.
[INFO 2022-03-24 19:49:55,491 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:50:06,749 base.py:143] 50164/50000 training steps completed.
[INFO 2022-03-24 19:50:06,749 base.py:147] Running testing.
[INFO 2022-03-24 19:50:07,232 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:50:07,232 base.py:156] Saving run.
[INFO 2022-03-24 19:50:07,233 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:50:07,736 experiment.py:82] Saving the experiment at experiment/gym-dqn/current

wandb: Waiting for W&B process to finish, PID 456977... (success).
wandb:                                                                                
wandb: Run history:
wandb:                   0/epsilon █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
wandb:                0/train_loss ▁▁▁▁▁▁▁▃▂▁▁▃▃▁▁▃▄▄▁▁▄▁▁▁▁▁▄▄█▁▁▁▂▄▃▁▁▁▅█
wandb:                0/train_qval ▁▁▁▂▃▃▄▄▅▅▅▆▆▆▆▇▇▇▇▇▇▇██████████▇███████
wandb:                      0_step ▁▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▆▇▇▇▇▇███
wandb:       test/0_episode_length ██▁████████
wandb:               test/0_reward ██▁████████
wandb:    test/full_episode_length ██▁████████
wandb:                   test_step ▁▁▁▁▁▂▂▂▂▃▃▃▃▃▃▃▃▄▄▄▅▅▅▅▆▆▆▆▆▆▆▆▇▇▇▇████
wandb:      train/0_episode_length ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:              train/0_reward ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:   train/full_episode_length ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:                  train_step ▁▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▆▇▇▇▇▇███
wandb: 
wandb: Run summary:
wandb:                   0/epsilon 0.1
wandb:                0/train_loss 0.78445
wandb:                0/train_qval 101.70953
wandb:                      0_step 50164
wandb:       test/0_episode_length 220.0
wandb:               test/0_reward 220.0
wandb:    test/full_episode_length 220.0
wandb:                   test_step 11
wandb:      train/0_episode_length 200
wandb:              train/0_reward 200.0
wandb:   train/full_episode_length 200
wandb:                  train_step 50164
wandb: 
wandb: Synced 7 W&B file(s), 0 media file(s), 0 artifact file(s) and 1 other file(s)
wandb: Synced gym-dqn: https://wandb.ai/dapatil211/Hive/runs/323x6337
wandb: Find logs at: ./wandb/run-20220324_194800-323x6337/logs/debug.log
wandb:

Adds console logging to Hive. See attached output.

$ python -m hive.runners.single_agent_loop -p gym/dqn.yml
[INFO 2022-03-24 19:47:58,550 torch_utils.py:197] Registered PyTorch optimizers.
[INFO 2022-03-24 19:47:58,550 torch_utils.py:224] Registered PyTorch losses.
[INFO 2022-03-24 19:47:58,553 utils.py:142] Registered PyTorch initialization functions.
[INFO 2022-03-24 19:47:58,554 __init__.py:18] Registered function approximators.
[INFO 2022-03-24 19:47:58,557 __init__.py:18] Registered replays.
[INFO 2022-03-24 19:47:59,129 schedule.py:204] Registered schedules.
[INFO 2022-03-24 19:47:59,129 loggers.py:454] Registered loggers.
[INFO 2022-03-24 19:47:59,130 __init__.py:21] Registered agents.
[INFO 2022-03-24 19:47:59,633 __init__.py:46] Registered environments.
wandb: Currently logged in as: dapatil211 (use `wandb login --relogin` to force relogin)
wandb: wandb version 0.12.11 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.12.9
wandb: Syncing run gym-dqn
wandb:  View project at https://wandb.ai/dapatil211/Hive
wandb:  View run at https://wandb.ai/dapatil211/Hive/runs/323x6337
wandb: Run data is saved locally in /home/darshan/research/RLHive/wandb/run-20220324_194800-323x6337
wandb: Run `wandb offline` to turn off syncing.
[INFO 2022-03-24 19:48:07,404 single_agent_loop.py:214] Using config: 
{'agent': {'kwargs': {'act_dim': 2,
                      'batch_size': 128,
                      'device': 'cuda',
                      'discount_rate': 0.99,
                      'epsilon_schedule': {'kwargs': {'value': 0.1},
                                           'name': 'ConstantSchedule'},
                      'log_frequency': 100,
                      'logger': {},
                      'loss_fn': {'kwargs': {}, 'name': 'SmoothL1Loss'},
                      'min_replay_history': 500,
                      'obs_dim': (4,),
                      'optimizer_fn': {'kwargs': {}, 'name': 'Adam'},
                      'replay_buffer': {'kwargs': {'capacity': 1000,
                                                   'gamma': 0.99,
                                                   'observation_dtype': 'np.float64',
                                                   'observation_shape': (4,),
                                                   'stack_size': 1},
                                        'name': 'CircularReplayBuffer'},
                      'representation_net': {'kwargs': {'hidden_units': [256,
                                                                         256]},
                                             'name': 'MLPNetwork'},
                      'reward_clip': 1,
                      'target_net_update_schedule': {'kwargs': {'off_value': False,
                                                                'on_value': True,
                                                                'period': 100},
                                                     'name': 'PeriodicSchedule'},
                      'test_epsilon': 0.001,
                      'update_period_schedule': {'kwargs': {'off_value': False,
                                                            'on_value': True,
                                                            'period': 1},
                                                 'name': 'PeriodicSchedule'}},
           'name': 'DQNAgent'},
 'environment': {'kwargs': {'env_name': 'CartPole-v0'}, 'name': 'GymEnv'},
 'loggers': {'kwargs': {'logger_list': [{'kwargs': {}, 'name': 'ChompLogger'},
                                        {'kwargs': {'name': 'gym-dqn',
                                                    'project': 'Hive',
                                                    'resume': 'allow',
                                                    'start_method': 'fork'},
                                         'name': 'WandbLogger'}]},
             'name': 'CompositeLogger'},
 'max_steps_per_episode': 1000,
 'run_name': 'gym-dqn',
 'save_dir': 'experiment',
 'saving_schedule': {'kwargs': {'off_value': False,
                                'on_value': True,
                                'period': 10000},
                     'name': 'PeriodicSchedule'},
 'stack_size': 1,
 'test_episodes': 10,
 'test_frequency': 5000,
 'train_steps': 50000}
[INFO 2022-03-24 19:48:07,405 single_agent_loop.py:217] Created runner. Starting run!
[INFO 2022-03-24 19:48:07,405 base.py:130] Starting train loop
[INFO 2022-03-24 19:48:19,039 base.py:143] 5123/50000 training steps completed.
[INFO 2022-03-24 19:48:19,039 base.py:147] Running testing.
[INFO 2022-03-24 19:48:19,639 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:48:31,254 base.py:143] 10009/50000 training steps completed.
[INFO 2022-03-24 19:48:31,254 base.py:147] Running testing.
[INFO 2022-03-24 19:48:31,708 base.py:152] Testing results: {'0_episode_length': 217.4, '0_reward': 217.4, 'full_episode_length': 217.4}
[INFO 2022-03-24 19:48:31,708 base.py:156] Saving run.
[INFO 2022-03-24 19:48:31,709 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:48:42,861 base.py:143] 15013/50000 training steps completed.
[INFO 2022-03-24 19:48:42,861 base.py:147] Running testing.
[INFO 2022-03-24 19:48:43,242 base.py:152] Testing results: {'0_episode_length': 163.9, '0_reward': 163.9, 'full_episode_length': 163.9}
[INFO 2022-03-24 19:48:54,719 base.py:143] 20073/50000 training steps completed.
[INFO 2022-03-24 19:48:54,719 base.py:147] Running testing.
[INFO 2022-03-24 19:48:55,200 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:48:55,200 base.py:156] Saving run.
[INFO 2022-03-24 19:48:55,201 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:07,722 base.py:143] 25073/50000 training steps completed.
[INFO 2022-03-24 19:49:07,722 base.py:147] Running testing.
[INFO 2022-03-24 19:49:08,202 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:19,683 base.py:143] 30121/50000 training steps completed.
[INFO 2022-03-24 19:49:19,683 base.py:147] Running testing.
[INFO 2022-03-24 19:49:20,162 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:20,162 base.py:156] Saving run.
[INFO 2022-03-24 19:49:20,163 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:31,406 base.py:143] 35177/50000 training steps completed.
[INFO 2022-03-24 19:49:31,406 base.py:147] Running testing.
[INFO 2022-03-24 19:49:31,870 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:43,221 base.py:143] 40126/50000 training steps completed.
[INFO 2022-03-24 19:49:43,221 base.py:147] Running testing.
[INFO 2022-03-24 19:49:43,713 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:49:43,713 base.py:156] Saving run.
[INFO 2022-03-24 19:49:43,714 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:49:55,012 base.py:143] 45101/50000 training steps completed.
[INFO 2022-03-24 19:49:55,012 base.py:147] Running testing.
[INFO 2022-03-24 19:49:55,491 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:50:06,749 base.py:143] 50164/50000 training steps completed.
[INFO 2022-03-24 19:50:06,749 base.py:147] Running testing.
[INFO 2022-03-24 19:50:07,232 base.py:152] Testing results: {'0_episode_length': 220.0, '0_reward': 220.0, 'full_episode_length': 220.0}
[INFO 2022-03-24 19:50:07,232 base.py:156] Saving run.
[INFO 2022-03-24 19:50:07,233 experiment.py:82] Saving the experiment at experiment/gym-dqn/current
[INFO 2022-03-24 19:50:07,736 experiment.py:82] Saving the experiment at experiment/gym-dqn/current

wandb: Waiting for W&B process to finish, PID 456977... (success).
wandb:                                                                                
wandb: Run history:
wandb:                   0/epsilon █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
wandb:                0/train_loss ▁▁▁▁▁▁▁▃▂▁▁▃▃▁▁▃▄▄▁▁▄▁▁▁▁▁▄▄█▁▁▁▂▄▃▁▁▁▅█
wandb:                0/train_qval ▁▁▁▂▃▃▄▄▅▅▅▆▆▆▆▇▇▇▇▇▇▇██████████▇███████
wandb:                      0_step ▁▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▆▇▇▇▇▇███
wandb:       test/0_episode_length ██▁████████
wandb:               test/0_reward ██▁████████
wandb:    test/full_episode_length ██▁████████
wandb:                   test_step ▁▁▁▁▁▂▂▂▂▃▃▃▃▃▃▃▃▄▄▄▅▅▅▅▆▆▆▆▆▆▆▆▇▇▇▇████
wandb:      train/0_episode_length ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:              train/0_reward ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:   train/full_episode_length ▁▂▁▂▁▅▆▇███▂▅███▇█████████████████▆█████
wandb:                  train_step ▁▁▁▁▂▂▂▂▂▂▃▃▃▃▃▄▄▄▄▄▅▅▅▅▅▅▆▆▆▆▆▆▇▇▇▇▇███
wandb: 
wandb: Run summary:
wandb:                   0/epsilon 0.1
wandb:                0/train_loss 0.78445
wandb:                0/train_qval 101.70953
wandb:                      0_step 50164
wandb:       test/0_episode_length 220.0
wandb:               test/0_reward 220.0
wandb:    test/full_episode_length 220.0
wandb:                   test_step 11
wandb:      train/0_episode_length 200
wandb:              train/0_reward 200.0
wandb:   train/full_episode_length 200
wandb:                  train_step 50164
wandb: 
wandb: Synced 7 W&B file(s), 0 media file(s), 0 artifact file(s) and 1 other file(s)
wandb: Synced gym-dqn: https://wandb.ai/dapatil211/Hive/runs/323x6337
wandb: Find logs at: ./wandb/run-20220324_194800-323x6337/logs/debug.log
wandb:

Instead of adding logging commands everywhere, can it be shifted to a logger class like the Chomp and Wandb loggers. It can be very similar to here https://github.com/uoe-agents/epymarl/blob/main/src/utils/logging.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hnekoeiq hnekoeiq Awaiting requested review from hnekoeiq

@alirahkay alirahkay Awaiting requested review from alirahkay

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dapatil211 @sriyash421