performance gap of HRNetV2+OCR on cityscape val set using default config #91
Comments
|
Hi. Would you please share your training log? We want to make sure if there's any difference. |
|
@hsfzxjy 2020-01-23 08:38:18,729 Namespace(cfg='experiments/cityscapes/seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml', local_rank=3, opts=[], seed=304) |
|
@verymadmatt We perform class-balance for all the experiments on Cityscapes. We only use the Cityscapes train set for training. First, could you provide more details about your environmental information? We expect you to conduct experiments with Pytorch1.1. Second, please ensure that you have reproduced the performance of the HRNet baseline. |
|
@PkuRainBow Thanks for your reply. I'm using python 3.6, pytorch 1.1 and 4 P100 GPUs. Others just followed the 'requirements.txt' file. The mIoU increased to 0.8064 when i turned class_balance on, but still 1% lower than the reported. |
|
@hsfzxjy Please check the possible reasons. |
|
@hsfzxjy @PkuRainBow |
|
@verymadmatt Please be patient. We will check the possible problems and reply to you latter. |
hello, which version of cuda are you using? |
8.0. Which version should I use? |
I don`t know, I encountered the same problem as you。I just want to confirm whether our environment is consistent。 |
|
@verymadmatt There might be some bugs in the pushed code and @hsfzxjy will check the possible reasons and update the progress soon. Please be patient. |
|
@verymadmatt We recommend you to try our "HRNet + OCR" on the other two datasets including PASCAL-Context and LIP. |
Thanks for your response. So there're no bugs in the current release? |
|
@verymadmatt Yes, the performance on Cityscapes is not very stable and we recommend you to run multiple times currently. The performance on the other datasets is expected to be more stable. |
I read the code. But it seems that the parameter of class-balance is useless. In the 200th row of train.py, criterion = CrossEntropy(ignore_label=config.TRAIN.IGNORE_LABEL, |
|
@purse1996 I could not quite understand what do you mean by "useless". The code snippet you post just shows that we are using class balance. |
|
Sorry, I did not express clearly what I mean. Of course, class weight is useful. While no matter that LOSS.CLASS_BALANCE is True or False, class-balance weights are always in use in criterion = CrossEntropy(ignore_label=config.TRAIN.IGNORE_LABEL, |
|
The config multi-scale and flip set as NO both in training and testing
|
Hi, I'm trying to replicate the performance listed on the project page "HRNetV2-W48 + OCR val mIoU 81.6" on cityscape val set using the config file provided, i.e., "seg_hrnet_ocr_w48_train_512x1024_sgd_lr1e-2_wd5e-4_bs_12_epoch484.yaml". However, I can only get "Best_mIoU: 0.8033" which is ~1.3% lower than reported. Just wondering if there is any config setting I missed or extra train data was used in the reported result. Any help will be much appreciated. I noticed a previous open issue about class balance setting for the performance gap on cityscape using HRNetV2. Not sure if it is related. #67
The text was updated successfully, but these errors were encountered: