20% Speed-up by Singleton Parameter Class in PyTorch

[mikigom]

I fixed some codes for avoiding re-creation of repeated used numpy array or Pytorch Tensor.
Used singleton-type class of some fixed warp-related parameters.

I observed about 20% speed-up of training.

stdout of original code training

======= TRAINING START =======
start training...
it. 100/500000  lr=1e-04(GP),1e-02(C), loss=2.2995, time=2.53
it. 200/500000  lr=1e-04(GP),1e-02(C), loss=2.3195, time=4.95
it. 300/500000  lr=1e-04(GP),1e-02(C), loss=2.3041, time=7.38
it. 400/500000  lr=1e-04(GP),1e-02(C), loss=2.2965, time=9.81
it. 500/500000  lr=1e-04(GP),1e-02(C), loss=2.3205, time=12.24
it. 600/500000  lr=1e-04(GP),1e-02(C), loss=2.3168, time=14.66
it. 700/500000  lr=1e-04(GP),1e-02(C), loss=2.2857, time=17.09
it. 800/500000  lr=1e-04(GP),1e-02(C), loss=2.3116, time=19.57
it. 900/500000  lr=1e-04(GP),1e-02(C), loss=2.3089, time=22.00
it. 1000/500000  lr=1e-04(GP),1e-02(C), loss=2.2762, time=24.43
it. 1100/500000  lr=1e-04(GP),1e-02(C), loss=2.2667, time=28.92
it. 1200/500000  lr=1e-04(GP),1e-02(C), loss=2.2369, time=31.35
it. 1300/500000  lr=1e-04(GP),1e-02(C), loss=2.2433, time=33.79
it. 1400/500000  lr=1e-04(GP),1e-02(C), loss=2.2285, time=36.23
it. 1500/500000  lr=1e-04(GP),1e-02(C), loss=2.1207, time=38.69
it. 1600/500000  lr=1e-04(GP),1e-02(C), loss=2.0533, time=41.12
it. 1700/500000  lr=1e-04(GP),1e-02(C), loss=2.0706, time=43.58
it. 1800/500000  lr=1e-04(GP),1e-02(C), loss=1.9727, time=46.02
it. 1900/500000  lr=1e-04(GP),1e-02(C), loss=1.7569, time=48.48
it. 2000/500000  lr=1e-04(GP),1e-02(C), loss=1.6987, time=50.92

stdout of fixed code training

======= TRAINING START =======
start training...
it. 100/500000  lr=1e-04(GP),1e-02(C), loss=2.3077, time=2.02
it. 200/500000  lr=1e-04(GP),1e-02(C), loss=2.2855, time=3.94
it. 300/500000  lr=1e-04(GP),1e-02(C), loss=2.2927, time=5.86
it. 400/500000  lr=1e-04(GP),1e-02(C), loss=2.3020, time=7.78
it. 500/500000  lr=1e-04(GP),1e-02(C), loss=2.2870, time=9.70
it. 600/500000  lr=1e-04(GP),1e-02(C), loss=2.2665, time=11.61
it. 700/500000  lr=1e-04(GP),1e-02(C), loss=2.2809, time=13.53
it. 800/500000  lr=1e-04(GP),1e-02(C), loss=2.1885, time=15.45
it. 900/500000  lr=1e-04(GP),1e-02(C), loss=2.2466, time=17.37
it. 1000/500000  lr=1e-04(GP),1e-02(C), loss=2.1404, time=19.28
it. 1100/500000  lr=1e-04(GP),1e-02(C), loss=2.2226, time=22.70
it. 1200/500000  lr=1e-04(GP),1e-02(C), loss=2.1020, time=24.63
it. 1300/500000  lr=1e-04(GP),1e-02(C), loss=2.1083, time=26.55
it. 1400/500000  lr=1e-04(GP),1e-02(C), loss=1.9734, time=28.47
it. 1500/500000  lr=1e-04(GP),1e-02(C), loss=1.9845, time=30.40
it. 1600/500000  lr=1e-04(GP),1e-02(C), loss=2.0275, time=32.32
it. 1700/500000  lr=1e-04(GP),1e-02(C), loss=1.9226, time=34.25
it. 1800/500000  lr=1e-04(GP),1e-02(C), loss=1.8156, time=36.17
it. 1900/500000  lr=1e-04(GP),1e-02(C), loss=1.8724, time=38.10
it. 2000/500000  lr=1e-04(GP),1e-02(C), loss=1.6415, time=40.02