Cannot converge on image compression task

[dwgan]

Hi,

This work looks great.

I was trying to implement it on image compression task. I use this model as follows, and the input shape is 16, 32, 96, 96.

    def forward(self, x):
        b, c, h, w = x.shape
        x = x.reshape(b, c, -1).permute(0, 2, 1)
        x = self.act1(x)
        x = self.drop1(x)
        x = self.fc1(x)
        x = self.act2(x)
        x = self.drop2(x)
        x = self.fc2(x)
        x = x.permute(0, 2, 1).reshape(b, c, h, w)
        return x

However, my model cannot converge during training.

Training:  75%|██████████████                    | 1171/1563 [02:38<00:53,  7.36it/s, loss=nan]

Could you please give me some advice?

Thanks you very much.

[truong04]

I face the same problem in NLP, where my model loss does not decrease

[dwgan]

@truong04 I finally found that the problem was caused by using the wrong function. We should use KAT_Group2D to process the image signal, otherwise it cannot converge.