poor performance of diabetes prediction

[Andy-jqa]

Why is the performance of using 07_diabetes_logistic.py to predict diabetes pretty poor?

[kopxiong]

I guess it's due to the optimizer setting, since I modified the update rule to:
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)
the loss decreases dramatically.

[Andy-jqa]

@kopxiong It makes sense to change the hyperparameters. I've tried your suggestion, but the loss didn't drop to a satisfying level. Have you checked the y_pred of the final model?

[hunkim]

@kopxiong Any thoughts? Since the goal of these tutorials is understanding the concept, I did not pay attention to the accuracy.

[kopxiong]

@Andy-jqa @hunkim Sorry for the late reply. I think we can improve the model's performance from the following aspects:

1. add more layers or increase the number of nodes in each layer, since more units mean more powerful representation capability, like
   super(Model, self).__init__()
self.l1 = torch.nn.Linear(8, 32) #(8, 6)
self.l2 = torch.nn.Linear(32, 16) #(6, 4)
self.l3 = torch.nn.Linear(16, 1) #(4, 1)
2. use Adam optimizer to RMSProp instead of SGD
3. use more epochs for training (like 2000 or more)
4. sigmoid activation function maybe not good, try ReLU or LeakyReLU (I tried ReLU, but didn't work)

maybe we also should add some regularization terms to avoid overfitting.

[akramsystems]

set lr = 8 and epoch i set to 10000
got around 2% error

[akramsystems]

also made more nodes in the hidden layer so i set my layers to
torch.nn.Linear(8, 30)
torch.nn.Linear(30, 10)
torch.nn.Linear(10, 1)

[elcolie]

I can not get loss below than 0.6
Any ideas?

file.zip