Negative total loss

[za13]

I'm doing segmentation on images with 3 classes, instead of 2. So I changed

dice_loss = sm.losses.DiceLoss(class_weights=np.array([1, 2, 0.5]))

to

dice_loss = sm.losses.DiceLoss(class_weights=np.array([1, 2, 3, 0.5]))

I've noticed that although the validation losses have stayed positive and are decreasing, the training losses went from positive and decreased to negative and continue to be more negative:

Epoch 50/300
9/9 [==============================] - 1171s 130s/step - loss: 0.0156 - iou_score: 0.6439 - f1-score: 0.6901 - val_loss: 0.1600 - val_iou_score: 0.6910 - val_f1-score: 0.7310
Epoch 51/300
9/9 [==============================] - 1173s 130s/step - loss: 0.0022 - iou_score: 0.6491 - f1-score: 0.6941 - val_loss: 0.1517 - val_iou_score: 0.6811 - val_f1-score: 0.7224
Epoch 52/300
9/9 [==============================] - 1184s 132s/step - loss: -0.0077 - iou_score: 0.6527 - f1-score: 0.6963 - val_loss: 0.1413 - val_iou_score: 0.6990 - val_f1-score: 0.7380
Epoch 53/300
9/9 [==============================] - 1176s 131s/step - loss: -0.0087 - iou_score: 0.6476 - f1-score: 0.6930 - val_loss: 0.1337 - val_iou_score: 0.7204 - val_f1-score: 0.7594
Epoch 54/300
9/9 [==============================] - 1166s 130s/step - loss: -0.0052 - iou_score: 0.6516 - f1-score: 0.6959 - val_loss: 0.1294 - val_iou_score: 0.7034 - val_f1-score: 0.7439

Despite the negative training loss, the predictions from the test set continue to improve as the epoch number increases

I already checked that the masks are in 0-1 instead of 0-255

Is this ok? Or is this a sign that there's something wrong with the code or my setup?

[qubvel]

It is ok, because you provide weights larger than 1. You can norm weight to make loss stay positive, but it does not necessary.

P.S. be careful with loss absolute value, fairly I recommend to set weights in range (0..1)

[za13]

You mean instead of class_weights=np.array([1, 2, 3, 0.5]), change it to something like class_weights=np.array([0.25,0.25,0.25,0.25])?

[qubvel]

First of all, weights are optional, you can not provide them at all (automatically will be set all weights to 1). But if you would like to weight classes - yes make something like np.array([0.1, 0.5, 0.2, 0.05])

[qubvel]

I have just check losses code and it should not be negative with any weights, if masks are in range (0..1). Check your code again)

[za13]

I confirmed that the masks are between 0-1 by running the following code just before history=model.fit_generator(...:

for i in range(0,35):
    image, mask = train_dataset[i]
    print(np.max(mask[..., 0]))
    print(np.max(mask[..., 1]))
    print(np.max(mask[..., 2]))
    for j in range(0,mask.shape[0]):
        for k in range(0,mask.shape[1]):
            if (mask[j,k, 0]!=0.0 and mask[j,k, 0]!=1.0):
                print("WRONG Value - mask[j,k, 0]: ",mask[j,k, 0])
            elif (mask[j,k, 1]!=0.0 and mask[j,k, 1]!=1.0):
                print("WRONG Value - mask[j,k, 1]: ",mask[j,k, 1])
            elif (mask[j,k, 2]!=0.0 and mask[j,k, 2]!=1.0):
                print("WRONG Value - mask[j,k, 2]: ",mask[j,k, 2])
        
    
print("valid")
for i in range(0,35):
    image, mask = valid_dataset[i]
    print(np.max(mask[..., 0]))
    print(np.max(mask[..., 1]))
    print(np.max(mask[..., 2]))
    for j in range(0,mask.shape[0]):
        for k in range(0,mask.shape[1]):
            if (mask[j,k, 0]!=0.0 and mask[j,k, 0]!=1.0):
                print("WRONG Value - mask[j,k, 0]: ",mask[j,k, 0])
            elif (mask[j,k, 1]!=0.0 and mask[j,k, 1]!=1.0):
                print("WRONG Value - mask[j,k, 1]: ",mask[j,k, 1])
            elif (mask[j,k, 2]!=0.0 and mask[j,k, 2]!=1.0):
                print("WRONG Value - mask[j,k, 2]: ",mask[j,k, 2])

The code I used for this is attached. Note that I made some slight changes to Dataset so it could perform multiple augmentations for a given image. I set nAug=40, which allowed me to generate 40 different augmentations per image

errorWeights.txt

[jovahe]

@qubvel
It can be as loss = sm.losses.DiceLoss(class_weights=[65.0,2.0,1.0,14.0,141.0,24.0]), but not loss = sm.losses.dice_loss(class_weights=[65.0,2.0,1.0,14.0,141.0,24.0]).
How can I used sm.losses.dice_loss(or cce_dice_loss) with class_weights?

[qubvel]

@jovahe
sm.losses.dice_loss is simply instance of sm.losses.DiceLoss()
so you cant doing that
Use DiceLoss class to create custom loss instance with weights/etc. parameters

[jovahe]

@qubvel OK，thanks

[hello1010101010101world]

@za13 @qubvel Hello！I hava the same problem when I set classweight[1.5897457025300863, 19.558464414520287, 32.333179824234826, 13.486144037069105, 0.0, 17.706329452750897, 40.07563246575418].My loss is total_loss = dice_loss + (1 * focal_loss). I am sure my mask is between 0 or 1.The classweight calculation method is:
def ENet_weighting(image_files=None, num_classes=10):
'''
The custom class weighing function as seen in the ENet paper.
INPUTS:
- image_files(list): a list of image_filenames which element can be read immediately
OUTPUTS:
- class_weights(list): a list of class weights where each index represents each class label and the element is the class weight for that label.
'''
# initialize dictionary with all 0
label_to_frequency = {}
for i in range(num_classes):
label_to_frequency[i] = 0

for n in range(len(image_files)):
    image, img_width, img_height, img_bands, img_geotrans, img_pro = readTif(image_files[n])

    # For each label in each image, sum up the frequency of the label and add it to label_to_frequency dict
    for i in range(num_classes):
        class_mask = np.equal(image, i)
        class_mask = class_mask.astype(np.float32)
        class_frequency = np.sum(class_mask)

        label_to_frequency[i] += class_frequency

# perform the weighing function label-wise and append the label's class weights to class_weights
class_weights = []
total_frequency = sum(label_to_frequency.values())
for label, frequency in label_to_frequency.items():
    class_weight = 1 / np.log(1.02 + (frequency / total_frequency))
    class_weights.append(class_weight)
#bike class has no sample but I will create ,so I set class_weights[4] = 0.0
class_weights[4] = 0.0
return class_weights

So,Have you solved your problem?@za13 And what is your suggestion? @qubvel . Thank you!!!