How to properly use class_weights and class_indexes?

[thiagoribeirodamotta]

Hello,

I'm having a hard time understanding how to use the class_weights and class_indexes parameters. As per the documentation:

class_weights: Array (np.array) of class weights (len(weights) = num_classes).
class_indexes: Optional integer or list of integers, classes to consider, if None all classes are used.

Can class_weights values be greater than one? I've worked with values up to 1600, but quickly noticed that the scores weren't between [0,1] anymore, so it seemed a wiser choice to make those values sum 1 at the end of the day, just to make comparisons clearer. But I'm not really sure if there are any performance issues or if the algorithm behaves badly with those values.

These are the values I'm currently working with:
Weights: [1, 1, 1, 1, 1, 1, 1, 0.05]
Indexes: [1, 11, 12, 17, 18, 19, 22, 255] (each representing a class_ID)

What this means is that I don't want my network to pay attention to the background (ID = 255, weight = 0.05), thus making all the real structures way more important than the background itself. I can't just not label the background since Unet requires all pixels to be labeled.

What I've noticed though is that the Indexes order stated above is not necessarily the same as what the model is using (I think the model has them sorted from lowest to highest). Should they match? Or is there any matching being done inside the model.compile call?

How does the class_indexes arguments actually works? Will it just ignore a class if its index is not listed or will it pass a class_weight of 0 to it?

What happens if I just don't fill one of the weights value or pass 0 to it? Will it have the same behavior as not passing a index to class_indexes?

After further tests, I noticed that playing with class_weights and class_indexes values always hits a local minima of 0.1250 (or 1/8, where 8 is the number of classes I have at the moment) without ever being able to exit it (setting EarlyStopping to 15 and min_delta to 0.0001). This happened both at the very beggining of training and at the very end of training. When not setting either parameters, this behavior never happens. Setting Indexes to [0, 1, 2, 3, 4, 5, 6] (excluding the last one on purpose) also makes it get stuck on a local minima of value 0.0156, happening both with and without class_weights being set, independent of the weight value set, either on the beggining of training or at the end of it. Everytime when a local minima is reached and the algorithm gets stuck on it, the val_iou_score was about 0.4, so a substancial drop was perceived.

After even more tests, it seems that using either just Categorial Crossentropy OR Jaccard Loss, instead of both at the same time, removed the local minima stuck cases completly.

Thank you for all the effort done in this project. This and the other ones are nothing short of amazing!

[qubvel]

Hi @thrmotta

1. class_indexes behaviour
   You have to specify which classes to use to calculate score. e.g. you have tensor with shape (1, 32, 32, 15) as ground truth mask (same as prediction out of model) if you set class_indexes = np.array([0, 1, 2]) score will be calculated over a tensor with shape (1, 32, 32, 3) - first three channels of tensors, so basically this argument represent channel numbers.

loss(gt, pr, class_index=np.array([0, 1, 2])) 
loss(gt[..., :3], pr[..., :3]) # the same

2. class_weights is just multiplication factor for loss/score components

[sumanttyagi]

----> 9 loss1 = categorical_crossentropy(class_weights = classwt1)

TypeError: call() got an unexpected keyword argument 'class_weights'

how to deal with it

[JarvisSan22]

@sumanttyagi had the same issue today.
Check the losses.py and the weights are values of the classes not the function what are set at the end of the file.
So basicaly you need to import the class , for categorical_crosstropy thats
from segmentation_models.losses import CategoricalCELoss

https://github.com/qubvel/segmentation_models/blob/94f624b7029deb463c859efbd92fa26f512b52b8/segmentation_models/losses.py