Question about the eval function

[whtitefall]

Hi,

I have a question about the eval function in chem/finetune.py, I list the code that I don't understand below,

    y_true = torch.cat(y_true, dim = 0).cpu().numpy()
    y_scores = torch.cat(y_scores, dim = 0).cpu().numpy()

    roc_list = []
    for i in range(y_true.shape[1]):
        #AUC is only defined when there is at least one positive data.
        if np.sum(y_true[:,i] == 1) > 0 and np.sum(y_true[:,i] == -1) > 0:
            is_valid = y_true[:,i]**2 > 0
            roc_list.append(roc_auc_score((y_true[is_valid,i] + 1)/2, y_scores[is_valid,i]))

I have the following questions,

1. I don't understand that why are we looping over the columns instead of the rows of y_true
2. I understand that AUC is defined when there's at least one positive data, but why we need to use np.sum(y_true[:,i] == -1) > 0 here, does -1 represents positive data also ?
3. what's the reason that we need to + 1 and /2 for the valid y_true here?

Sorry for my long questions and thank you for your time!

[weihua916]

Re 1: It’s because each column represents a task.

Re 2: 1 represents positive, -1 represents negative, 0 represents unlabeled. ROC-AUC is defined only when there is at least one positive and negative data.

Re 3: Because we convert -1 (negative) and +1 (positive) into 0 and 1, respectively.