float tensor pass into int*

[ENXING]

Code

venom/end2end/gpt2_pytorch.py

Line 74 in 8ddaf38

mask = torch.zeros(tensor.shape, dtype=tensor.dtype)

input tensor is float, so mask is float type. but "SrNMTensor" initialization with parameters "mask" and call to_sparse_sr_nm

venom/end2end/grouped_nmv_tensor.py

Line 692 in 8ddaf38

self.to_sparse_sr_nm(dense_.cpu().to(dtype=torch.float32), mask_)

And it will call c function with mask int type

venom/end2end/grouped_nmv_tensor.py

Line 529 in 8ddaf38

extern "C" void func2({dtype}* sparse, int* masks, {dtype}* hA_values, int *hA_columns, int *hA_metadata){{

this is a bug.

[LopezCastroRoberto]

Thanks for reporting this! This may be a typo that was introduced in code cleaning and refactoring. I will take a look at it.

[ENXING]

@LopezCastroRoberto Thanks. I would appreciate if you can add a validation between sparse matrix multiplication in torch and calling your library in the sparse_dense_mul_dispatch. I find they don't match.

Also, Do I understand correctly: Weight with mask is the equivalent as your sparse matrix representation? I feel confused why there is this line if that is true, it uses ha_column to set column index instead of mask matrix alone. I expect HA_column and ha_value, ha_meta are output matrix, but here HA_column is input.

venom/end2end/grouped_nmv_tensor.py

Line 555 in 8ddaf38

columns[col_i*{m_fixed} + col_ii] =

venom/end2end/grouped_nmv_tensor.py

Line 556 in 8ddaf38

hA_columns[bm_i*col_k*col_kk*{m_fixed} + mcol_i*col_kk*{m_fixed} + col_i*{m_fixed} + col_ii];

[p81sunshine]

Hi, ENXING.
I have been carefully reviewing this code recently. Do you have any contact information? Maybe we can discuss it in detail.