[ONNX Importer] Incorrect return type (!torch.none) for `QLinear*` ops from "com.microsoft" domain

[pravg-amd]

Some of the onnx zoo models have" QLinearAdd" op which results in return type !torch.none on importing the model.

  %88 = torch.operator "onnx.QLinearAdd"(%87, %43, %42, %46, %47, %48, %45, %44) : (!torch.vtensor<[1,4096],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[4096],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -> !torch.none

Steps to reproduce:

Build SHARK-TestSuite by following the steps below

https://github.com/nod-ai/SHARK-TestSuite/tree/main/alt_e2eshark

Run the following command

python run.py -va -t bvlcalexnet-12-int8

The model will be available test-run/bvlcalexnet-12-int8/

bvlcalexnet-12-int8
squeezenet1.0-12-int8
caffenet-12-int8
densenet-12-int8
vgg16-12-int8
mobilenetv2-12-int8

Updated List*:

FasterRCNN-12-int8
MaskRCNN-12-int8
arcfaceresnet100-11-int8
bvlcalexnet-12-int8
caffenet-12-int8
efficientnet-lite4-11-int8
fcn-resnet50-12-int8
googlenet-12-int8
mnist-12-int8
mobilenetv2-12-int8
resnet50-v1-12-int8
shufflenet-v2-12-int8
ssd-12-int8
vgg16-12-int8
zfnet512-12-int8

[pravg-amd]

This op is not part of https://onnx.ai/onnx/operators/

doc: https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#com.microsoft.QLinearAdd

More information regarding this: onnx/onnx#5895

[pravg-amd]

ONNX Runtime ticket to track the support -> microsoft/onnxruntime#24028

[pravg-amd]

ORT changes are merged . Working on supporting the op using the ort shape inference

[pravg-amd]

QLinearLeakyRelu - Model : yolov3-12-int8

 %673 = torch.operator "onnx.QLinearLeakyRelu"(%671, %43, %42, %41, %40) {torch.onnx.alpha = 1.000000e-01 : f32} : (!torch.vtensor<[?,32,?,?],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -> !torch.none

[pravg-amd]

QLinearConcat - Model : version-RFB-320-int8

    %358 = torch.operator "onnx.QLinearConcat"(%158, %157, %356, %149, %148, %355, %143, %142, %357, %155, %154) {torch.onnx.axis = 1 : si64} : (!torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[1,16,30,40],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[1,16,30,40],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[1,16,30,40],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -> !torch.none

[pravg-amd]

onnx.QLinearGlobalAveragePool - Model: squeezenet1.0-12-int8

%238 = torch.operator "onnx.QLinearGlobalAveragePool"(%237, %173, %172, %175, %174) {torch.onnx.channels_last = 0 : si64} : (!torch.vtensor<[1,1000,13,13],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -
> !torch.none

[pravg-amd]

@vivekkhandelwal1 IR with ort shape inference for QLinearAdd

func.func @test_qlinearadd(%arg0: !torch.vtensor<[1,4096],ui8>, %arg1: !torch.vtensor<[],f32>, %arg2: !torch.vtensor<[],ui8>, %arg3: !torch.vtensor<[4096],ui8>, %arg4: !torch.vtensor<[],f32>, %arg5: !torch.vtensor<[],ui8>, %arg6: !torch.vtensor<[],f32>, %arg7: !torch.vtensor<[],ui8>) -> !torch.vtensor<[1,4096],ui8> attributes {torch.onnx_meta.ir_version = 5 : si64, torch.onnx_meta.opset_version = 10 : si64, torch.onnx_meta.producer_name = "backend-test", torch.onnx_meta.producer_version = ""} {
  %0 = torch.operator "onnx.QLinearAdd"(%arg0, %arg1, %arg2, %arg3, %arg4, %arg5, %arg6, %arg7) : (!torch.vtensor<[1,4096],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[4096],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -> !torch.vtensor<[1,4096],ui8>
  return %0 : !torch.vtensor<[1,4096],ui8>
}

[vivekkhandelwal1]

@vivekkhandelwal1 IR with ort shape inference for QLinearAdd

func.func @test_qlinearadd(%arg0: !torch.vtensor<[1,4096],ui8>, %arg1: !torch.vtensor<[],f32>, %arg2: !torch.vtensor<[],ui8>, %arg3: !torch.vtensor<[4096],ui8>, %arg4: !torch.vtensor<[],f32>, %arg5: !torch.vtensor<[],ui8>, %arg6: !torch.vtensor<[],f32>, %arg7: !torch.vtensor<[],ui8>) -> !torch.vtensor<[1,4096],ui8> attributes {torch.onnx_meta.ir_version = 5 : si64, torch.onnx_meta.opset_version = 10 : si64, torch.onnx_meta.producer_name = "backend-test", torch.onnx_meta.producer_version = ""} {
  %0 = torch.operator "onnx.QLinearAdd"(%arg0, %arg1, %arg2, %arg3, %arg4, %arg5, %arg6, %arg7) : (!torch.vtensor<[1,4096],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[4096],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -> !torch.vtensor<[1,4096],ui8>
  return %0 : !torch.vtensor<[1,4096],ui8>
}

The lowering for the QLinearAdd op is added here: llvm/torch-mlir#4113

[vivekkhandelwal1]

Hi @pravg-amd, can you please add the repro IRs for other ops?

[pravg-amd]

QLinearLeakyRelu

func.func @test_qlinearleakyrelu(%arg0: !torch.vtensor<[?,32,?,?],ui8>, %arg1: !torch.vtensor<[],f32>, %arg2: !torch.vtensor<[],ui8>, %arg3: !torch.vtensor<[],f32>, %arg4: !torch.vtensor<[],ui8>) -> !torch.vtensor<[?,32,?,?],ui8> attributes {torch.onnx_meta.ir_version = 5 : si64, torch.onnx_meta.opset_version = 10 : si64, torch.onnx_meta.producer_name = "backend-test", torch.onnx_meta.producer_version = ""} {
  %0 = torch.operator "onnx.QLinearLeakyRelu"(%arg0, %arg1, %arg2, %arg3, %arg4) {torch.onnx.alpha = 1.000000e-01 : f32} : (!torch.vtensor<[?,32,?,?],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -> !torch.vtensor<[?,32,?,?],ui8>
  return %0 : !torch.vtensor<[?,32,?,?],ui8>
}

[pravg-amd]

QLinearConcat

func.func @test_qlinearconcat(%arg0: !torch.vtensor<[],f32>, %arg1: !torch.vtensor<[],ui8>, %arg2: !torch.vtensor<[?,?,?,?],ui8>, %arg3: !torch.vtensor<[],f32>, %arg4: !torch.vtensor<[],ui8>, %arg5: !torch.vtensor<[?,?,?,?],ui8>, %arg6: !torch.vtensor<[],f32>, %arg7: !torch.vtensor<[],ui8>) -> !torch.vtensor<[?,?,?,?],ui8> attributes {torch.onnx_meta.ir_version = 5 : si64, torch.onnx_meta.opset_version = 10 : si64, torch.onnx_meta.producer_name = "backend-test", torch.onnx_meta.producer_version = ""} {
  %0 = torch.operator "onnx.QLinearConcat"(%arg0, %arg1, %arg2, %arg3, %arg4, %arg5, %arg6, %arg7) {torch.onnx.axis = 1 : si64} : (!torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[?,?,?,?],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>, !torch.vtensor<[?,?,?,?],ui8>, !torch.vtensor<[],f32>, !torch.vtensor<[],ui8>) -> !torch.vtensor<[?,?,?,?],ui8>
  return %0 : !torch.vtensor<[?,?,?,?],ui8>
}