rotate + lm_head quantization

[RanchiZhao]

Hi, I find that when using rotation, lm_head is really hard to quantize, especially when it is per-channel, do you have any idea to fix this?

[HandH1998]

I think it is normal that quantizing lm_head has much impact on the accuracy, since it is directly relevant with the sampling. In addition, it cannot bring a significant increase in inference speed, as it doesn't take much time in all GEMM calculations of inference. So it is better to not quantize lm_head.