Error occurred when fine-tuning using LoRA and IA3 methods.

[Gerald-li]

When fine-tuning the ProSST model (tested both 2046 and 4096 versions) and Prime model, we encountered errors while using LoRA and IA3 fine-tuning methods. The error message is approximately:

File "/hpcfs/fhome/puchx/.cache/huggingface/modules/transformers_modules/modeling_prosst.py", line 868, in forward
inputs_embeds.masked_fill_(
RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

Do you know what causes this error? Other fine-tuning methods (freeze, full, and ses-adapter) work without any issues.
Thanks!!

[tyang816]

@code4luck Fix this bug asap plz.

[zlr-zmm]

Hello,

Thank you for your interest in our work. We sincerely apologize for not responding sooner. We have reviewed our code and fixed some existing issues, but unfortunately, we were unable to reproduce the error you encountered.

If you haven’t resolved this issue yet, could you please provide more detailed information? This would be extremely helpful for improving our project.

Best regards