How to run the fine-tuning using slurm?

[AlessioQuercia]

Could you provide a slurm script to run the fine-tuning code? Apparently there are some issues with deepspeed, by just using the provided instructions.

[kongds]

Thank you for your interest in our work.

Our experiments are directly conducted with deepspeed over multiple nodes by the provided script (without slurm).
Maybe you need some configs to make slurm run on multiple nodes. (Or you can run script on single node by change --num_nodes=4 to --num_nodes=1)