Add custom options to sbatch command in SLURM

[nilskober]

Dear Submitit team,

I am using submitit to submit jobs to a SLURM cluster. For reasons, I need to execute sbatch with a particular argument when submitting jobs to a certain partition.

Is there any chance you could add an option to pass customs arguments to be used with sbatch?

It would need to be processed in _make_submission_command() in submitit/slurm/slurm.py.

Thanks for maintaining this helpful library.

Best,
Nils

[gwenzek]

which argument is that ? can you link to sbatch documentation ?

can't it be written in the submitted file itself like we do for other args ?

submitit/submitit/slurm/slurm.py

Line 390 in f52443f

def _make_sbatch_string(

[nilskober]

Thanks @gwenzek for your quick reply! It is a custom argument and not part of the official SLURM documentation.

Our chair cooperates with a company that finances certain partitions of our cluster, let's call it "mycompany" for the moment. Every time we submit a job to one of these partitions, we are requested to confirm that we are allowed to use this partition for the job by pressing y. Alternatively, we can submit our a jobs with sbatch --mycompany name_of_script.sh. Unfortunately, adding this option to the sbatch file itself does not suppress the confirmation request.

[gwenzek]

the interactive aspect will make it hard to implement.
can't this --mycompany flag be written inside the sbatch file itself ?
But at FAIR we do have some project specific partition, and only specific users can use them. But there is no specific flag or prompt confirmation.

[nilskober]

I'll have to check that with our admin but as of now, it is not possible to add the --mycompany flag inside the sbatch file itself.

I completely understand that interactive confirmation is hard to implement. But it would be sufficient if I was able to pass on the --mycompany flag to submitit. When calling sbatch with this flag, no further confirmation is prompted.

I've forked the repo and changed _make_submission_command in slurm.py to:

def _make_submission_command(self, submission_file_path: Path) -> List[str]:
        return ["sbatch", "--mycompany", str(submission_file_path)]`

This works for me. But it would be preferable if you could set such flags directly in submitit. As far as I understand the structure of your code, a list of flags could be passed to the __init__ function of SlurmExecutor.

An alternative would be to modify _internal_update_parameters() of SlurmExecutor to handle a parameter like sbatch-flags which could be a list of strings that is then added in _make_submission_command().

Would you be open to such changes and which of the two variants do you prefer?