/
More about the srun, sbatch, and salloc commands

More about the srun, sbatch, and salloc commands

Page Contents

Related Topics

Flags for Slurm Command

All the commands, i.e. srun, sbatch, and salloc require a set of flags, the values that they can take are as follows:

  1. --cluster

    1. chip-cpu

    2. chip-gpu

  2. --account

    1. pi_<name of the principal investigator>

  3. --partition (Refer to chip Partitions and Usage for more details.)

    1. general

    2. 2018

    3. 2021

    4. 2024 (can only be accessed by contributing PIs)

  4. --qos (refers to Quality of service. Refer to chip-cpu QOS Restrictions for more details.

  5. --time (refers to the total amount of time the user wants to allocate for the job)

    1. in HH:MM:SS

  6. --mem (refers to the upper bound on the CPU memory the user needs for that job from that node)

    1. the memory limit can be checked by concatenating the “free -h” command with the srun command for a particular node, and then the value should be a number representing the memory in MB.

  7. --cpus-per-task (refers to the number of CPU cores needed by the user) - Optional

    1. Defaults to 1 if the flag is not provided. Refer to Cluster Specifications for more details.

  8. --ntasks (refers to the number of parallel tasks the user wants the job to run) - Optional

    1. Defaults to 1 if the flag is not provided

  9. --nodes (refers to the number of worker nodes needed by the user to split the task into) - Optional

    1. Defaults to 1 if the flag is not provided. Refer to Cluster Specifications for more details.

  10. --nodelist (refers to the specific node the user wants to run their jobs on) - Optional

  11. --constraint (refers to some constraint specifying the one of many parameters on the compute node, such as the GPU cards on the node) - Optional

  12. --gres (refers to generic resources, and is only needed for chip-gpu)

    1. gpu:n, where n refers o the number of GPU cards needed by the user

  13. --pty (refers to allocating a pseudo-terminal, for running an interactive session, and is only needed for the srun command)

    1. $SHELL

    2. Path to shell - ex: /bin/bash, /bin/zsh, etc

srun

Using the srun command, a user can submit a slurm submission script to create job “steps” in a parallel job or launch processes interactively via the terminal. For running the srun commands on the cpu and the gpu, the section below highlights the key differences.

srun on chip-cpu

For running the srun command on the cpu, the following parameters are required:

  1. --cluster, which should be set to

    1. chip-cpu

  2. --account, which should be set to

    1. pi_<name of the principal investigator>

  3. --partition (Refer to chip Partitions and Usage for more details.)

    1. gpu

    2. gpu-general

    3. pi_YOURPINAME (Only available to contributing PI groups)

  4. --qos

  5. --time

  6. --mem

  7. --pty

Hence, it should look like the following:

srun --cluster=chip-cpu --account=pi_doit --partition=general --qos=normal --time=1:00:00 --mem=5000 --pty $SHELL

srun on chip-gpu

When running on the gpu, the parameters required are:

  1. --cluster, which should be set to

    1. chip-gpu

  2. --account, which should be set to

    1. pi_<name of the principal investigator>

  3. --mem

  4. --time

  5. --gres

  6. --pty

Hence, it should look like the following:

srun --cluster=chip-gpu --account=pi_doit --mem=5000 --time=1:00:00 --gres=gpu:1 --pty $SHELL

When using the srun command, note that in addition to the --account, --mem, --time, and --pty flags,
to run a job on chip-cpu, the --cluster flag needs to be set to a value of chip-cpu, and the flags --partition and --qos and their values are to be included, while to run a job on chip-gpu, the --cluster flag needs to be set to a value of chip-gpu, and the flag --gres and its value is to be included.

sbatch

Using the sbatch command, a user can take a bash script as its input and execute a series of tasks using the compute resources. Functionally, it does the same thing as the srun command, but it does not let the user do so in an interactive way like the srun command does.

sbatch on chip-cpu

Create a file (ending with the extension .slurm) using your favorite text editor that should look like the following:

[user@chip ~]$ cat sbatchTest.slurm #!/bin/bash #SBATCH --cluster=chip-cpu #SBATCH --mem=5000M #SBATCH --time=01:00:00 #SBATCH --account=pi_doit #SBATCH --partition=general #SBATCH --qos=normal module load Python python test.py [user@chip ~]$ sbatch sbatchTest.slurm

As shown above, after the file is created, run the sbatch command to submit the job to the cluster.

sbatch on chip-gpu

Create a file (ending with the extension .slurm) using your favorite text editor that should look like the following:

[user@chip ~]$ cat sbatchTest.slurm #!/bin/bash #SBATCH --cluster=chip-gpu #SBATCH --mem=5000M #SBATCH --time=01:00:00 #SBATCH --account=pi_doit #SBATCH --gres=gpu:2 module load Python python test.py [user@chip ~]$ sbatch sbatchTest.slurm

As shown above, after the file is created, run the sbatch command to submit the job to the cluster.

When using the sbatch command, similar to the srun command, note that in addition to the --account, --mem, and --timeflags,
to run a job on chip-cpu, the --cluster flag needs to be set to a value of chip-cpu, and the flags --partition and --qos and their values are to be included, while to run a job on chip-gpu, the --cluster flag needs to be set to a value of chip-gpu, and the flag --gres and its value is to be included.

salloc

The salloc command can be used to allocate the specified computing resources to an user, and hence the usage is similar to that of srun but without the command the users needs to run once the resources have been allocated, e.g.

salloc --cluster=chip-cpu --account=pi_doit --partition=general --qos=normal --time=1:00:00 --mem=5000