Translating between Torque, Slurm, and HTCondor
Submit Options
| Description | Torque/Moab | Slurm | HTCondor |
|---|---|---|---|
| Script directive | #PBS | #SBATCH | NA |
| Queue/Partition | -q <queue> | -p <partition> |
requirements = (<partition> == True) +partition = "<partition>" |
| Node count | -l nodes=<count> | -N <min>[-max]> | NA |
| Core count | -l ppn=<count> | -n <count> OR -c <count> |
request_cpus = <count> |
| Wall clock limit | -l walltime=<hh:mm:ss> | -t <min> OR -t <days-hh:mm:ss> | periodic_remove = (time() - JobStartDate) > (<seconds>) |
| Stdout | -o <filename> | -o <filename> | output = <filename> |
| Stderr | -e <filename> | -e <filename> | error = <filename> |
| Copy environment | -V | --export=ALL | getenv = true |
| Email notification | -m [a|b|e] |
--mail-type=[ALL, END, FAIL, BEGIN, NONE] |
notification = [Always, Complete, Error, Never] |
| Email address | -M <user_list> | --mail-user=<user_list> | notify_user = <user_list> |
| Job name | -N <name> | -J <name> OR --job-name=<name> | batch_name = <name> |
| Working directory | -d <path> OR -w <path> |
-D <path> | NA |
| Memory per node | -l mem=<count[kb, mb, gb, tb]> | --mem=<count[K, M, G, T]> | request_memory = <count> G |
| Memory per core | -l pmem=<count[kb, mb, gb, tb]> | --mem-per-cpu=<count[K, M, G, T]> | NA |
| Virtual memory per node | -l vem=<count[kb, mb, gb, tb]> | NA | NA |
| Virtual memory per core | -l pvmem=<count[kb, mb, gb, tb]> | NA | NA |
| Memory per job | -L tasks=1:memory=<count[kb, mb, gb, tb]> | --mem=<count[K, M, G, T]> | request_memory = <count> G |
| Job arrays | -t <arrayspec> | --array=<arrayspec> | queue seq <first> [<increment>] <last> | |
| Variable list | -v <var>=<val>[,<var>=<val>] | --export=<var>=<val>[,<var>=<val>] | environment = "<var>=<val>" |
| Script args | -F <arg1>[,<arg2>,...] | sbatch script <arg1>[,<arg2>,...] |
Commands
| Description | Torque/Moab | Slurm | HTCondor |
|---|---|---|---|
| Job alter | qalter | scontrol update | condor_qedit |
| Job connect to | NA | srun --jobid <jobid> --pty bash -l | condor_ssh_to_job <jobid> |
| Job delete | qdel <jobid> | scancel <jobid> | condor_rm <jobid> |
| Job delete all user's jobs | qdel all | scancel --user=<user> | condor_rm <user> |
| Job info detailed | qstat -f <jobid> | scontrol show job <jobid> | condor_q -long <jobid> |
| Job info detailed | qstat -f <jobid> | scontrol show job <jobid> | condor_q -analyze -verbose <jobid> |
| Job info detailed | qstat -f <jobid> | scontrol show job <jobid> | condor_q -better-analyze -verbose <jobid> |
| Job info detailed | qstat -f <jobid> | scontrol show job <jobid> | condor_q -better-analyze -reverse -verbose <jobid> |
| Job show all | qstat -1n | squeue | condor_q -g -all |
| Job show all verbose | qstat -1n | squeue -all | condor_q -g -all -nobatch |
| Job show all verbose | qstat -1n | squeue -all | condor_q -g -all -nobatch -run |
| Job show DAGs | NA | NA | condor_q -dag -nobatch |
| Job submit | qsub | sbatch | condor_submit |
| Job submit simple | echo "sleep 27" | qsub | srun sleep 27 | condor_run "sleep 27" & |
| Job submit interactive | qsub -I | srun --pty bash | condor_submit -i |
| Node show free nodes | nodesfree | sinfo --states=idle --partition=<partition> -N | condor_status -const 'PartitionableSlot && Cpus == TotalCpus' |
| Node show resources | qstat -q | sjstat -c | |
| Node show state | pbsnodes -l all | sinfo -Nl | condor_status -state |
Variables
| Description | Torque/Moab | Slurm | HTCondor |
|---|---|---|---|
| Job Name | PBS_JOBNAME | SLURM_JOB_NAME | |
| Job ID | PBS_JOBID | SLURM_JOBID | |
| Tasks per node | PBS_NUM_PPN | SLURM_NTASKS_PER_NODE | |
| Cores per step on this node | PBS_NUM_PPN | SLURM_CPUS_ON_NODE | |
| Queue/Partition submitted to | PBS_O_QUEUE | SLURM_JOB_PARTITION | |
| Queue/Partition running on | PBS_QUEUE | SLURM_JOB_PARTITION | |
| User | PBS_O_LOGNAME | SLURM_JOB_USER | |
| Number of nodes in job | PBS_NUM_NODES | SLURM_NNODES | |
| Number of nodes in job | PBS_NUM_NODES | SLURM_JOB_NUM_NODES | |
| Submit Host | PBS_O_HOST | SLURM_SUBMIT_HOST | |
| Working dir | PBS_O_WORKDIR | PWD | |
| Machine file | PBS_NODEFILE | NA |
