Welcome to the Info TEST server!

Skip to content. | Skip to navigation

Sections
Info Services > Computing Guide > Cluster Processing > Appendix > Translating between Slurm, HTCondor, and Torque

Translating between Slurm, HTCondor, and Torque

This page attempts to describe the differences in various cluster scheduling systems.  Some basic understanding of each system is expected.  While Torque and Slurm are very similar in their usage, HTCondor is somewhat different.  With Torque and Slurm you can use command-line arguments to specify the requirements of a job.  With HTCondor you need to create a Submit Description File that specifies the requirements and defines the script to execute.

These translations are not meant to be exact.  You should have basic understanding of the cluster systems involved.  There are some examples at the end of this document.

 As of 2025-08-20 the NRAO no longer supports to has Torque/Moab installed.  References remain here for informational purposes only.

Submit Options

DescriptionSlurmHTCondorTorque/Moab
Script directive #SBATCH NA #PBS
Queue/Partition -p <partition>

requirements = (<partition> == True)

+partition = "<partition>"

-q <queue>

Node count -N <min>[-max]> NA -l nodes=<count>
Core count -n <count> OR -c <count>
request_cpus = <count> -l ppn=<count>
Wall clock limit -t <min> OR -t <days-hh:mm:ss> periodic_remove = (time() - JobStartDate) > (<seconds>) -l walltime=<hh:mm:ss>
Stdout -o <filename> output = <filename> -o <filename>
Stderr -e <filename> error = <filename> -e <filename>
Copy environment --export=ALL getenv = true -V
Email notification

--mail-type=[ALL, END, FAIL, BEGIN, NONE]

notification = [Always, Complete, Error, Never] -m [a|b|e]
Email address --mail-user=<user_list> notify_user = <user_list> -M <user_list>
Job name -J <name> OR --job-name=<name> batch_name = <name> -N <name>
Working directory -D <path> initialdir -d <path> OR -w <path>
Memory per node --mem=<count[K, M, G, T]> request_memory = <count> G -l mem=<count[kb, mb, gb, tb]>
Memory per core --mem-per-cpu=<count[K, M, G, T]> NA -l pmem=<count[kb, mb, gb, tb]>
Virtual memory per node NA NA -l vem=<count[kb, mb, gb, tb]>
Virtual memory per core NA NA -l pvmem=<count[kb, mb, gb, tb]>
Memory per job --mem=<count[K, M, G, T]> request_memory = <count> G -L tasks=1:memory=<count[kb, mb, gb, tb]>
Job arrays --array=<arrayspec>  queue seq <first> [<increment>] <last> | -t <arrayspec>
Variable list --export=<var>=<val>[,<var>=<val>] environment = "<var>=<val> [<var>=<val>]" -v <var>=<val>[,<var>=<val>]
Script args sbatch script <arg1>[,<arg2>,...]   -F <arg1>[,<arg2>,...]

 

 

 

Commands

DescriptionSlurmHTCondorTorque/Moab
Job alter scontrol update condor_qedit qalter
Job connect to srun --jobid <jobid> --pty bash -l condor_ssh_to_job <jobid> NA
Job delete scancel <jobid> condor_rm <jobid> qdel <jobid>
Job delete all user's jobs scancel --user=<user> condor_rm <user> qdel all
Job info detailed scontrol show job <jobid> condor_q -long <jobid> qstat -f <jobid>
Job info detailed scontrol show job <jobid> condor_q -analyze -verbose <jobid> qstat -f <jobid>
Job info detailed scontrol show job <jobid> condor_q -better-analyze -verbose <jobid> qstat -f <jobid>
Job info detailed scontrol show job <jobid> condor_q -better-analyze -reverse -verbose <jobid> qstat -f <jobid>
Job show all squeue condor_q -global -all qstat -1n
Job show all verbose squeue -all condor_q -global -all -nobatch qstat -1n
Job show all verbose squeue -all condor_q -global -all -nobatch -run qstat -1n
Job show DAGs NA condor_q -dag -nobatch NA
Job show completed sacct -j <jobid> condor_q -l <jobid> tracejob <jobid>
Job show completed sacct --format=jobid,user,jobname,partition,alloccpus,reqmem,maxrss,maxvmsize,state,nodelist,start,end,exitcode -j <jobid> condor_q -l <jobid>
Job submit sbatch condor_submit qsub
Job submit simple srun sleep 27 condor_run "sleep 27" & echo "sleep 27" | qsub
Job submit interactive srun --pty bash condor_submit -i qsub -I
Node show free nodes sinfo --states=idle --partition=<partition> -N condor_status -const 'PartitionableSlot && Cpus == TotalCpus' nodesfree
Node show resources sjstat -c qstat -q
Node show state sinfo -Nl condor_status -state pbsnodes -l all

 

 

 

Variables

 

DescriptionSlurmHTCondorTorque/Moab
Job Name SLURM_JOB_NAME PBS_JOBNAME
Job ID SLURM_JOBID PBS_JOBID
Tasks per node SLURM_NTASKS_PER_NODE PBS_NUM_PPN
Cores per step on this node SLURM_CPUS_ON_NODE PBS_NUM_PPN
Queue/Partition submitted to SLURM_JOB_PARTITION PBS_O_QUEUE
Queue/Partition running on SLURM_JOB_PARTITION PBS_QUEUE
User SLURM_JOB_USER PBS_O_LOGNAME
Number of nodes in job SLURM_NNODES PBS_NUM_NODES
Number of nodes in job SLURM_JOB_NUM_NODES PBS_NUM_NODES
Submit Host SLURM_SUBMIT_HOST PBS_O_HOST
Working dir PWD PBS_O_WORKDIR
Machine file NA PBS_NODEFILE

 

 

 

Example Commands

 

Slurm

 

sbatch --export ALL -J casatest01 --mem=16G -D /lustre/aoc/observers/nm-4386 -t 0-2:30:00 --mail-type=END,FAIL run_casa.sh


HTCondor

 Create a Submit Description File (E.g. run_casa.htc) 

executable = run_casa.sh
batch_name = casatest01
request_memory = 16 G
notification = Always
environment = "CASA_HOME=/home/casa/packages/RHEL7/release/current PPR_FILENAME=PPR.xml"
initialdir = /lustre/aoc/observers/nm-4683

log = condor.$(ClusterId).log
stdout = condor.$(ClusterId).log
stderr = condor.$(ClusterId).log

queue

Then submit that file

condor_submit run_casa.htc

 

While you can set a wall clock limit for an HTCondor job, it isn't advised in most cases.

 

Torque

 

qsub -V -N casatest01 -l pmem=16gb,pvmem=16gb -d /lustre/aoc/observers/nm-4386 -l walltime=2:30:00 -m ae run_casa.sh

 

Search All NRAO