Available Hardware Resources
NMASC (nmpost)
The NMASC provides a processing cluster called the nmpost cluster to support CASA and AIPS execution, and a Lustre filesystem for a data storage.
Cluster
The nmpost cluster consists of 120 nodes. Nodes 091-120 are reserved for the HTCondor scheduler and are not part of the common Torque/Moab scheduler. Nodes 061-090 are reserved for our VLASS project. The remaining 60 nodes are available for batch processing, 50 of which can also be used for interactive processing. None of the nodes have local storage, instead each node has a 40Gb/s QDR Infiniband connection to the NMASC Lustre filesystem. The cluster supports automatic EVLA pipeline processing, archive retrievals, batch processing requests and interactive processing sessions.
The operating system is Red Hat Enterprise Linux 7. The cluster scheduling software is Slurm or Torque/Moab or HTCondor.
| Nodes | CPU | Sockets | Cores per Socket | Clock | CPU Mark | RAM | Interconnect | Queues | Scheduler |
|---|---|---|---|---|---|---|---|---|---|
|
nmpost{001..010} |
E5-2630v3 | 2 | 8 | 2.4GHz | 17,041 | 512GB | 40Gb/s QDR Infiniband | interactive, batch | Slurm |
|
nmpost{011..020} |
Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 512GB | 40Gb/s QDR Infiniband | batch | Torque |
|
nmpost{021..040} |
E5-2640v4 | 2 | 10 | 2.4GHz | 20,394 | 512GB | 40Gb/s QDR Infiniband | interactive, batch | Slurm |
|
nmpost{041..060} |
Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 512GB | 40Gb/s QDR Infiniband | interactive, batch | Slurm |
| nmpost{061..090} | Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 512GB | 40Gb/s QDR Infiniband | vlass | Torque |
| nmpost{091..120} | Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 512GB | NA | NA | HTCondor |
CPU Marks obtained from https://www.cpubenchmark.net/cpu_list.php
Lustre Filesystem
Lustre is a distributed parallel filesystem commonly used in HPC environments. Each cluster node sees the same shared filesystem. The NMASC Lustre filesystem is made up of ten storage servers each with four RAID arrays (40 total arrays) which are each 44TB in size. The total storage volume is 1.8PB. Individual cluster nodes can read and write to the Lustre filesystem in excess of 1GByte/sec. The entire filesystem can sustain roughly 15GB/s aggregate I/O.
The Lustre filesystem appears as /lustre/aoc on all Lustre enabled client computers.
Public Workstations
The NMASC has nine workstations for local visitors. The systems vary in processor, memory and storage since work is expected to be done mostly on the compute cluster, but all have 10Gb/s ethernet connections to the Lustre filesystem. Instructions for reserving workstations can be found in the Computing section of the Visiting the DSOC web page.
NAASC (cvpost)
The NAASC provides a processing cluster called the cvpost cluster to support CASA execution, and a Lustre filesystem for a data storage.
Cluster
The cvpost cluster consists of 64 nodes, all of which are available to users for either batch or interactive processing. None of the nodes have local storage, instead each node has a 40Gb/s QDR Infiniband connection to the NAASC Lustre filesystem. The cluster supports automatic ALMA pipeline processing, archive retrievals, batch processing requests and interactive processing sessions.
The operating system is Red Hat Enterprise Linux 7. The cluster scheduling software is Torque/Moab.
| Nodes | CPU | Sockets | Cores per Socket | Clock | CPU Mark | RAM | Interconnect | Queues |
|---|---|---|---|---|---|---|---|---|
| cvpost001 | E5-2670 | 2 | 8 | 2.6GHz | 15,695 | 256GB | 40Gb/s QDR Infiniband | batch |
| cvpost{002..004} | E5-2640v3 | 2 | 8 | 2.6GHz | 18,094 | 256GB | 40Gb/s QDR Infiniband | batch |
| cvpost005 | E5-2670 | 2 | 8 | 2.6GHz | 15,695 | 256GB | 40Gb/s QDR Infiniband | batch |
| cvpost{006..007} | E5-2640v3 | 2 | 8 | 2.6GHz | 18,094 | 256GB | 40Gb/s QDR Infiniband | batch |
| cvpost{008..012} | E5-2670 | 2 | 8 | 2.6GHz | 15,695 | 256GB | 40Gb/s QDR Infiniband | batch |
| cvpost013 | E5-2670 | 2 | 8 | 2.6GHz | 15,695 | 256GB | 40Gb/s QDR Infiniband | NA |
| cvpost{014..017} | E5-2670 | 2 | 8 | 2.6GHz | 15,695 | 256GB | 40Gb/s QDR Infiniband | interactive, batch |
| cvpost{018..023} | E5-2640v3 | 2 | 8 | 2.6GHz | 18,094 | 256GB | 40Gb/s QDR Infiniband | interactive, batch |
| cvpost{024..026} | E5-2670 | 2 | 8 | 2.6GHz | 15,695 | 256GB | 40Gb/s QDR Infiniband | interactive, batch |
| cvpost{027..030} | E5-2630v3 | 2 | 8 | 2.4GHz | 10,271 | 256GB | 40Gb/s QDR Infiniband | NA |
| cvpost{101..123} | Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 512GB | 40Gb/s QDR Infiniband | interactive2, batch2 |
| cvpost{124..135} | Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 512GB | 40Gb/s QDR Infiniband | plwg |
| cvpost{136..137} | Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 768GB | 40Gb/s QDR Infiniband | plwg |
| cvpost{138..139} | Gold 6136 | 2 | 12 | 3.0GHz | 33,555 | 512GB | 40Gb/s QDR Infiniband | NA |
Lustre Filesystem
Lustre is a distributed parallel filesystem commonly used in HPC environments. Each cluster node sees the same shared filesystem. The NAASC Lustre filesystem is made up of four storage servers each with four RAID arrays (16 total arrays) which are each 64TB in size. The total storage volume is 1.1PB. Individual nodes can read and write to the Lustre filesystem in excess of 1GByte/sec, the entire filesystem can sustain roughly 10GB/s aggregate I/O.
The Lustre filesystem appears as /lustre/naasc on all Lustre enabled client computers.
Public Workstations
The NAASC has five workstations for local visitors. The systems have 8 x Intel E5-1660 Xeon 3.0 GHz processors, 32 GB RAM, 3 TB local disk space (accessible as /home/<hostname>_1 and /home/<hostname>_2) and a 10Gb/s ethernet connection to the Lustre filesystem.
