Page Content

Overview of clusters, limits and job processing

For details, please also read the Linux-Cluster subchapters!

Cluster specifications			Limits					Cluster- / Partition-specific Slurm job settings	Typical job type	Cluster system
Slurm cluster	Slurm partition	Nodes in partition	Node range per job min - max	Maximum runtime (hours)	Maximum running jobs per user	Maximum submitted jobs per user	Memory limit (GByte)	Cluster- / Partition-specific Slurm job settings	Typical job type	Cluster system
cm2	cm2_large	404 (overlapping partitions)	25 - 64	48	2	30	56 per node	--clusters=cm2 --partition=cm2_large --qos=cm2_large	large distributed memory parallel (MPI) job	CoolMUC-2 28-way Haswell-EP nodes with Infiniband FDR14 interconnect and 2 hardware threads per physical core
cm2	cm2_std	404 (overlapping partitions)	3 - 24	72	4	50		--clusters=cm2 --partition=cm2_std --qos=cm2_std	standard distributed memory parallel (MPI) job
cm2_tiny	cm2_tiny	300	1 - 4	72	10	50		--clusters=cm2_tiny	small distributed memory parallel (MPI) job single-node shared memory parallel job
serial	serial_std	96 (overlapping partitions)	1 - 1	96	dynamically adjusted depending on workload	250		--clusters=serial --partition=serial_std --mem=<memory_per_node>MB	single-core jobs Shared use of compute nodes among users! Default memory = mem_node/ N_{cores_node}
serial	serial_long	96 (overlapping partitions)	1 - 1	> 72 (currently 480)	dynamically adjusted depending on workload	250		--clusters=serial --partition=serial_long --mem=<memory_per_node>MB
inter	cm2_inter	12	1 - 4	2	1	2		--clusters=inter --partition=cm2_inter	interactive test jobs Do not run production jobs!
	teramem_inter	1	1 - 1 (up to 64 logical cores)	240	1	2	approx. 60 per physical core	--clusters=inter --partition=teramem_inter	shared memory thread-parallel job	Teramem HP DL580 shared memory system in total 96 physical cores each physical core has 2 hyperthreads
	mpp3_inter	3	1 - 3	2	1	2	approx. 90 DDR plus 16 HBM per node	--clusters=inter --partition=mpp3_inter	interactive test jobs Do not run production jobs!	CoolMUC-3 64-way Knight's Landing 7210F nodes with Intel Omnipath 100 interconnect and 4 hardware threads per physical core
mpp3	mpp3_batch	145	1 - 32	48	50	dynamically adjusted depending on workload	approx. 90 DDR plus 16 HBM per node	--clusters=mpp3 --partition=mpp3_batch	shared memory thread-parallel job distributed memory parallel job

Submit hosts

Submit hosts are usually login nodes that permit to submit and manage batch jobs.

Cluster segment	Submit hosts
CooLMUC-2	lxlogin1, lxlogin2, lxlogin3, lxlogin4
CooLMUC-3	lxlogin8, lxlogin9
Teramem	lxlogin8
IvyMUC	lxlogin10

However note that cross-submission of jobs to other cluster segments is also possible. The only thing you need to take care of is that different cluster segments support different instructions sets, so you need to make sure that your software build produces the appropriate binary that can execute on the targeted cluster segment.

Documentation of SLURM

SLURM Workload Manger (commands and links to examples).

Page tree

Job Processing on the Linux-Cluster

Overview of clusters, limits and job processing

Submit hosts

Documentation of SLURM