A new cluster segment, “CooLMUC-2,” with the same processor and cooling technology as SuperMUC Phase 2, was installed and commissioned in two stages. The first part of the application (state-funded large-scale equipment pursuant to Article 143c of the German Constitution) was already approved in 2014. Installation of the first subsystem began in December 2014, and user operation commenced in May 2015. In addition to additional computing nodes, the installation of Phase 2 (large research equipment pursuant to Art. 91b GG) also included six adsorption chillers installed in collaboration with Sortech AG to use the waste heat from the computers to generate cooling capacity. The use of improved technologies and control engineering enables reliable year-round cooling of the remaining air-cooled components of phase 2 of SuperMUC. The final expansion of CooLMUC-2 ranks 261st on the TOP500 list, November 2015 edition. In addition, the provision of a GPFS-based high-performance file system as scratch storage has eliminated a growing bottleneck in the processing of I/O-heavy computing jobs. CooLMUC-2 replaces the previous “CooLMUC-1” cluster, which is scheduled to be decommissioned in 2016. The Nehalem-based sgi ICE cluster was already taken out of user operation in the summer of 2015. This was followed in the fall by the decommissioning of the existing serial cluster; accordingly, a subset of the nodes was moved to CooLMUC-2.

The first steps toward integrating the future big data infrastructure with the HPC systems were taken with the connection of the CooLMUC-2 login nodes to the Data Science Storage (DSS). In addition to productive computing operations, the CoolMUC-2 system also served as a research object for innovative and energy-efficient cooling concepts. In addition to the hot water cooling system that had been established at the LRZ for years, it also had six adsorption chillers from Sortech. These made it possible to generate cooling from the waste heat of the computer nodes with low electrical energy consumption, which was used to cool the storage system of SuperMUC Phase2. The technology proved to be very reliable and efficient: in 2016, an average of 120 kW of waste heat at 45°C was used to generate approximately 50 kW of cooling at 21°C. The coefficient of performance of the overall system was 12. This means that 1 kW of electrical energy had to be expended for every 12 kW of cooling capacity. This made the adsorption chillers about three times more efficient than traditional compressor-based chillers.

After 9 years of operation, the CoolMUC-2 system was shut-off Friday, 13.12.2024.

CoolMUC-2: System Overview

Overview of cluster specifications and limits

Overview of job processing

--clusters=cm2
--partition=cm2_large
--qos=cm2_large

squeue -M cm2 -u $USER
scancel -M cm2 <JOB-ID>
sacct -M cm2 -X -u $USER --starttime=2021-01-01T00:00:01

--clusters=cm2
--partition=cm2_std
--qos=cm2_std

--clusters=cm2_tiny

squeue -M cm2_tiny -u $USER
scancel -M cm2_tiny <JOB-ID>
sacct -M cm2_tiny -X -u $USER --starttime=2021-01-01T00:00:01

--clusters=serial
--partition=serial_std
--mem=<memory_per_node_MB>M

squeue -M serial -u $USER
scancel -M serial <JOB-ID>
sacct -M serial -X -u $USER --starttime=2021-01-01T00:00:01

--clusters=serial
--partition=serial_long
--mem=<memory_per_node_MB>M

--clusters=inter
--partition=cm2_inter

squeue -M inter -u $USER
scancel -M inter <JOB-ID>
sacct -M inter -X -u $USER --starttime=2021-01-01T00:00:01

--clusters=inter
--partition=cm2_inter_large_mem
--mem=<memory_per_node_MB>M

--clusters=inter
--partition=cm4_inter_large_mem

--clusters=inter
--partition=teramem_inter
--mem=<memory_per_node_MB>M

--clusters=inter
--partition=mpp3_inter

--clusters=mpp3
--partition=mpp3_batch

squeue -M mpp3 -u $USER
scancel -M mpp3 <JOB-ID>
sacct -M mpp3 -X -u $USER --starttime=2021-01-01T00:00:01