Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Installation PhasePhase 1
(decomissioned end of 2018)
Phase 2
(decomissioned end of 20182019)
Installation Date2011201220132015

Islandtype

Fat NodesThin NodesMany Cores NodesHaswell Nodes
SystemBladeCenter HX5IBM System x iDataPlex dx360M4IBM System x iDataPlex dx360M4Lenovo NeXtScale nx360M5 WCT
Processor TypeWestmere-EX
Xeon E7-4870 10C
Sandy Bridge-EP
Xeon E5-2680
8C
Ivy-Bridge and Xeon Phi 5110PHaswell Xeon Processor E5-2697 v3
Nominal Frequency [GHz]2.42.7 1.052.62
Performance per core4 DP Flops/cycle =
9.6 DP GFlop/s
2-wide SSE2 add +
 2-wide SSE2 mult
8 DP Flops/cycle =
21.6 DP GFlops/s
4-wide AVX add +
4-wide AVX mult

16 DP Flops/cycle =
16.64 DP GFlops/s
8-wide  fused multiply-adds every cycle using 4 threads

16 DP Flops/cycle =
41.6 DP GFlops/s
two 4-wide fused multiply-adds

Total Number of nodes2059216323072
Total Number of cores8,200147,4563,840 (Phi)86,016
Total Peak Performance [PFlop/s]0.0783.20.064 (Phi)3.58
Total Linpack Performance [PFlop/s]0.0652.897 n.a.2.814
Total size of memory [TByte]52288 2.56194
Total Number of Islands118 16
Typical Power Consumption [MW]< 2.3 ~1.1
Components
Nodes per Island205512 32512
Processors per Node42

 2 (IvyB) 2.6 GHz + 2 Phi 5110P

2
Cores per Processor108 8 (IvyB) + 60 (Phi)14
Cores per Node4016 16 (host) + 120 (Phi)28
Logical CPUs per Node (Hyperthreading)8032 32 (host) + 480 (Phi) 56
Memory and Caches
Memory per Core [GByte]
(typically available for applications)
6.4
(~6.0)
2
(~1.5)
 4 (host) + 2 x 0.13 (Phi)2.3
(2.1)
Graphical representation of processor topologywestmere.pngsandbridge.pnghost.png
phi.png
haswell.png
haswell.big.png
Size of shared Memory per node [GByte]25632 64 (host) + 2 x 8 (Phi)

64
(8 nodes in job class big: 256)

Bandwidth to Memory per node [Gbyte/s]136.4102.4 Phi: 384137
Level 3 Cache Size (shared) [Mbyte]4x302x20
4x18
Level 2 Cache Size per core [kByte]256256 Phi: 512256
Level 1 Cache Size [kByte]3232 3232
Latency Access Memory [cycles] / Bandwidth per core [GB/s]
~160 /8.8
~200 / 6.7
Level 3 Latency [cycles] /BW per Core [GB/s]
~ 30 / 31
36 / 39

Level 2 Latency [cycles]1 /BW per Core [GB/s]


12 / 42
12 / 92

Level 1 Latency [cycles]1 /BW per Core [GB/s]

44 /130
4 / 343
Interconnect
TechnologyInfiniband QDRInfiniband FDR10Infiniband FDR10 Infiniband FDR14
Intra-Island Topologynon-blocking Treenon-blocking Tree
Inter-Island TopologyPruned Tree 4:1n.a.Pruned Tree 4:1
Bisection bandwidth of Interconnect [TByte/s]12.5
 5.1
Servers
Login Servers for users2715
Storage
Size of parallel storage (SCRATCH/WORK) [Pbyte]15
Size of NAS storage (HOME) [PByte]3.5 (+ 3.5 for replication)
Aggregated bandwidth to/from parallel storage [GByte/s]250
Aggregated bandwidth to/from NAS storage [GByte/s]12
Capacity of Archive and Backup Storage [PByte]> 30
System Software
Operating SystemSuse Linux Enterprise Server (SLES)
BatchsystemIBM Loadleveler
Parallel Filesystem for SCRATCH and WORKIBM GPFS
File System for HOMENetApp NAS
Archive and Backup SoftwareIBM TSM
System ManagementxCat from IBM
MonitoringIcinga, Splunk

...