VSC-5

VSC-5 is the latest addition to the Vienna Scientific Cluster infrastructure, supplied by Megware. The system comprises 1,540 AMD EPYC Milan processors and 98,560 cores (more than double of the VSC-4), and features a hardware partition of CPU and GPU nodes connected via Mellanox HDR infiniband (200 Gbit/s bandwidth). 

The CPU partition features:

710 compute nodes equipped with two AMD EPYC 7713 (Milan), 64 cores (in total 90,880 cores) with a base frequency of 2GHz. Most of the nodes are equipped with 512GB of memory, but there are also large memory nodes (120 nodes with 1TB and 20 nodes 2TB). Each node has a 2TB NVME disk. The system is complemented with 10 login nodes with identical processors as the compute nodes.

The GPU partition features:

60 compute nodes equipped with two AMD EPYC 7713 (Milan) processors, 64 cores (in total 7,680 cores) operating at 2GHz and 512 GB memory, while each node is additionally equipped with two NVIDIA A100 cards (40GB memory). Each node has a 2TB NVME disk. The system is complemented with 3 master nodes for node deployment.

Cooling:

The VSC-5 mostly uses a direct hot water cooling system ColdCon. It operates at temperatures up to 50°C and allows energy-efficient free cooling throughout the year. Direct water cooling permits direct extraction of the heat produced in the compute nodes, in particular by the CPUs, the memory DIMMS, the SSDs and the Infiniband adapters. The waste heat can be reused to produce hot water or support the existing heating infrastructure.

For the cooling of other components like power supplies, PDUs, infiniband and ethernet switches air-cooling via in-row coolers is used.

The GPU partition as well as login and service nodes are also air-cooled.

Performance:

The current configuration of the CPU partition achieves a benchmark value (Rmax) of 2.31 PFlop/s with a theoretical peak performance (Rpeak) of 3.13 PFlop/s. In addition, the GPU partition of VSC-5 delivers 1.17 PFlop/s. Thus, VSC-5 has an aggregated peak performance of 4.30 PFlop/s. 

VCS-5 in the TOP500 list (Rank 301 in June 2022)

Insight into VSC-5 - video by Megware