- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
Compute nodes as well as login nodes are equipped with
AMD EPYC 7742 64-Core Processor
detailed information will be provided later. Please check for additional infos AMD Rome 7742 With respect to node and processor details cf. here.
Hawk deploys an Infiniband HDR based interconnect with a 9-dimensional enhanced hypercube topology. Please refer to here with respect to the latter. Infiniband HDR has a bandwidth of 200 Gbit/s and a MPI latency of ~1.3us per link. The full bandwidth of 200 Gbit/s can be used if communicating between the 16 nodes connected to the same node of the hypercube (cf. here). Within the hypercube, the higher the dimension, the less bandwidth is available. Topology aware scheduling is used to exclude major performance fluctuations. This means that larger jobs can only be requested with defined node numbers (64, 128, 256, 512, 1024, 2048 and 4096) in regular operation. This restriction ensures optimal system utilization while simultaneously exploiting the network topology. Jobs with a node number of < 128 nodes are processed in a special partition. Jobs over 4096 nodes are processed at special times.
Tuned MPI: In order to use the MPI implementation provided by HPE, please load the Message Passing Toolkit (MPT) module mpt (not ABI-compatible to other MPI implementations) or hmpt (ABI-compatible to MPICH-derivatives).
User Guide: For detailed information cf. the HPE Message Passing Interface (MPI) User Guide.
Performance optimization: With respect to MPI performance optimization by means of tuning environment variables please cf. Tuning of MPT
Home directories as well as workspaces are handled in the same way as on Hazel Hen, so please cf. Storage Description regarding details.
Pre- and post processing
Within HLRS simulation environment special nodes for pre- and post processing tasks are available. Such nodes could be requested via the batch system (follow this link for more info). Available nodes are
table... 4 nodes 2 TB Memory 2 Socket AMD ...x TB local storage shared usage model 1 Node 4 TB Memory 2 Socket AMD x TB local storage shared usage model
more specialized nodes e.g. graphics, vector, DataAnalytics, ... are available in the Vulcan cluster
- Software Optimization Guide for AMD EPYC Rome Processors
- Open-Source Register Reference for AMD EPYC Rome Processors
(in particular describing available hardware performance counters)
- Software Optimization Guide for AMD Family 15h
(although depicting an older family of AMD processors, the optimization approaches shown in this document are also applicable to the AMD EPYC Rome processor deployed in Hawk)