- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -

Difference between revisions of "HPE Hawk"

From HLRS Platforms
Jump to navigationJump to search
 
(171 intermediate revisions by 6 users not shown)
Line 1: Line 1:
'''Hawk''' is the next generation HPC system at HLRS. It will replace the existing [[Cray XC40|HazelHen]] system.
 
The installation is planed to take place in Q4 2019. For more detailed information see the [[Hawk installation schedule]].
 
  
This Page is under construction!
+
{{Note
 +
| text = Please be sure to read at least the [[10_minutes_before_the_first_job]] document and consult the [[General HWW Documentation]] before you start to work with any of our systems.
 +
}}
  
[[Batch_System_PBSPro_(Hawk)|Batch System PBSPro (Hawk)]]
+
----
  
== MPI ==
 
  
In order to use the MPI implementation provided by HPE, please load the Message Passing Toolkit (MPT) module ''mpt'' (not ABI-compatible to other MPI implementations) or ''hmpt'' (ABI-compatible to MPICH-derivatives).
+
{| style="border:0; margin: 0;" width="100%" cellspacing="10"
For detailed information see the [http://www.hpe.com/support/mpi-ug-036 HPE Message Passing Interface (MPI) User Guide].
 
  
<br>
+
| valign="top" style="padding: 0; border: 1px solid #aaaaaa; margin-bottom: 0;" |
 +
<div style="font-size: 105%; padding: 0.4em; background-color: #eeeeee; border-bottom: 1px solid #aaaaaa; text-align: center;">'''Introduction'''</div>
 +
<div style="background: #ffffff; padding:0.2em 0.4em;">
 +
{| style="border: 0; margin: 0;" cellpadding="3"
 +
| valign="top" | 
 +
* [[Hawk_installation_schedule#Terms_of_Use | Terms of use ]]
 +
* [[HPE_Hawk_access|Access]]
 +
* [[HPE_Hawk_Hardware_and_Architecture|Hardware and Architecture]]
 +
|}
 +
</div>
  
== TODO ==
 
  
* 2019-08-15, niethammer@hlrs.de:
 
** MPT = Message Passing Toolkit
 
** MPI from the mpt module uses the SGI ABI
 
** for the MPI compiler wrappers to detect the correct compiler please set MPICC_CC, MPICXX_CXX, MPIF80_F90, MPIF08_F08 to the corresponding compiler commands
 
** Should e.g. applications using cae/platform_mpi use perfboost?
 
  
* 2019-08-07, dick@hlrs: hmpt always uses GCC (the 1st in PATH) -> adapt s.t. other compilers can also be used. Or do we have to use -lmpi instead?
+
| valign="top" style="padding: 0; border: 1px solid #aaaaaa; margin-bottom: 0;" |
 +
<div style="font-size: 105%; padding: 0.4em; background-color: #eeeeee; border-bottom: 1px solid #aaaaaa; text-align: center;">'''Troubleshooting'''</div>
 +
<div style="background: #ffffff; padding:0.2em 0.4em;">
 +
{| style="border: 0; margin: 0;" cellpadding="3"
 +
| valign="top" | 
 +
* [[HPE_Hawk_Support|Support (contact/staff)]]
 +
* [[HPE_Hawk_FAQ|FAQ]]
 +
* [http://websrv.hlrs.de/cgi-bin/hwwweather?task=viewmachine&machine=hawk Status,Maintenance for hawk]
 +
* [[HPE_Hawk_News|News]]
 +
|}
 +
</div>
  
* 2019-08-07, dick@hlrs: hmpt is ABI-compatible with MPICH-derivativs, but not so mpt
 
** user should know about this!
 
** @HPE: is hmpt a MPICH-derivative, but not so mpt?
 
  
* 2019-08-07, dick@hlrs: unclear that (h)mpt provides MPI lib -> call it "mpi/hmpt" and "mpi/mpt" instead?
+
|}
  
* 2019-08-07, dick@hlrs: remove MPI delivered with RHEL
 
  
* 2019-08-07, dick@hlrs: Intel loads gcc module -> use (LD_)LIBRARY_PATH intead
 
  
* 2019-08-07, dick@hlrs: be careful: cc points to /usr/bin/gcc!
 
  
* 2019-08-14, khabi/offenhaeuser@hlrs.de: How to pin OpenMP-threads in hybrid jobs (naive approach pins 2 threads to _same_ core instead of two different), i.e.: how to do aprun -d?
 
  
* 2019-07-14, khabi/offenhaeuser@hlrs.de:  
+
{| style="border:0; margin: 0;" width="100%" cellspacing="10"
  
<br>
+
| valign="top" style="padding: 0; border: 1px solid #aaaaaa; margin-bottom: 0;" |
 +
<div style="font-size: 105%; padding: 0.4em; background-color: #eeeeee; border-bottom: 1px solid #aaaaaa; text-align: center;">'''Documentation'''</div>
 +
<div style="background: #ffffff; padding:0.2em 0.4em;">
 +
{| style="border: 0; margin: 0;" cellpadding="3"
 +
| valign="top" | 
 +
* [[Batch_System_PBSPro_(Hawk)|Batch System]]
 +
* [[Module environment(Hawk)|Module Environment]]
 +
* [[Storage_(Hawk)| Storage Description ]]
 +
* [[Compiler(Hawk)|Compiler]]
 +
* [[MPI(Hawk)|MPI]]
 +
* [[Libraries(Hawk)|Libraries]]
 +
* [[Manuals(Hawk)|Manuals]]
 +
|}
 +
</div>
  
== Modulefile best practices ==
 
  
* Set an environment variable to the root path of your installation (cf. e.g. MPI_ROOT in /usr/share/Modules/modulefiles/hmpt/2.19).
 
* Set not only CPATH but also respective variables used by PGI / Intel / etc. -> someone has to figure out the list of those variables, same w.r.t. (LD_)LiBRARY_PATH.
 
* Include your Name (finger does not work anymore), E-Mail and date of installation into the modulefile.
 
* It's possible to hold the modulefile(s) together with the actual installation in the respective directory and just create symlinks in /opt/hlrs/modulefiles/.
 
* Directory structure of /opt/hlrs/ shall be replicated in /opt/hlrs/unsupported-modulefiles/.
 
* In case of dependencies, load explicit versions instead of default one!
 
  
As an example a modulefile for package "foo" version 1.23 (within the category "performance") should look like:
+
| valign="top" style="padding: 0; border: 1px solid #aaaaaa; margin-bottom: 0;" |
+
<div style="font-size: 105%; padding: 0.4em; background-color: #eeeeee; border-bottom: 1px solid #aaaaaa; text-align: center;">'''Utilities'''</div>
#%Module1.0
+
<div style="background: #ffffff; padding:0.2em 0.4em;">
BASE_DIR=/opt/hlrs/
+
{| style="border: 0; margin: 0;" cellpadding="3"
CAT=performance
+
| valign="top"
PACKAGE=Foo
+
* [[CAE_utilities|CAE Utilities]]
VERSION=1.23
+
* [[MKL | MKL Fortran Interfaces ]]
+
|}
PACKAGE_ROOT=$BASE_DIR/$CAT/$PACKAGE/$VERSION
+
</div>
 
setenv PACKAGE_ROOT $PACKAGE_ROOT
 
setenv PACKAGE_VERSION $VERSION
 
 
prepend-path PATH                $PACKAGE_ROOT/bin
 
prepend-path LD_LIBRARY_PATH    $PACKAGE_ROOT/lib        # library search path at time of execution
 
prepend-path LIBRARY_PATH        $PACKAGE_ROOT/lib        # equivalent of "-L" for C, C++, and Fortran; works for gcc and clang
 
prepend-path CPATH              $PACKAGE_ROOT/include    # equivalent to "-I" for C and C++; works with gcc and clang
 
prepend-path CPLUS_INCLUDE_PATH  $PACKAGE_ROOT/include    # equivalent to "-I" only for C++ compiler; usually not needed as CPATH will do
 
#prepend-path C_INCLUDE_PATH      $PACKAGE_ROOT/include    # equivalent to "-I" only for C compiler; usually not needed as CPATH will do
 
###  not verified yet, seems to be intel specific: prepend-path FPATH              $PACKAGE_ROOT/include    # same for Fortran
 
  
== Test cases best practices ==
+
|}
  
Test cases will help to identify and determine the scaling/ performance behavior of the new system.
 
Ideally, those test cases can be compared to other systems as well to get a full picture.
 
  
To do:
+
----
* Definition of a best practice guideline on how to set up a correct test case
+
[[Help | Help for Wiki Usage]]
** Only measurement of time-stepping loop or equivalent excluding the initialization phase or cleanup
 
** Well defined measure of computational progress (e.g. LUPS, DoF-UPS, Iterations/s or Flop/s)
 
** Ideally, the test case is mostly automated with scripts and does also the evaluation on top with a meaningful result file
 

Latest revision as of 11:51, 20 November 2020