- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -

CRAY XC40 compiler wrapper: Difference between revisions

From HLRS Platforms
Jump to navigationJump to search
No edit summary
No edit summary
 
Line 38: Line 38:
$> ftn -craype-verbose myAppWithFFTW.f90 -o my.exe
$> ftn -craype-verbose myAppWithFFTW.f90 -o my.exe


ftn_driver.exe -hcpu=haswell -hstatic -D__CRAYXC -D__CRAY_HASWELL -D__CRAYXT_COMPUTE_LINUX_TARGET -hnetwork=aries hello.f90 -Wl,-rpath=/opt/cray/cce/8.5.1/craylibs/x86-64 -ulast_user_arg -I /opt/cray/cce/8.5.1/craylibs/x86-64/include -I/usr/include -L/opt/cray/cce/8.5.1/CC/x86-64/lib/x86-64 -Wl,-rpath=/opt/cray/cce/8.5.1/CC/x86-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 -Wl,-rpath=/opt/cray/gcc-libs -L/usr/lib64 -Wl,-rpath=/usr/lib64 -L/lib64 -Wl,-rpath=/lib64 -ugcc_base=/opt/gcc/4.8.1/snos -uno_driver_libs -Wl,/opt/cray/cce/8.5.1/craylibs/x86-64/libmodules.a,/opt/cray/cce/8.5.1/craylibs/x86-64/libomp.a,/opt/cray/cce/8.5.1/craylibs/x86-64/libopenacc.a, -I/opt/cray/fftw/3.3.4.9/haswell/include -I/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/include -I/opt/cray/libsci/16.07.1/CRAY/8.3/x86_64/include -I/opt/cray/rca/1.0.0-2.0502.60530.1.62.ari/include -I/opt/cray/alps/5.2.4-2.0502.9774.31.11.ari/include -I/opt/cray/xpmem/0.1-2.0502.64982.5.3.ari/include -I/opt/cray/gni-headers/4.0-1.0502.10859.7.8.ari/include -I/opt/cray/dmapp/7.0.1-1.0502.11080.8.76.ari/include -I/opt/cray/pmi/5.0.10-1.0000.11050.0.0.ari/include -I/opt/cray/ugni/6.0-1.0502.10863.8.29.ari/include -I/opt/cray/udreg/2.3.2-1.0502.10518.2.17.ari/include -I/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/../include -I/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/..//include -I/opt/cray/wlm_detect/1.0-1.0502.64649.2.1.ari/include -I/opt/cray/krca/1.0.0-2.0502.63139.4.31.ari/include -I/opt/cray-hss-devel/7.2.0/include -L/opt/cray/fftw/3.3.4.9/haswell/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/lib -L/opt/cray/libsci/16.07.1/CRAY/8.3/x86_64/lib -L/opt/cray/rca/1.0.0-2.0502.60530.1.62.ari/lib64 -L/opt/cray/alps/5.2.4-2.0502.9774.31.11.ari/lib64 -L/opt/cray/xpmem/0.1-2.0502.64982.5.3.ari/lib64 -L/opt/cray/dmapp/7.0.1-1.0502.11080.8.76.ari/lib64 -L/opt/cray/pmi/5.0.10-1.0000.11050.0.0.ari/lib64 -L/opt/cray/ugni/6.0-1.0502.10863.8.29.ari/lib64 -L/opt/cray/udreg/2.3.2-1.0502.10518.2.17.ari/lib64 -L/opt/cray/atp/2.0.2/libApp -L/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/../ -L/opt/cray/wlm_detect/1.0-1.0502.64649.2.1.ari/lib64 -Wl,-Ttext-segment=0x20000000,-zmax-page-size=0x20000000 -Wl,--whole-archive,-lhugetlbfs,--no-whole-archive -Wl,--no-as-needed,-lAtpSigHandler,-lAtpSigHCommData -Wl,--undefined=_ATP_Data_Globals -Wl,--undefined=__atpHandlerInstall -lpthread -lfftw3f_mpi -lfftw3f_threads -lfftw3f -lfftw3_mpi -lfftw3_threads -lfftw3 -lmpichf90_cray -lrt -lpthread -lugni -lpmi -lsci_cray_mpi_mp -lm -lf -lsci_cray_mp -lmpich_cray -lrt -lpthread -lugni -lpmi -lsci_cray_mp -lcraymp -lm -lpthread -lf -lpgas-dmapp -lfi -lu -lrt -Wl,--undefined=dmapp_get_flag_nbi -ldmapp -lugni -ludreg -lpthread -lm -lcray-c++-rts -lcraystdc++ -lxpmem -ldmapp -lpthread -lpmi -lpthread -lalpslli -lpthread -lwlm_detect -lugni -lpthread -lalpsutil -lpthread -lrca -ludreg -lquadmath -lm -lomp -lcraymp -lpthread -lrt -ldl -lcray-c++-rts -lcraystdc++ -lm -lmodules -lm -lfi -lm -lquadmath -lcraymath -lm -lgfortran -lquadmath -lf -lm -lpthread -lu -lrt -ldl -lcray-c++-rts -lcraystdc++ -lm -lcsup -Wl,--as-needed,-latomic,--no-as-needed -Wl,--as-needed,-lgfortran,--no-as-needed -Wl,--whole-archive,-ltcmalloc_minimal,--no-whole-archive -lstdc++ -lpthread -Wl,--start-group,-lc,-lcsup,-lgcc_eh,-lm,-lgcc,--end-group -Wl,-T/opt/cray/cce/8.5.1/craylibs/x86-64/2.23.1.cce.ld
ftn_driver.exe -hcpu=haswell -hstatic -D__CRAYXC -D__CRAY_HASWELL -D__CRAYXT_COMPUTE_LINUX_TARGET -hnetwork=aries hello.f90 -Wl,-rpath=/opt/cray/cce/8.5.1/craylibs/x86-64 -ulast_user_arg -I /opt/cray/cce/8.5.1/craylibs/x86-64/include -I/usr/include -L/opt/cray/cce/8.5.1/CC/x86-64/lib/x86-64 -Wl,-rpath=/opt/cray/cce/8.5.1/CC/x86-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 -Wl,-rpath=/opt/cray/gcc-libs -L/usr/lib64 -Wl,-rpath=/usr/lib64 -L/lib64 -Wl,-rpath=/lib64 -ugcc_base=/opt/gcc/4.8.1/snos -uno_driver_libs -Wl,/opt/cray/cce/8.5.1/craylibs/x86-64/libmodules.a,/opt/cray/cce/8.5.1/craylibs/x86-64/libomp.a,/opt/cray/cce/8.5.1/craylibs/x86-64/libopenacc.a, -I/opt/cray/fftw/3.3.4.9/haswell/include -I/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/include -I/opt/cray/libsci/16.07.1/CRAY/8.3/x86_64/include -I/opt/cray/rca/1.0.0-2.0502.60530.1.62.ari/include -I/opt/cray/alps/5.2.4-2.0502.9774.31.11.ari/include -I/opt/cray/xpmem/0.1-2.0502.64982.5.3.ari/include -I/opt/cray/gni-headers/4.0-1.0502.10859.7.8.ari/include -I/opt/cray/dmapp/7.0.1-1.0502.11080.8.76.ari/include -I/opt/cray/pmi/5.0.10-1.0000.11050.0.0.ari/include -I/opt/cray/ugni/6.0-1.0502.10863.8.29.ari/include -I/opt/cray/udreg/2.3.2-1.0502.10518.2.17.ari/include -I/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/../include -I/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/..//include -I/opt/cray/wlm_detect/1.0-1.0502.64649.2.1.ari/include -I/opt/cray/krca/1.0.0-2.0502.63139.4.31.ari/include -I/opt/cray-hss-devel/7.2.0/include -L/opt/cray/fftw/3.3.4.9/haswell/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/lib -L/opt/cray/libsci/16.07.1/CRAY/8.3/x86_64/lib -L/opt/cray/rca/1.0.0-2.0502.60530.1.62.ari/lib64 -L/opt/cray/alps/5.2.4-2.0502.9774.31.11.ari/lib64 -L/opt/cray/xpmem/0.1-2.0502.64982.5.3.ari/lib64 -L/opt/cray/dmapp/7.0.1-1.0502.11080.8.76.ari/lib64 -L/opt/cray/pmi/5.0.10-1.0000.11050.0.0.ari/lib64 -L/opt/cray/ugni/6.0-1.0502.10863.8.29.ari/lib64 -L/opt/cray/udreg/2.3.2-1.0502.10518.2.17.ari/lib64 -L/opt/cray/atp/2.0.2/libApp -L/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/../ -L/opt/cray/wlm_detect/1.0-1.0502.64649.2.1.ari/lib64 -Wl,-Ttext-segment=0x20000000,-zmax-page-size=0x20000000 -Wl,--whoUntitled 1le-archive,-lhugetlbfs,--no-whole-archive -Wl,--no-as-needed,-lAtpSigHandler,-lAtpSigHCommData -Wl,--undefined=_ATP_Data_Globals -Wl,--undefined=__atpHandlerInstall -lpthread -lfftw3f_mpi -lfftw3f_threads -lfftw3f -lfftw3_mpi -lfftw3_threads -lfftw3 -lmpichf90_cray -lrt -lpthread -lugni -lpmi -lsci_cray_mpi_mp -lm -lf -lsci_cray_mp -lmpich_cray -lrt -lpthread -lugni -lpmi -lsci_cray_mp -lcraymp -lm -lpthread -lf -lpgas-dmapp -lfi -lu -lrt -Wl,--undefined=dmapp_get_flag_nbi -ldmapp -lugni -ludreg -lpthread -lm -lcray-c++-rts -lcraystdc++ -lxpmem -ldmapp -lpthread -lpmi -lpthread -lalpslli -lpthread -lwlm_detect -lugni -lpthread -lalpsutil -lpthread -lrca -ludreg -lquadmath -lm -lomp -lcraymp -lpthread -lrt -ldl -lcray-c++-rts -lcraystdc++ -lm -lmodules -lm -lfi -lm -lquadmath -lcraymath -lm -lgfortran -lquadmath -lf -lm -lpthread -lu -lrt -ldl -lcray-c++-rts -lcraystdc++ -lm -lcsup -Wl,--as-needed,-latomic,--no-as-needed -Wl,--as-needed,-lgfortran,--no-as-needed -Wl,--whole-archive,-ltcmalloc_minimal,--no-whole-archive -lstdc++ -lpthread -Wl,--start-group,-lc,-lcsup,-lgcc_eh,-lm,-lgcc,--end-group -Wl,-T/opt/cray/cce/8.5.1/craylibs/x86-64/2.23.1.cce.ld
</pre>
</pre>


Line 50: Line 50:
CRAYPE_LINK_TYPE=dynamic
CRAYPE_LINK_TYPE=dynamic
</pre>
</pre>
== Building software for the front end nodes ==
By default the environment compiles and links software for the compute node architecture.
To build software for the front end nodes the CPU architecture has to be changed with
<pre>
$> module sw craype-haswell craype-sandybridge
</pre>


== documentation ==
== documentation ==
Line 66: Line 75:
|PrgEnv-cray || Cray Compilation Environment || crayftn, craycc, crayCC
|PrgEnv-cray || Cray Compilation Environment || crayftn, craycc, crayCC
|-
|-
|PrgEnv-intel || Intel Composer Suite || ifort, icc, icpc
|PrgEnv-intel || Intel CoUntitled 1mposer Suite || ifort, icc, icpc
|-
|-
|PrgEnv-gnu || GNU Compiler Collection || gfortran, gcc, g++
|PrgEnv-gnu || GNU Compiler Collection || gfortran, gcc, g++

Latest revision as of 08:32, 4 May 2018

Please use the compiler wrappers to compile for the compute nodes.

compiler wrapper

On HazelHen different compilers are available for different languages and different vendors. For all the different compilers please use compiler wrappers to compile your application for the compute nodes.

wrapper language
ftn FORTRAN
cc C
CC C++

Thus if you want to compile your application , e.g. including MPI, you just:

$> ftn hello.f90 -o my.exe

change options

On HazelHen compilers from CRAY, GNU, INTEL and PGI are available. The actual compiler used depend on the selected programming environment (cray/intel/gnu/pgi), which can be switched using e.g.:

$> module switch PrgEnv-cray PrgEnv-gnu

(Please review also the information about the module environment). The advantage of the module environment is that the modules automatically add the related compiler and linker flags the compile/link line. Thus you do not have to take care about these. For example, if you want to use FFTW, just load the module:

$> module load fftw

and recompile without additional options for this:

$> ftn myAppWithFFTW.f90 -o my.exe

You can see all the used compile and link options using the wrapper option -craype-verbose

$> ftn -craype-verbose myAppWithFFTW.f90 -o my.exe

ftn_driver.exe -hcpu=haswell -hstatic -D__CRAYXC -D__CRAY_HASWELL -D__CRAYXT_COMPUTE_LINUX_TARGET -hnetwork=aries hello.f90 -Wl,-rpath=/opt/cray/cce/8.5.1/craylibs/x86-64 -ulast_user_arg -I /opt/cray/cce/8.5.1/craylibs/x86-64/include -I/usr/include -L/opt/cray/cce/8.5.1/CC/x86-64/lib/x86-64 -Wl,-rpath=/opt/cray/cce/8.5.1/CC/x86-64/lib/x86-64 -L/opt/gcc/4.8.1/snos/lib64 -Wl,-rpath=/opt/cray/gcc-libs -L/usr/lib64 -Wl,-rpath=/usr/lib64 -L/lib64 -Wl,-rpath=/lib64 -ugcc_base=/opt/gcc/4.8.1/snos -uno_driver_libs -Wl,/opt/cray/cce/8.5.1/craylibs/x86-64/libmodules.a,/opt/cray/cce/8.5.1/craylibs/x86-64/libomp.a,/opt/cray/cce/8.5.1/craylibs/x86-64/libopenacc.a, -I/opt/cray/fftw/3.3.4.9/haswell/include -I/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/include -I/opt/cray/libsci/16.07.1/CRAY/8.3/x86_64/include -I/opt/cray/rca/1.0.0-2.0502.60530.1.62.ari/include -I/opt/cray/alps/5.2.4-2.0502.9774.31.11.ari/include -I/opt/cray/xpmem/0.1-2.0502.64982.5.3.ari/include -I/opt/cray/gni-headers/4.0-1.0502.10859.7.8.ari/include -I/opt/cray/dmapp/7.0.1-1.0502.11080.8.76.ari/include -I/opt/cray/pmi/5.0.10-1.0000.11050.0.0.ari/include -I/opt/cray/ugni/6.0-1.0502.10863.8.29.ari/include -I/opt/cray/udreg/2.3.2-1.0502.10518.2.17.ari/include -I/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/../include -I/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/..//include -I/opt/cray/wlm_detect/1.0-1.0502.64649.2.1.ari/include -I/opt/cray/krca/1.0.0-2.0502.63139.4.31.ari/include -I/opt/cray-hss-devel/7.2.0/include -L/opt/cray/fftw/3.3.4.9/haswell/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/lib -L/opt/cray/dmapp/default/lib64 -L/opt/cray/mpt/7.4.1/gni/mpich-cray/8.3/lib -L/opt/cray/libsci/16.07.1/CRAY/8.3/x86_64/lib -L/opt/cray/rca/1.0.0-2.0502.60530.1.62.ari/lib64 -L/opt/cray/alps/5.2.4-2.0502.9774.31.11.ari/lib64 -L/opt/cray/xpmem/0.1-2.0502.64982.5.3.ari/lib64 -L/opt/cray/dmapp/7.0.1-1.0502.11080.8.76.ari/lib64 -L/opt/cray/pmi/5.0.10-1.0000.11050.0.0.ari/lib64 -L/opt/cray/ugni/6.0-1.0502.10863.8.29.ari/lib64 -L/opt/cray/udreg/2.3.2-1.0502.10518.2.17.ari/lib64 -L/opt/cray/atp/2.0.2/libApp -L/opt/cray/cce/8.5.1/craylibs/x86-64/pkgconfig/../ -L/opt/cray/wlm_detect/1.0-1.0502.64649.2.1.ari/lib64 -Wl,-Ttext-segment=0x20000000,-zmax-page-size=0x20000000 -Wl,--whoUntitled 1le-archive,-lhugetlbfs,--no-whole-archive -Wl,--no-as-needed,-lAtpSigHandler,-lAtpSigHCommData -Wl,--undefined=_ATP_Data_Globals -Wl,--undefined=__atpHandlerInstall -lpthread -lfftw3f_mpi -lfftw3f_threads -lfftw3f -lfftw3_mpi -lfftw3_threads -lfftw3 -lmpichf90_cray -lrt -lpthread -lugni -lpmi -lsci_cray_mpi_mp -lm -lf -lsci_cray_mp -lmpich_cray -lrt -lpthread -lugni -lpmi -lsci_cray_mp -lcraymp -lm -lpthread -lf -lpgas-dmapp -lfi -lu -lrt -Wl,--undefined=dmapp_get_flag_nbi -ldmapp -lugni -ludreg -lpthread -lm -lcray-c++-rts -lcraystdc++ -lxpmem -ldmapp -lpthread -lpmi -lpthread -lalpslli -lpthread -lwlm_detect -lugni -lpthread -lalpsutil -lpthread -lrca -ludreg -lquadmath -lm -lomp -lcraymp -lpthread -lrt -ldl -lcray-c++-rts -lcraystdc++ -lm -lmodules -lm -lfi -lm -lquadmath -lcraymath -lm -lgfortran -lquadmath -lf -lm -lpthread -lu -lrt -ldl -lcray-c++-rts -lcraystdc++ -lm -lcsup -Wl,--as-needed,-latomic,--no-as-needed -Wl,--as-needed,-lgfortran,--no-as-needed -Wl,--whole-archive,-ltcmalloc_minimal,--no-whole-archive -lstdc++ -lpthread -Wl,--start-group,-lc,-lcsup,-lgcc_eh,-lm,-lgcc,--end-group -Wl,-T/opt/cray/cce/8.5.1/craylibs/x86-64/2.23.1.cce.ld

link type

On HazelHen the default link type is static, on the login nodes as well as on the mom/compute nodes. You can specify the link type using the -dynamic or -static compiler/linker option, e.g.:

ftn -dynamic mysrc.f90 -o myapp.exe

OR set the environment variable, e.g.:

 
CRAYPE_LINK_TYPE=dynamic


Building software for the front end nodes

By default the environment compiles and links software for the compute node architecture. To build software for the front end nodes the CPU architecture has to be changed with

$> module sw craype-haswell craype-sandybridge


documentation

Nevertheless, if you want to fine tune your applications, there a lot of compiler options you could try, like inlining, vectorization, etc. You find all these compiler options in the related man page, e.g.:

$> man crayftn

The compilers for the different PrgEnv are:

PrgEnv Description compiler
PrgEnv-cray Cray Compilation Environment crayftn, craycc, crayCC
PrgEnv-intel Intel CoUntitled 1mposer Suite ifort, icc, icpc
PrgEnv-gnu GNU Compiler Collection gfortran, gcc, g++
PrgEnv-pgi Portland Group Compilers pgf90, pgcc, pgCC