- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -
Darshan: Difference between revisions
No edit summary |
(Update mpirun command to use darshan) |
||
| (9 intermediate revisions by one other user not shown) | |||
| Line 2: | Line 2: | ||
{{Infobox software | {{Infobox software | ||
| description = Darshan is a set of libraries that can characterize MPI-IO and POSIX file access within typical HPC applications in a nonintrusive way. It can be used to investigate I/O behavior of a MPI-program. | | description = Darshan is a set of libraries that can characterize MPI-IO and POSIX file access within typical HPC applications in a nonintrusive way. It can be used to investigate I/O behavior of a MPI-program. | ||
Please make sure to always set MPI_SHEPHERD=true when using Darshan. Otherwise you might crash the nodes you are working on. | Please make sure to always set <tt>MPI_SHEPHERD=true</tt> when using Darshan. Otherwise you might crash the nodes you are working on. | ||
| developer = Argonne National Laboratory | | developer = Argonne National Laboratory | ||
| available on = [[HPE_Hawk]] | | available on = [[HPE_Hawk]] | ||
| Line 16: | Line 16: | ||
== Usage == | == Usage == | ||
Darshan can be used with compile time wrappers or by dynamic library preloading. For detailed description, see [https://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html 'Darshan-runtime installation and usage']. | Darshan can be used with compile time wrappers or by dynamic library preloading. For detailed description, see [https://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html 'Darshan-runtime installation and usage']. | ||
=== Running program with Darshan instrumentation on Hawk using module === | === Running program with Darshan instrumentation on Hawk using module environment === | ||
In Hawks module environment Darshan version 3.1.7 is available. If you need a different version you can install it in your home directory and follow the description mentioned above. | In Hawks module environment Darshan version 3.1.7 is available. If you need a different version you can install it in your home directory and follow the description mentioned above. | ||
| Line 31: | Line 31: | ||
module load darshan-runtime | module load darshan-runtime | ||
}} | }} | ||
create directory to store Darshan logfile. It should have the folder structure year/month/day. | create directory to store Darshan logfile. It should have the folder structure year/month/day. Alternatively you can use darshans script ''darshan-mk-log-dirs.pl'' to create a wider folderstructure. You can find the script in the darshan library path. | ||
{{Command | {{Command | ||
| command = | | command = | ||
| Line 41: | Line 41: | ||
export DARSHAN_LOGPATH=. | export DARSHAN_LOGPATH=. | ||
}} | }} | ||
run your application with preloaded darshan library | run your application with preloaded darshan library - here for Open MPI using '-x': | ||
{{Command | {{Command | ||
| command = | | command = | ||
LD_PRELOAD= | mpirun -x LD_PRELOAD=$DARSHAN_RUNTIME_HOME/lib/libdarshan.so-np 128 ./your_application | ||
}} | }} | ||
You will get a darshan logfile in the previously created directory. Its name will have your username, your application name and a bunch of numbers in it and go with the ending ''.darshan''. | You will get a darshan logfile in the previously created directory. Its name will have your username, your application name and a bunch of numbers in it and go with the ending ''.darshan''. | ||
=== Using Darshan on Vulcan === | |||
Darshan is not preinstalled on Vulcan. You can install it in your HOME as follows: | |||
Get the source code of Darshan Version 3.1.7 from the official website: [https://ftp.mcs.anl.gov/pub/darshan/releases/ Index of /pub/darshan/releases/] | |||
Not all Versions work correctly on Vulcan, but Version 3.1.7 was testet successfully. Install Darshan-runtime with: | |||
{{Command | |||
| command = | |||
cd darshan-runtime/ | |||
./configure --with-mem-align=8 --with-log-path-by-env=path/for/logfiles --with-jobid-env=NONE CC=mpicc --prefix=path/to/build-directory | |||
make | |||
make install | |||
}} | |||
=== Parsing log file === | === Parsing log file === | ||
Latest revision as of 14:41, 20 April 2023
Note: This page is in progress.
| Darshan is a set of libraries that can characterize MPI-IO and POSIX file access within typical HPC applications in a nonintrusive way. It can be used to investigate I/O behavior of a MPI-program.
Please make sure to always set MPI_SHEPHERD=true when using Darshan. Otherwise you might crash the nodes you are working on. |
| ||||||||||||
Introduction
Darshan is developed by Mathematics and Computer Science Division of Argonne National Laboratory to allow investigation and tuning of the I/O behaviour of complex HPC applications. It is open-source and available on github. Many Tutorials and hands-on excercises are available online.
Usage
Darshan can be used with compile time wrappers or by dynamic library preloading. For detailed description, see 'Darshan-runtime installation and usage'.
Running program with Darshan instrumentation on Hawk using module environment
In Hawks module environment Darshan version 3.1.7 is available. If you need a different version you can install it in your home directory and follow the description mentioned above.
The easiest way to instrument your application is to preload the darshan library during runtime. You have to make sure your application is dynamically linked with the MPI library. If you are not sure if it is dynamically linked, type
If it shows the MPI library, your application is dynamically linked.
libmpi.so => /opt/hpe/hpc/mpt/mpt-2.23/lib/libmpi.so (0x00007f5d32b33000)
Load the needed software module,
create directory to store Darshan logfile. It should have the folder structure year/month/day. Alternatively you can use darshans script darshan-mk-log-dirs.pl to create a wider folderstructure. You can find the script in the darshan library path.
Export logpath environment variable,
run your application with preloaded darshan library - here for Open MPI using '-x':
You will get a darshan logfile in the previously created directory. Its name will have your username, your application name and a bunch of numbers in it and go with the ending .darshan.
Using Darshan on Vulcan
Darshan is not preinstalled on Vulcan. You can install it in your HOME as follows:
Get the source code of Darshan Version 3.1.7 from the official website: Index of /pub/darshan/releases/ Not all Versions work correctly on Vulcan, but Version 3.1.7 was testet successfully. Install Darshan-runtime with:
./configure --with-mem-align=8 --with-log-path-by-env=path/for/logfiles --with-jobid-env=NONE CC=mpicc --prefix=path/to/build-directory
make
make install
Parsing log file
A detailed documentation on parsing darshan outputs is available on 'Analyzing log files'.
Note that LaTeX is not preinstalled on HAWK, which is required if you want to parse your logfile using darshan-job-summary.pl.