Darshan

Note: This page is in progress.

Darshan is a set of libraries that can characterize MPI-IO and POSIX file access within typical HPC applications in a nonintrusive way. It can be used to investigate I/O behavior of a MPI-program. Please make sure to always set MPI_SHEPHERD=true when using Darshan. Otherwise you might crash the nodes you are working on.

Developer: Argonne National Laboratory Platforms: HPE_Hawk Category: Performance Analyzer License: free software Website: Darshan homepage

Introduction

Darshan is developed by Mathematics and Computer Science Division of Argonne National Laboratory to allow investigation and tuning of the I/O behaviour of complex HPC applications. It is open-source and available on github. Many Tutorials and hands-on excercises are available online.

Usage

Darshan can be used with compile time wrappers or by dynamic library preloading. For detailed description, see 'Darshan-runtime installation and usage'.

Running program with Darshan instrumentation on Hawk using module environment

In Hawks module environment Darshan version 3.1.7 is available. If you need a different version you can install it in your home directory and follow the description mentioned above.

The easiest way to instrument your application is to preload the darshan library during runtime. You have to make sure your application is dynamically linked with the MPI library. If you are not sure if it is dynamically linked, type

ldd your_application

If it shows the MPI library, your application is dynamically linked.

libmpi.so => /opt/hpe/hpc/mpt/mpt-2.23/lib/libmpi.so (0x00007f5d32b33000)

Load the needed software module,

module load darshan-runtime

create directory to store Darshan logfile. It should have the folder structure year/month/day. Alternatively you can use darshans script darshan-mk-log-dirs.pl to create a wider folderstructure. You can find the script in the darshan library path.

mkdir "$(date +"%Y")" && mkdir "$(date +"%Y")"/"$(date +"%m")" && mkdir "$(date +"%Y")"/"$(date +"%m")"/"$(date +"%d")"

Export logpath environment variable,

export DARSHAN_LOGPATH=.

run your application with preloaded darshan library - here for Open MPI using '-x':

mpirun -x LD_PRELOAD=$DARSHAN_RUNTIME_HOME/lib/libdarshan.so-np 128 ./your_application

You will get a darshan logfile in the previously created directory. Its name will have your username, your application name and a bunch of numbers in it and go with the ending .darshan.

Using Darshan on Vulcan

Darshan is not preinstalled on Vulcan. You can install it in your HOME as follows:

Get the source code of Darshan Version 3.1.7 from the official website: Index of /pub/darshan/releases/ Not all Versions work correctly on Vulcan, but Version 3.1.7 was testet successfully. Install Darshan-runtime with:

cd darshan-runtime/

./configure --with-mem-align=8 --with-log-path-by-env=path/for/logfiles --with-jobid-env=NONE CC=mpicc --prefix=path/to/build-directory

make

make install

Parsing log file

A detailed documentation on parsing darshan outputs is available on 'Analyzing log files'.

Note that LaTeX is not preinstalled on HAWK, which is required if you want to parse your logfile using darshan-job-summary.pl.

See also

• Software Development Tools, Compilers & Libraries

External Links

• Darshan homepage