- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -

Difference between revisions of "Vampirtrace"

From HLRS Platforms
Jump to navigationJump to search
(Solutions to common issues: Full /tmp file system; maximum buffer flushes)
Line 110: Line 110:
  OTF_WBuffer_guarantee() failed.
  OTF_WBuffer_guarantee() failed.
Check if the /tmp has filled up to 100% by executing
Check if /tmp has filled up to 100% by executing
{{Command|command=df -h <nowiki>|</nowiki> grep /tmp}}
{{Command|command=df -h <nowiki>|</nowiki> grep /tmp}}
A solution is to use a workspace in combination with the  <tt>VT_PROFM_LDIR</tt> variable:
A solution is to use a workspace in combination with the  <tt>VT_PROFM_LDIR</tt> variable:

Revision as of 12:19, 3 July 2012

The VampirTrace library allows MPI communication events of a parallel program to be recorded as a trace file. Additionally, certain program-specific events can also be included. To record MPI communication events, simply relink the program with the VampirTrace library. A new compilation of the program source code is only necessary if program-specific events should be added.
Developer: ZIH TU Dresden and JSC/FZ Juelich
Platforms: NEC Nehalem Cluster
Category: Performance Analyzer
License: BSD License
Website: VampirTrace homepage


Tracing an application with Vampir is done in three steps:

  1. Compiling with the vampirtrace wrappercompiler
  2. Running the application
  3. Analysing the trace files using vampir or vampir-server

For detailed usage of Vampirtrace see the user manual which can either be found on your local machine or in the internet.


Compiling with the vampirtrace wrappercompiler

First load the needed software module:

module load performance/vampirtrace

Now you can compile your application using the wrapper compilers vtcc, vtc++ or vtf90 as described in the user VampirTrace manual.

Note: When using vampirtrace in combination with MPI you will need at least to specify the name of your mpi compiler to VampirTrace
vtf90 -vt:f90 mpif90
vtcc  -vt:cc  mpicc
vtcxx -vt:cxx mpicxx

Generating the trace files

Run your applicatin as usual. This will generate the needed tracefiles

BW-Grid special:

On the BW-Grid you first have to login to a node as some libs are not accessible from the frontend

qsub -I -l walltime=1:00:00

Viewing traces with vampir or Vampir-Server

There are two GUIs available to view your trace files: Vampir and Vampir-Server. For both you will need read access to all generated output files (*.def.z, *.events.z, *.otf).

Note: Because Vampir and Vampir-Server are GUIs don't forget to use X-forwarding when you login via ssh.


Vampir is the sequential version of the graphical trace file analyzing software. It is capable of analyzing smaller trace files and should not be used for files which have 100MB or more.


Vampir-Server is the parallel version of the graphical trace file analyzing software. It can be found on the SX-8 frontend (a1.hww.de). The tool consists of a client and a server. The server itself is a parallel MPI Program.

First load the necessary software module

module load vampir-server

Now start the server and afterwards the client

on laki:

mpirun -np 4 vngd &

on hermit:

aprun -n 4 vngd &

Remember the port number which will be displays as you will need it later on. Then start the graphical client


The last step is to connect the client to the server using the port number offered you from the server. This is done under

file <math>\rightarrow</math> Connect to Server ...

Then you can open your traces via

file <math>\rightarrow</math> Open Tracefile  ...

Select the desired *.otf file

Trouble shooting

Incomplete trace files

[0]VampirTrace: Maximum number of buffer flushes reached (VT_MAX_FLUSHES=1)
[0]VampirTrace: Tracing switched off permanently

Trace data did not fit into a single VampirTrace buffer. Unlimit the number of buffer reuses with


Not enough space on /tmp

[0]VampirTrace: FATAL: OTF_WStream_writeLeave failed:
ERROR in function OTF_WBuffer_writeUint64, ...
OTF_WBuffer_guarantee() failed.

Check if /tmp has filled up to 100% by executing

df -h | grep /tmp

A solution is to use a workspace in combination with the VT_PROFM_LDIR variable:

export VT_PROFM_LDIR=$(ws_allocate vt_trace_tmp 5)

Library issues on NEC

Try to export LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/opt/NECcomp/compiler100/ia64/lib

See also

External Links