- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -

Vampirtrace: Difference between revisions

From HLRS Platforms
Jump to navigationJump to search
No edit summary
 
(22 intermediate revisions by 2 users not shown)
Line 1: Line 1:
During a program run of an application, VampirTrace generates an OTF trace file, which can be analyzed and visualized by the visualisation tool Vampir.
{{Infobox software
The VampirTrace library allows MPI communication events of a parallel program to be recorded as a trace file. Additionally, certain program-specific events can also be included.
| description = The VampirTrace library allows MPI communication events of a parallel program to be recorded as a trace file. Additionally, certain program-specific events can also be included. To record MPI communication events, simply relink the program with the VampirTrace library. A new compilation of the program source code is only necessary if program-specific events should be added.
To record MPI communication events, simply relink the program with the VampirTrace library. A new compilation of the program source code is only necessary if program-specific events should be added.
| developer              = ZIH TU Dresden and JSC/FZ Juelich
| available on          = [[NEC Nehalem Cluster]]
| category              = [[:Category:Performance Analyzer|Performance Analyzer]]
| license                = BSD License
| website                = [http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/vampirtrace/index_html/document_view?cl=enhttp://gcc.gnu.org/ VampirTrace homepage]
}}


== Introduction ==
== Introduction ==
Tracing with Vampir in three steps:  
Tracing an application with Vampir is done in three steps:  


# Compiling with the vampirtrace wrappercompiler
# Compiling with the vampirtrace wrappercompiler
Line 16: Line 21:
=== Compiling with the vampirtrace wrappercompiler ===
=== Compiling with the vampirtrace wrappercompiler ===


First load the needed software module:
{{Command
| command = module load performance/vampirtrace
}}
Now you can compile your application using the wrapper compilers <tt>vtcc</tt>, <tt>vtc++</tt> or <tt>vtf90</tt> as described in the user VampirTrace manual.
{{Note
| text = When using vampirtrace in combination with MPI you will need at least to specify the name of your mpi compiler to VampirTrace
<pre>
vtf90 -vt:f90 mpif90
vtcc  -vt:cc  mpicc
vtcxx -vt:cxx mpicxx
</pre>
}}


First you should load the needed software module:
You can specify the amount of instrumentation to be include in the trace using <tt>-vt</tt> options, e.g. <tt>-vt:mpi</tt> for MPI instrumentation,
<tt>-vt:compinst</tt> for compiler-based function instrumentation:
{{Command|command =
vtcc -vt:mpi -vt:inst compinst -o myapp myapp.c
}}


module load performance/vampirtrace


Now you should compile your application using the wrapper compilers vtcc, vtf90, ... as described in the user VampirTrace manual.
==== Hermit special: ====


For MPI you will need at least to specify the name of your mpi compiler to vampirtrace  
{{Warning|text=On Hermit it is necessary to add the <tt>-vt:mpi</tt> option. Otherwise the wrong vampirtrace libraries will be used.}}
{{Command
| command = vtcc -vt:mpi [other options] my_app
}}
==== BW-Grid special: ====


vtcc  -vt:cc  mpicc
On the BW-Grid you first have to login to a node for compilation as some libraries are not accessible from the frontend:
vtf90 -vt:f90 mpif90
{{Command
| command = qsub -I -l walltime=1:00:00
}}


=== Generating the trace files ===
=== Generating the trace files ===


Run your applicatin as usual. This will generate the needed tracefiles
Run your application as usual. This will generate the needed trace files.
 
==== BW-Grid special: ====


On the BW-Grid you first have to login to a node as some libs are not accessible from the frontend
Which data are collected during the run can be controlled with the ''VT_*'' options. Options which may be helpful during analysis are ''VT_BUFFER_SIZE'', ''VT_MAX_FLUSHES'', ''VT_PFORM_LDIR'' or ''VT_METRICS''. See the Vampirtrace documentation for more details.


qsub -I -l walltime=1:00:00
=== Viewing traces ===


To analyze the generated traces you can use [[Vampir]]. For large traces (>500MB), e.g. generated with more than 16 PEs in the application run, it is recommended to use [[Vampir#VampirServer | VampirServer]].


=== Viewing traces with vampir or Vampir-Server ===
Because Vampir and Vampir-Server are GUIs don't forget to use X-forwarding when you login via ssh.
You will need read access to all generated output files (<code>*.def.z, *.events.z, *.oft</code>).


==== Vampir ====
== Trouble shooting ==
Vampir is the sequential version of the graphical trace file analyzing software. It is capable of analyzing smaller trace files and should not be used for files which have 100MB or more.
In this section common errors and solutions to overcome the problems are listed.
 
==== Vampir-Server ====
Vampir-Server is the parallel version of the graphical trace file analyzing software. It can be found on the [[SX-8]] frontend (a1.hww.de). The tool consists of a client and a server. The server itself is a parallel MPI Program.
 
First we have to load the necessary software modules
 
module load vampir-server/1.10.0
 
Now start the server
 
vngd &
 
and remember the port number it displays. Then start the graphical client
 
vng
 
The last step is to connect the client to the server. This is done under
 
file <math>\rightarrow</math> Connect to Server ...
 
 
Then you can open your traces via


  file <math>\rightarrow</math> Open Tracefile ...
==== Incomplete trace files ====
  [0]VampirTrace: Maximum number of buffer flushes reached (VT_MAX_FLUSHES=1)
  [0]VampirTrace: Tracing switched off permanently
Trace data did not fit into a single VampirTrace buffer. Unlimit the number of buffer reuses with
{{Command|command = export VT_MAX_FLUSHES=0}}


Select the '''' *.oft '''' file
==== Not enough space on /tmp ====
[0]VampirTrace: FATAL: OTF_WStream_writeLeave failed:
ERROR in function OTF_WBuffer_writeUint64, ...
OTF_WBuffer_guarantee() failed.


== Trouble shooting ==
Check if /tmp has filled up to 100% by executing
{{Command|command=df -h <nowiki>|</nowiki> grep /tmp}}
A solution is to use a workspace in combination with the  <tt>VT_PROFM_LDIR</tt> variable:
{{Command|command = export VT_PROFM_LDIR=$(ws_allocate vt_trace_tmp 5)}}


If the program crashes unexpectedly it is possible that there is not enough space on the node local /tmp. A solution is to use a workspace in combination with the  VT_PROFM_LDIR variable
==== Library issues on NEC ====
Try to export <tt>LD_LIBRARY_PATH</tt>
{{Command
| command = export LD_LIBRARY_PATH=/opt/NECcomp/compiler100/ia64/lib
}}


In some cases problems concerning library issues occure. In this case it could help
== See also ==
* [[Software Development Tools, Compilers & Libraries]]


export LD_LIBRARY_PATH=/opt/NECcomp/compiler100/ia64/lib
== External Links ==
* [http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/vampirtrace/index_html/document_view?cl=en Vampirtrace Homepage]

Latest revision as of 14:02, 25 January 2013

The VampirTrace library allows MPI communication events of a parallel program to be recorded as a trace file. Additionally, certain program-specific events can also be included. To record MPI communication events, simply relink the program with the VampirTrace library. A new compilation of the program source code is only necessary if program-specific events should be added.
Developer: ZIH TU Dresden and JSC/FZ Juelich
Platforms: NEC Nehalem Cluster
Category: Performance Analyzer
License: BSD License
Website: VampirTrace homepage


Introduction

Tracing an application with Vampir is done in three steps:

  1. Compiling with the vampirtrace wrappercompiler
  2. Running the application
  3. Analysing the trace files using vampir or vampir-server

For detailed usage of Vampirtrace see the user manual which can either be found on your local machine or in the internet.

Usage

Compiling with the vampirtrace wrappercompiler

First load the needed software module:

module load performance/vampirtrace

Now you can compile your application using the wrapper compilers vtcc, vtc++ or vtf90 as described in the user VampirTrace manual.

Note: When using vampirtrace in combination with MPI you will need at least to specify the name of your mpi compiler to VampirTrace
vtf90 -vt:f90 mpif90
vtcc  -vt:cc  mpicc
vtcxx -vt:cxx mpicxx


You can specify the amount of instrumentation to be include in the trace using -vt options, e.g. -vt:mpi for MPI instrumentation, -vt:compinst for compiler-based function instrumentation:

vtcc -vt:mpi -vt:inst compinst -o myapp myapp.c


Hermit special:

Warning: On Hermit it is necessary to add the -vt:mpi option. Otherwise the wrong vampirtrace libraries will be used.
vtcc -vt:mpi [other options] my_app

BW-Grid special:

On the BW-Grid you first have to login to a node for compilation as some libraries are not accessible from the frontend:

qsub -I -l walltime=1:00:00


Generating the trace files

Run your application as usual. This will generate the needed trace files.

Which data are collected during the run can be controlled with the VT_* options. Options which may be helpful during analysis are VT_BUFFER_SIZE, VT_MAX_FLUSHES, VT_PFORM_LDIR or VT_METRICS. See the Vampirtrace documentation for more details.

Viewing traces

To analyze the generated traces you can use Vampir. For large traces (>500MB), e.g. generated with more than 16 PEs in the application run, it is recommended to use VampirServer.


Trouble shooting

In this section common errors and solutions to overcome the problems are listed.

Incomplete trace files

[0]VampirTrace: Maximum number of buffer flushes reached (VT_MAX_FLUSHES=1)
[0]VampirTrace: Tracing switched off permanently

Trace data did not fit into a single VampirTrace buffer. Unlimit the number of buffer reuses with

export VT_MAX_FLUSHES=0


Not enough space on /tmp

[0]VampirTrace: FATAL: OTF_WStream_writeLeave failed:
ERROR in function OTF_WBuffer_writeUint64, ...
OTF_WBuffer_guarantee() failed.

Check if /tmp has filled up to 100% by executing

df -h | grep /tmp

A solution is to use a workspace in combination with the VT_PROFM_LDIR variable:

export VT_PROFM_LDIR=$(ws_allocate vt_trace_tmp 5)


Library issues on NEC

Try to export LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/opt/NECcomp/compiler100/ia64/lib


See also

External Links