- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -

Advisor: Difference between revisions

From HLRS Platforms
Jump to navigationJump to search
 
(38 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==
{{Infobox software
{{Infobox software
| description = Intel® '''Advisor XE''' is a low-weight threading assistant for C, C++, C# and Fortran. It guides developers through threading design, automating analyses required for fast and correct implementation. It helps developers to add parallelism to their existing C/C++ or Fortran programs.
| description = Intel® '''Advisor XE''' is a low-weight threading assistant for C, C++, C# and Fortran. It guides developers through threading design, automating analyses required for fast and correct implementation. It also helps developers to add parallelism to their existing C/C++ or Fortran programs.  
 
Apart from thread parallelism, Advisor also supports analyzing MPI-parallel applications.  The overall efficiency of an MPI-parallel loop/function can be measured by manually adding individual bandwidths and performances. For example, one runs an application on two MPI ranks, attaching Advisor to both the MPI tasks. Let us assume that for a specific loop/function the bandwidths corresponding to rank-1 and rank-2 turn out to be  X1 GB/sec, X2 GB/sec and performances Y1 GF/sec, Y2 GF/sec, respectively. Then, the total bandwidth would be (X1+X2) GB/sec and total performance (Y1+Y2) GF/sec. If the values of X1, X2 or Y1,Y2 are significantly different, then there is a load imbalance between the ranks. If the load balance among MPI-ranks is good, running Advisor on a single rank would be enough.


Apart from thread parallelism, Advisor also supports analyzing MPI-parallel applications.  The overall efficiency of an MPI-parallel loop/function can be measured by manually adding individual bandwidths and performances. For example, one runs an application on "n" MPI ranks, attaching Advisor to each rank. Let us assume that for a specific loop/function the bandwidths corresponding to rank-1 to rank-n turn out to be (X1, X2,...,Xn) GB/sec and the respective performances are (Y1, Y2,...,Yn) GF/sec. Then, the total bandwidth would be (X1+X2+...+Xn) GB/sec and total performance (Y1+Y2+...+Yn) GF/sec. Note that, if there is a significant deviation among the values of Xi or Yi (i=1,2,...n), then there is a load imbalance  among the ranks and one may add "--mpi-trace" flag to the survey command and repeat the analysis. However, if the load balance among MPI-ranks is good, running Advisor on a single rank would be enough.


| logo = [[Image:intel-logo.png]]
| logo = [[Image:intel-logo.png]]
Line 13: Line 12:
| website                = [http://software.intel.com/en-us/intel-advisor-xe Intel® Advisor XE homepage]  
| website                = [http://software.intel.com/en-us/intel-advisor-xe Intel® Advisor XE homepage]  
}}  
}}  
 
In brief, one may use the Intel Advisor XE to:
In summary, one may use the Intel Advisor XE to:
* find the most time-consuming serial code regions in your program.
* find the most time-consuming serial code regions in your program.
* analyse Roofline plot, which highlights hot functions/loops and suggests necessary optimizations. With the help of Roofline plot, one can confirm whether an application is memory-bound or compute-bound.  
* analyse Roofline plot, which highlights hot functions/loops and suggests necessary optimizations. With the help of Roofline plot, one can confirm whether an application is memory-bound or compute-bound.  
Line 25: Line 23:
* check for data sharing problems that could prevent the application from working correctly when parallelized.
* check for data sharing problems that could prevent the application from working correctly when parallelized.


Slides for a  general introduction about the Advisor can be found [https://kb.hlrs.de/platforms/upload/General_presentation_Advisor_2017.pdf here].  
'''''Slides for a  general introduction about the Advisor can be found [https://kb.hlrs.de/platforms/upload/General_presentation_Advisor_2017.pdf here].'''''


For the slides on the memory access pattern, one may click [https://kb.hlrs.de/platforms/upload/MAP_Analysis_Advisor_2017.pdf here].  
'''''For the slides on the memory access pattern, one may click [https://kb.hlrs.de/platforms/upload/MAP_Analysis_Advisor_2017.pdf here].'''''


Slides on vectorization and dependency can be found [https://kb.hlrs.de/platforms/upload/MAP_Analysis_Advisor_2017.pdf here].  
'''''Slides on vectorization and dependency can be found [https://kb.hlrs.de/platforms/upload/MAP_Analysis_Advisor_2017.pdf here].'''''


For the last three points in the introduction section, one may refer to the [https://kb.hlrs.de/platforms/upload/Tutorial_2013_Advisor.pdf tutorial here].
'''''For the last three points starting from "insert Intel Advisor XE annotations..." in the introduction section, one may refer to the [https://kb.hlrs.de/platforms/upload/Tutorial_2013_Advisor.pdf tutorial here].'''''


== Why Intel Advisor? ==
== Why Intel Advisor? ==
Line 44: Line 42:
* whether there are dependencies hindering vectorization
* whether there are dependencies hindering vectorization


* where different loops/functions lie on the Roofline plot etc.
* where different loops/functions lie on the Roofline plot and if there is a room for improvement, etc.  
 
Intel Advisor not only provide answers to all the above-mentioned queries, but also suggests solutions, for example, what kind of optimizations one needs to implement in order to improve the performance of the application.
 


Intel Advisor not only provides answer to all the above-mentioned queries, but also suggests solutions, for example, what kind of optimizations one needs to implement in order to improve the performance of an application.


== How to use Intel Advisor? ==
== How to use Intel Advisor? ==
Line 79: Line 75:
   advixe-cl -collect tripcounts -flop -project-dir results_advisor  ./a.out
   advixe-cl -collect tripcounts -flop -project-dir results_advisor  ./a.out


Here, survey is an internal tool which locates non-vectorized and poorly vectorized loops/functions and estimates performance gain with efficient vectorization. Trip counts introduces counters to measure time spent in a particular loop/function, and the flag “-flop” enables the flop counter.
Here, survey is an internal tool which locates non-vectorized and poorly vectorized loops/functions and estimates performance gain with efficient vectorization. Trip counts introduces counters to measure time spent in a particular loop/function, and the flag “-flop” enables the flop counter. It is necessary to collect survey, tripcounts and flops in order to see the Roofline plot.


Results can be visualized using Advisor GUI,  
Results can be visualized using Advisor GUI,  
Line 85: Line 81:
   advixe-gui results_advisor/e000/e000.advixeexp
   advixe-gui results_advisor/e000/e000.advixeexp


Visualizing results on Hawk could be slow, one may thus alternatively pack up all the results in a read-only file as follows  
Visualizing results on Hawk could be slow if the GUI contents are being transferred via thin DSL lines. Therefore, one shall try using [https://kb.hlrs.de/platforms/index.php/Hawk_PrePostProcessing VNC] for the purpose of visualization. Alternatively, one may pack up all the results in a read-only file as follows  


   advixe-cl --snapshot --project-dir=results_advisor --cache-sources path_to_source_code --cache-binaries path_to_binary
   advixe-cl --snapshot --project-dir=results_advisor --cache-sources path_to_source_code --cache-binaries path_to_binary


Above command will create a file snapshot000.advixeexpz which requires very less memory as compared to the original results_advisor directory can be easily copied to the local machine. The file can be viewed in GUI as,
Above command will create a file snapshot000.advixeexpz which requires very less memory as compared to the original results_advisor directory and thus can be easily copied/stored to the local machine. The file can be viewed in GUI as,


   advixe-gui snapshot000.advixeexpz
   advixe-gui snapshot000.advixeexpz
Line 95: Line 91:
=== Running Advisor on MPI parallel application ===  
=== Running Advisor on MPI parallel application ===  


Collect survey, tripcounts and flops as follows,  
In order to get useful information from the Advisor analysis, it is necessary to pin MPI ranks according to the machine hardware.
This is possible only when command line arguments are exported as variables. For example, if one would like to use OpenMPI and wants to distribute 8 MPI ranks uniformly over the Hawk node. This can be achieved with the following steps:
 
Export number of MPI ranks
export NUM_MPI=8
 
Then export pinning in a variable
 
export MPIRUN_OPTIONS="--bind-to cpu-list:ordered --cpu-list 0,16,32,48,64,80,96,112 -report-bindings"


  mpirun -np $num_of_mpi_tasks advixe-cl -collect survey -project-dir results_advisor  ./a.out
Export survey and tripcounts command as follows


  mpirun -np $num_of_mpi_tasks advixe-cl -collect tripcounts -flop -project-dir results_advisor ./a.out
export ADVISOR_SURVEY="advixe-cl -collect survey -project-dir results_advisor"


Above command will create Advisor reports for all the ranks. In case, one would like to run Advisor only on the single rank, then do the following,
export ADVISOR_TRIPCOUNTS="advixe-cl -collect tripcounts -flop -project-dir results_advisor"
 
Now collect survey and tripcounts as
 
mpirun -np ${NUM_MPI} ${MPIRUN_OPTIONS} ${ADVISOR_SURVEY} ./a.out
 
wait
 
mpirun -np ${NUM_MPI} ${MPIRUN_OPTIONS} ${ADVISOR_TRIPCOUNTS}  ./a.out
   
   
  mpirun -np $num_of_mpi_tasks-1 ./a.out : -np 1 advixe-cl -collect survey -project-dir results_advisor  ./a.out
Note that in the batch-scipt, as shown above, it is recommended to put a "wait" command between two "mpirun" commands. Above commands will create Advisor reports for all the ranks. In case, in the above example, one would like to run Advisor only on the single rank, then do the following,


  mpirun -np $num_of_mpi_tasks-1 ./a.out : -np 1 advixe-cl -collect tripcounts -flop -project-dir results_advisor  ./a.out
export MPIRUN_OPTIONS_1="--bind-to cpu-list:ordered --cpu-list 0,16,32,48,64,80,96"
 
export MPIRUN_OPTIONS_2="--bind-to cpu-list:ordered --cpu-list 112"
   
   
===== Running Advisor on MPI parallel application with pinning =====
Now collect survey and tripcounts as follows,
 
mpirun -np ${NUM_MPI}-1 ${MPIRUN_OPTIONS_1} ./a.out : -np 1 ${MPIRUN_OPTIONS_2} ${ADVISOR_SURVEY} ./a.out
 
wait
 
mpirun -np ${NUM_MPI}-1 ${MPIRUN_OPTIONS_1} ./a.out : -np 1 ${MPIRUN_OPTIONS_2} ${ADVISOR_TRIPCOUNTS}  ./a.out


===  Running Advisor on MPI+OpenMP parallel application ===  
===  Running Advisor on MPI+OpenMP parallel application ===  


The following example employs 32 MPI tasks distributed uniformly over both the sockets with 2 OpenMP threads per MPI tasks on a Hawk node.   
The following example employs 32 MPI ranks distributed uniformly over both the sockets with 2 OpenMP threads per MPI tasks on a Hawk node.   


   module load mpt
   module load mpt

Latest revision as of 09:42, 20 January 2022

Introduction

Intel® Advisor XE is a low-weight threading assistant for C, C++, C# and Fortran. It guides developers through threading design, automating analyses required for fast and correct implementation. It also helps developers to add parallelism to their existing C/C++ or Fortran programs.

Apart from thread parallelism, Advisor also supports analyzing MPI-parallel applications. The overall efficiency of an MPI-parallel loop/function can be measured by manually adding individual bandwidths and performances. For example, one runs an application on "n" MPI ranks, attaching Advisor to each rank. Let us assume that for a specific loop/function the bandwidths corresponding to rank-1 to rank-n turn out to be (X1, X2,...,Xn) GB/sec and the respective performances are (Y1, Y2,...,Yn) GF/sec. Then, the total bandwidth would be (X1+X2+...+Xn) GB/sec and total performance (Y1+Y2+...+Yn) GF/sec. Note that, if there is a significant deviation among the values of Xi or Yi (i=1,2,...n), then there is a load imbalance among the ranks and one may add "--mpi-trace" flag to the survey command and repeat the analysis. However, if the load balance among MPI-ranks is good, running Advisor on a single rank would be enough.

Intel-logo.png
Developer: Intel
Platforms:
Category: Performance Analyzer
License: Commercial
Website: Intel® Advisor XE homepage

In brief, one may use the Intel Advisor XE to:

  • find the most time-consuming serial code regions in your program.
  • analyse Roofline plot, which highlights hot functions/loops and suggests necessary optimizations. With the help of Roofline plot, one can confirm whether an application is memory-bound or compute-bound.
  • analyse memory access pattern to know if the memory is being accessed in unit stride or non-unit stride.
  • explore loop-carried dependencies hindering efficient vectorization.
  • find out if there are data-type conversions hindering vectorization in the code.
  • estimate the load imbalance and parallel efficiency in an MPI-parallel application.
  • insert Intel Advisor XE annotations to identify these as possible parallel code regions.
  • predict the approximate parallel performance characteristics of the proposed parallel code regions.
  • check for data sharing problems that could prevent the application from working correctly when parallelized.

Slides for a general introduction about the Advisor can be found here.

For the slides on the memory access pattern, one may click here.

Slides on vectorization and dependency can be found here.

For the last three points starting from "insert Intel Advisor XE annotations..." in the introduction section, one may refer to the tutorial here.

Why Intel Advisor?

Before checking the parallel efficiency of an application, it is necessary to understand how the application behaves at the core and node level. For example,

  • whether it is memory bound or compute bound
  • how good is the vectorization
  • how is the memory access pattern
  • whether there are dependencies hindering vectorization
  • where different loops/functions lie on the Roofline plot and if there is a room for improvement, etc.

Intel Advisor not only provides answer to all the above-mentioned queries, but also suggests solutions, for example, what kind of optimizations one needs to implement in order to improve the performance of an application.

How to use Intel Advisor?

First, compile your application with the flag "-g" followed by other optimization flags, for example on Hawk "-O2 (or -O3) -march=core-avx2". Then, set up an environment for the Advisor by loading the corresponding module.

For example, on Hawk

 module load advisor

On Vulcan

 module load performance/advisor

If you have installed Intel oneAPI on your laptop then,

 source /opt/intel/oneapi/setvars.sh

Running Advisor on OpenMP parallel application

Select the number of OpenMP threads as,

 export OMP_NUM_THREADS=num_of_threads

and bind them as,

 export OMP_PROC_BIND=spread

Afterwards, collect survey, tripcounts and flops as follows,

  advixe-cl -collect survey -project-dir results_advisor  ./a.out
  advixe-cl -collect tripcounts -flop -project-dir results_advisor  ./a.out

Here, survey is an internal tool which locates non-vectorized and poorly vectorized loops/functions and estimates performance gain with efficient vectorization. Trip counts introduces counters to measure time spent in a particular loop/function, and the flag “-flop” enables the flop counter. It is necessary to collect survey, tripcounts and flops in order to see the Roofline plot.

Results can be visualized using Advisor GUI,

  advixe-gui results_advisor/e000/e000.advixeexp

Visualizing results on Hawk could be slow if the GUI contents are being transferred via thin DSL lines. Therefore, one shall try using VNC for the purpose of visualization. Alternatively, one may pack up all the results in a read-only file as follows

  advixe-cl --snapshot --project-dir=results_advisor --cache-sources path_to_source_code --cache-binaries path_to_binary

Above command will create a file snapshot000.advixeexpz which requires very less memory as compared to the original results_advisor directory and thus can be easily copied/stored to the local machine. The file can be viewed in GUI as,

  advixe-gui snapshot000.advixeexpz

Running Advisor on MPI parallel application

In order to get useful information from the Advisor analysis, it is necessary to pin MPI ranks according to the machine hardware. This is possible only when command line arguments are exported as variables. For example, if one would like to use OpenMPI and wants to distribute 8 MPI ranks uniformly over the Hawk node. This can be achieved with the following steps:

Export number of MPI ranks

export NUM_MPI=8

Then export pinning in a variable

export MPIRUN_OPTIONS="--bind-to cpu-list:ordered --cpu-list 0,16,32,48,64,80,96,112 -report-bindings"

Export survey and tripcounts command as follows

export ADVISOR_SURVEY="advixe-cl -collect survey -project-dir results_advisor"
export ADVISOR_TRIPCOUNTS="advixe-cl -collect tripcounts -flop -project-dir results_advisor"

Now collect survey and tripcounts as

mpirun -np ${NUM_MPI} ${MPIRUN_OPTIONS} ${ADVISOR_SURVEY} ./a.out 
wait
mpirun -np ${NUM_MPI} ${MPIRUN_OPTIONS} ${ADVISOR_TRIPCOUNTS}  ./a.out

Note that in the batch-scipt, as shown above, it is recommended to put a "wait" command between two "mpirun" commands. Above commands will create Advisor reports for all the ranks. In case, in the above example, one would like to run Advisor only on the single rank, then do the following,

export MPIRUN_OPTIONS_1="--bind-to cpu-list:ordered --cpu-list 0,16,32,48,64,80,96"
export MPIRUN_OPTIONS_2="--bind-to cpu-list:ordered --cpu-list 112"

Now collect survey and tripcounts as follows,

mpirun -np ${NUM_MPI}-1 ${MPIRUN_OPTIONS_1} ./a.out : -np 1 ${MPIRUN_OPTIONS_2} ${ADVISOR_SURVEY} ./a.out 
wait
mpirun -np ${NUM_MPI}-1 ${MPIRUN_OPTIONS_1} ./a.out : -np 1 ${MPIRUN_OPTIONS_2} ${ADVISOR_TRIPCOUNTS}  ./a.out

Running Advisor on MPI+OpenMP parallel application

The following example employs 32 MPI ranks distributed uniformly over both the sockets with 2 OpenMP threads per MPI tasks on a Hawk node.

  module load mpt
  export MPI_SHEPHERD=1 
  export MPI_DSM_CPULIST=0-127/2:allhosts
  export OMP_NUM_THREADS=2
  export OMP_PROC_BIND=close
  export MPI_OPENMP_INTEROP=1

One can then run Advisor, same as described in the above section.

  mpirun -np $num_of_mpi_tasks-1 ./a.out : -np 1 advixe-cl -collect survey -project-dir results_advisor  ./a.out
  mpirun -np $num_of_mpi_tasks-1 ./a.out : -np 1 advixe-cl -collect tripcounts -flop -project-dir results_advisor  ./a.out

Additional analysis - memory access pattern and dependencies

While visualizing the results, Advisor might suggest performing additional analysis like memory access pattern and dependencies. One may collect the same, for example, as follows,

  mpirun -np $num_of_mpi_tasks-1 ./a.out : -np 1 advixe-cl -collect map -project-dir results_advisor  ./a.out
  mpirun -np $num_of_mpi_tasks-1 ./a.out : -np 1 advixe-cl -collect dependencies -project-dir results_advisor  ./a.out

Note that above analysis is possible only after collecting survey and tripcounts.

See also

External links