Phoenix Introduction

Hardware and Architecture

The HLRS DGRID Cluster platform consists of one front node for interactive access (forntend.dgrid.hlrs.de) and serveral compute nodes for execution of parallel programs.

Two different compute node CPU architecture types are installed:

• IBM cell
• Intel Xeon 5150 (woodcrest)

Features

• Operating System: ScientificLinux 4.4 on Intel based nodes
• Operating System: Fedora Core5 on cell based nodes
• Batchsystem: Torque/Maui
• node-node interconnect: Infiniband + GigE
• Local Disk for scratch on each node
• Global Disk 100 TB
• GPFS
• OpenMPI
• Compiler: Intel, GCC, Java

Short overview of installed nodes

Function	Name	CPU	Memory	Disk	PBS properties	Interconnect
Compute Nodes	node01 - node28	Intel Xeon 5150 2.66 GHz	8 GB	160 GB	dgrid	GigE/Infiniband
Compute Nodes	cell01 - cell07	Cell Broadband Engine 3.2 GHz	1GB	40 GB	cell	GigE/Infiniband
Login Node	frontend	Intel Xeon 5150 2.66 GHz	16 GB	250 GB	-	GigE/Infiniband
I/O Server	data1 / data2	Intel Xeon 5130 2.0 GHz	16 GB	100 TB	-	GigE/Infiniband
Infrastructure (NTP,PBS,DNS,DHCP,FTP,GT4,...)	some virtual XEN based hosts)	Intel Xeon EM64T 3.4 GHz	8 GB	270 GB	-	GigE

Access

The only way to access frontend.dgrid.hww.de (frontend node of HLRS DGRID Cluster) from outside is through ssh

Usage

The frontend node

frontend.dgrid.hlrs.de

is intended as single point of access to the entire cluster. Here you can set your environment, move your data, edit and compile your programs and create batch scripts. Any interactive usage of the frontend node which causes a high cpu/memory load are NOT allowed (production runs).

The compute nodes for running parallel jobs are available only through the Batch system on the frontend node.

HOME Directories

---

All user HOME directories for every compute node of the cluster are located on the I/O Server data1.dgrid.hlrs.de. The compute nodes and login node (frontend) have the HOME directories mounted via NFS. On every node of the cluster the path to your HOME is the same. The filesystem space on HOME is limited by a quota of 700MB! Please note the Filesystem Policy!

---

Environment Settings

---

In order to use some software features like special MPI versions, or Compilers, you have to perform some environmental settings. To modify the default HLRS DGRID environmental settings on login, you can create a file $HOME/.profile which contains your own envirenmental settings. The login shell is a bash shell which reads $HOME/.profile during login!

Environment Settings using command module

The environmental setting using this methode will not be saved and will be lost for a new session. A new session (login, new job) will have the default HLRS DGRID environment. The Cluster system uses modules in the user environment to support multiple versions of software, such as compilers, and to create integrated software packages. As new versions of the supported software become available, they are added automatically to the programming environment, while earlier versions are retained to support legacy applications. By specifying the module to load, you can choose the default version of an application, or another version. Modules also porvide a simple mechanism for updating certain environment variables, such as PATH, MANPATH, and LD_LIBRARY_PATH. The following topics describe the fundamentals of using the modules environment.

• to invoke the module command, type:

  module option args

module help modulecommand

The help command will provide more detailed information on the specified module. Without argument modulecommand you will get online help for the module command.

module avail

The avail option displays all the modules that are available on the system. Where there is more than one version of a module, the default version is denoted by (default).

module list

The list option displays all the modules that are currently loaded into your user environment.

module add / module load modulename

The add option and the load option have the same function - to load the specified module into your user environment.

module rm / module unload modulename

The rm option and the unload option have the same function - to unload the specified module from your user environment. Before loading a module that replaces another version of the same package, you should always unload the module that is to be replaced.

module display modulename

The display option shows the changes that the specified module will make in your environment, for example, what will be added to the PATH and MANPATH environment variables.

module switch modulename/currentversionmodulename/newversion

The switch option replaces the currently loaded version of a module with a different version. When the new verion is loaded, the man page for the specified software will also be updated.

• using $HOME/.modulerc

This file can be used to load or to define your own environment during each login. An example looks like this:

#%Module1.0#

set version 1.0
module load use.own