CRAY XE6 Hardware and Architecture

Hardware of Installation step 0

• 84 compute nodes / 1344 cores
  • dual socket G34
    • 2x Opteron(tm) Processor 6128 (Magny-Cours) mit 8 Cores @ 2 GHz
    • 512KB L2 Cache, 12MB L3 Cache
    • HyperTransport HT3, 6.4GT/s=102.4 GB/s
    • 2*8= 16 Cores/Node, Peak Performance: 2CPUs*8Cores/CPU*2GCycle/s*4FLOP/Cycle= 128 GFLOP/s
  • 32GB memory
• 12 service nodes
  • 6-Core AMD Opteron(tm) Processor 23 (D0)
    • 2.2GHz
    • 16GB memory
• High Speed Network CRAY Gemini
• LSI FibreChannel RAID System (SATA)
  • Parallel Filesystem
    • Lustre version 1.8.2
    • capacity 11 TB
    • 3 OST's, 1 MDS, Network Gemini
• System Management Workstation (SMW)

Architecture

• System Management Workstation (SMW)
  • system administrator's console for managing a Cray system like monitoring, installing/upgrading software, controls the hardware, starting and stopping the XE6 system.

• service nodes are classified in:
  • login nodes (xe601.hww.de) for users to access the system
  • boot nodes which provides the OS for all other nodes, licenses,...
  • network nodes which provides e.g. external network connections for the compute nodes
  • Cray Data Virtualization Service (DVS): is an I/O forwarding service that can parallelize the I/O transactions of an underlying POSIX-compliant file system.
  • sdb node for services like ALPS, torque, moab, cray management services,...
  • I/O nodes for e.g. lustre
  • MOM (torque) nodes for placing user jobs of the batch system in to execution

• compute nodes
  • are only available for user using the batch system and the Application Level Placement Scheduler (ALPS), see running applications.

Features

• Cray Linux Environment (CLE) 3.1 operating system
• Operating System is based on SUSE Linux Enterprise Server (SLES) 11
• Cray Gemini interconnection network
• Cluster Compatibility Mode (CCM) functionality enables cluster-based independent software vendor (ISV) applications to run without modification on Cray systems.
• Batch System: torque, moab
• many development tools available:
  • Compiler: Cray, PGI, GNU,
  • Debugging: DDT, ATP,...
  • Performance Analysis: CrayPat, Cray Apprentice, PAPI
  • Libraries: BLAS, LAPACK, FFTW, PETSc, MPT,....

Pictures