- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -

Difference between revisions of "NEC Cluster Hardware and Architecture (laki + laki2)"

From HLRS Platforms
(Hardware)
(Hardware)
Line 1: Line 1:
 
=== Hardware ===
 
=== Hardware ===
* 385 compute are of type NEC HPC-144 Rb-1 Server (see [http://www.nec.com/de/prod/solutions/lx-series/index.html  NEC Products])
+
* 404 compute are of type NEC HPC-144 Rb-1 Server (see [http://www.nec.com/de/prod/solutions/lx-series/index.html  NEC Products])
 
** dual CPU compute nodes: 2x [http://ark.intel.com/Product.aspx?id=37109&processor=X5560&spec-codes=SLBF4 Intel Xeon X5560] "Gainestown" ([http://www.intel.com/products/processor/xeon5000/ 5000 Sequence] [http://www.intel.com/products/processor/xeon5000/specifications.htm specifications])
 
** dual CPU compute nodes: 2x [http://ark.intel.com/Product.aspx?id=37109&processor=X5560&spec-codes=SLBF4 Intel Xeon X5560] "Gainestown" ([http://www.intel.com/products/processor/xeon5000/ 5000 Sequence] [http://www.intel.com/products/processor/xeon5000/specifications.htm specifications])
 
*** 4 cores, 8 threads
 
*** 4 cores, 8 threads
Line 10: Line 10:
 
** compute node RAM: triple-channel memory
 
** compute node RAM: triple-channel memory
 
*** standard: 12 GB RAM
 
*** standard: 12 GB RAM
*** 26 nodes upgraded to 24GB, 48GB or 144GB RAM
+
*** 20 nodes upgraded to 24GB, 48GB or 144GB RAM
 
**** 2 nodes of 144GB Memory nodes have additional a 6TB local scratch disk installed
 
**** 2 nodes of 144GB Memory nodes have additional a 6TB local scratch disk installed
**** 2 nodes of 144GB Memory nodes have additional a 2TB local scratch disk installed
+
**** 1 nodes of 144GB Memory nodes have additional a 2TB local scratch disk installed
** 20 compute nodes have additional [http://www.nvidia.com/object/tesla_computing_solutions.html Nvidia Tesla S1070 GPU's] installed.
+
** 16 compute nodes have additional [http://www.nvidia.com/object/tesla_computing_solutions.html Nvidia Tesla S1070 GPU's] installed.
  
 
*''' Pre- & Postprocessing node''' (smp node)
 
*''' Pre- & Postprocessing node''' (smp node)

Revision as of 12:45, 24 June 2014

Hardware

  • 404 compute are of type NEC HPC-144 Rb-1 Server (see NEC Products)
    • dual CPU compute nodes: 2x Intel Xeon X5560 "Gainestown" (5000 Sequence specifications)
      • 4 cores, 8 threads
      • 2.80 GHz (3.20 Ghz max. Turbo frequency)
      • 8MB L3 Cache
      • 1333 MHz Memory Interface, 6.4 GT/s QPI
      • TDP 95W, 45nm technology
      • "Nehalem" microarchitecture
    • compute node RAM: triple-channel memory
      • standard: 12 GB RAM
      • 20 nodes upgraded to 24GB, 48GB or 144GB RAM
        • 2 nodes of 144GB Memory nodes have additional a 6TB local scratch disk installed
        • 1 nodes of 144GB Memory nodes have additional a 2TB local scratch disk installed
    • 16 compute nodes have additional Nvidia Tesla S1070 GPU's installed.
  • Pre- & Postprocessing node (smp node)
    • 8x Intel Xeon X7542 6-core CPUs with 2.67GHz (8*6=48 Cores)
    • 1TB RAM
    • shared access
  • Visualisation node
    • 5 nodes each with 8 cores Intel W3540 and 24GB memory (4 for laki and 1 for laki2)
      • Nvidia Quadro FX5800
  • Node Upgrades (2012/2013)
    • 328 nodes Dual Intel 'Sandy Bridge' E5-2670 (204 for laki and 124 for laki2)
      • 2.6 Ghz, 8 Cores per processor, 16 Threads
      • 4 memory channels per processor, DDR3 1600Mhz memory
      • 36 nodes with 64GB RAM
      • 292 nodes with 32GB RAM
      • QDR Mellanox ConnectX-3 IB HCAs (40gbit)
  • Additional large memory nodes
    • 10 nodes Quad Socket AMD Opteron 6238
    • 2.6 Ghz, 12 cores per processor
    • 4 memory channels per processor, DDR3 1600Mhz memory
    • 256GB RAM
    • QDR Mellanox ConnectX-2 IB HCAs (40gbit)
    • 4 nodes have additional a 4TB local scratch disk



Architecture

The NEC Cluster platform (laki and laki2) consists of several frontend nodes for interactive access (for access details see Access) and several compute nodes of different types for execution of parallel programs.


Compute node types installed:

  • Intel Xeon 5560 (nehalem)
  • Intel Xeon E5-2670 (Sandy Bridge)
  • AMD Opteron 6238 (interlagos)
  • Nvidia Tesla S1070 (consisting of C1060 devices)
  • Large Memory nodes (144GB, 256GB)
  • Pre-Postprocessing node with very large memory (1TB)
  • Visualisation nodes with Nvidia Quadro FX5800
  • Different memory nodes (12GB, 24GB, 32GB, 48GB, 64GB)


Features

  • Operating System: ScientificLinux 6.4 (internal test on Windows HPC Server 2008)
  • Batchsystem: Torque/Moab
  • node-node interconnect: Infiniband + GigE
  • Global Disk 240 TB (lustre) for laki + 110TB (lustre) for laki2
  • Many Software Packages for Development