- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -

Difference between revisions of "NEC Cluster Disk Storage (laki + laki2)"

From HLRS Platforms
(SCRATCH directories)
 
(11 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
=== HOME Directories ===
 
=== HOME Directories ===
  
All user HOME directories for every compute node of the cluster are located on the distributed NEC GFS, shared with the NEC SX8 and NEC SX9 Cluster. The compute nodes and login node (frontend) have the HOME directories mounted via NFS. On every node of the cluster the path to your HOME is the same. The filesystem space on HOME is limited by a quota! Due to the limited network performance, the HOME filesystem is not intended for fast I/O and for large files!
+
Users' HOME directories are located on a shared RAID system and are mounted via NFS on all login (frontend) and compute nodes.
 +
The path to the HOME directories is consistent across all nodes. The filesystem space on HOME is limited by a quota and will be shared with the resources hermit, hornet, laki, laki2!
  
 +
{{Warning|text=Due to the limited network performance, the HOME filesystem is not intended for large files and fast I/O! Do not read or write files from many nodes (>200) as this will cause trouble for all users. Use single read process + Bcast approach or [[MPI-IO]] instead.}}
  
 
=== SCRATCH directories ===
 
=== SCRATCH directories ===
Line 8: Line 10:
 
*lustre
 
*lustre
 
<ul>
 
<ul>
  It's a fast distributed cluster filesystem using the infiniband network infrastructure. This filesystem is available on alle nodes and on the frontend/loging nodes. The capacity is 43TByte, the bandwith is about 400MByte/sec.
+
  It's a fast distributed cluster filesystem using the infiniband network infrastructure. This filesystem is available on alle nodes and on the frontend/loging nodes.  
 +
* The capacity on laki is ~240TByte, the bandwith is about 4 GB/sec. The system consists of 2 MDS servers, 4 OSS servers and 24 OST storage targets, each of them one RAID6 lun, 8+PQ, 12 OSTs of 1TB and 12 OSTs of 2 TB disks.
 +
* The capacity on laki2 is ~110TByte, the bandwith is about 3 GB/sec. The system consists of 2 MDS servers, 2 OSS servers and 8 OST storage targets, each of them one RAID6 lun,  8+PQ, 2 TB disks.
 
</ul>
 
</ul>
<font color=red>You are responsible to obtain it from the system. To get access to this global scratch filesystems you have to use the </font> [[https://kb.hlrs.de/platforms/index.php/Workspace_mechanism'''workspace mechanism''']].
+
Scratch directories are available on all compute and login (frontend) nodes via the [[workspace mechanism]].
 +
{{Note|text=To get the best performance using [[MPI-IO]] it may be necessary to use tune the file distribution }}
 +
{{Warning|text=Worspaces have a restriction: There is a '''maximum time limit''' for each workspace (30 days) after which they will be '''deleted automatically'''.}}
  
 
=== Filesystem Policy ===
 
=== Filesystem Policy ===
 
IMPORTANT! NO BACKUP!! There is NO backup done of any user data located on HWW Cluster systems. The only protection of your data is the redundant disk subsystem. This RAID system is able to handle a failure of one component. There is NO way to recover inadvertently removed data. Users have to backup critical data on their local site!
 
IMPORTANT! NO BACKUP!! There is NO backup done of any user data located on HWW Cluster systems. The only protection of your data is the redundant disk subsystem. This RAID system is able to handle a failure of one component. There is NO way to recover inadvertently removed data. Users have to backup critical data on their local site!

Latest revision as of 14:30, 12 June 2013

HOME Directories

Users' HOME directories are located on a shared RAID system and are mounted via NFS on all login (frontend) and compute nodes. The path to the HOME directories is consistent across all nodes. The filesystem space on HOME is limited by a quota and will be shared with the resources hermit, hornet, laki, laki2!

Warning: Due to the limited network performance, the HOME filesystem is not intended for large files and fast I/O! Do not read or write files from many nodes (>200) as this will cause trouble for all users. Use single read process + Bcast approach or MPI-IO instead.


SCRATCH directories

For large files and fast I/O, please use

  • lustre
    It's a fast distributed cluster filesystem using the infiniband network infrastructure. This filesystem is available on alle nodes and on the frontend/loging nodes.
    • The capacity on laki is ~240TByte, the bandwith is about 4 GB/sec. The system consists of 2 MDS servers, 4 OSS servers and 24 OST storage targets, each of them one RAID6 lun, 8+PQ, 12 OSTs of 1TB and 12 OSTs of 2 TB disks.
    • The capacity on laki2 is ~110TByte, the bandwith is about 3 GB/sec. The system consists of 2 MDS servers, 2 OSS servers and 8 OST storage targets, each of them one RAID6 lun, 8+PQ, 2 TB disks.

Scratch directories are available on all compute and login (frontend) nodes via the workspace mechanism.

Note: To get the best performance using MPI-IO it may be necessary to use tune the file distribution
Warning: Worspaces have a restriction: There is a maximum time limit for each workspace (30 days) after which they will be deleted automatically.


Filesystem Policy

IMPORTANT! NO BACKUP!! There is NO backup done of any user data located on HWW Cluster systems. The only protection of your data is the redundant disk subsystem. This RAID system is able to handle a failure of one component. There is NO way to recover inadvertently removed data. Users have to backup critical data on their local site!