CRAY XC40 Disk Storage

HOME Directories

Users' HOME directories are located on a shared RAID system and are mounted via NFS on all login (frontend) and compute nodes. The path to the HOME directories is consistent across all nodes. The filesystem space on HOME is limited by a quota!

Warning: Due to the limited network performance, the HOME filesystem is not intended for large files and fast I/O! Do not read or write files from many nodes (>200) as this will cause trouble for all users. Use single read process + Bcast approach or MPI-IO instead.

SCRATCH directories

For large files and fast I/O Lustre based scratch directories are available which make use of the high speed network infrastructure (Aries). Scratch directories are available on all compute and login (frontend) nodes via the workspace mechanism.

Note: To get the best performance using MPI-IO it may be necessary to use tune the file distribution

Warning: Worspaces have some restrictions: First there is a maximum time limit for each workspace (30 days) after which they will be deleted automatically. Second they have a group quota of 20TB and 1 million files by default. If a project requests less than 20TB and 1 million files, its group quota is set accordingly lower.

The available storage capacity is about 5.7PB

Filesystem Policy

IMPORTANT! NO BACKUP!! There is NO backup done of any user data located on HWW Cluster systems. The only protection of your data is the redundant disk subsystem. This RAID system is able to handle a failure of one component. There is NO way to recover inadvertently removed data. Users have to backup critical data on their local site!

Long term storage

For data which should be available longer than the workspace time limit allowed and for very important data storage, please use the High Performance Storage System HPSS