Storage usage policy

This page describe how to handle data within HLRS computing environment.

Please be aware, storage resources are optimized for a particular usage. For example work space filesystems are tuned for bandwidth of large parallel IO operations. This requires lots of components like disks, controllers, network connections, ... and make it expensive. To get the maximum benefit for all Users, following extra short guidelines should be read.

Important Notice! No Backup on any filesystem. Please copy important data into the archive.

Following filesystems are available:

• Home Directory - this storage type is availabel on all compute resources within HLRS network. Users should store e.g. profiles, script files for workflow tasks, sources for program development, ... But do not use this dircetory for number crunching (especially on large parallel) jobs!
• The workspace - here users jobs read / write large amount of data. IO has to be optimized to get considerable performance (e.g. use of optimized IO libraries), if unsure, please feel free to contact your project supervisor.
• TMP directory. On allmost all compute hosts, this is an "in-memory" filesystem, used for small temporary files. All data will be removed at the end of the job. Each node has its own TMP directory, not shared with other nodes, fast but small.

Usage guidelines for workspace filesystems

• The workspace filesystems are expensive resources, only data which is necessary for the ongoing work should be held within this directories. It is NOT a place for permanent (mid or long term) storage.
• If a project is suspended for a while, users have to free the disk space. Data could be transfered into the HPSS storage system
• Storge resources are overcommitted, this means the disk quota is not equal with a grant of storage space. (This is also true for compute resources)
• If the group's quota has a high usage (currently > 80 %) the performance of the filesystem degrades significantly for all users, To avoid this slowdown, NO Jobs of those groups are scheduled to run. All group members (with a registered E-Mail adress) will be notified by E-Mail
• Performance optimization is important, e.g. small size IO operations kills performance. Following bandwidth is possible for well optimzed applications.

File System	Host	Max. Performance
WS7	Hazel Hen	~75 GB/s
WS8	Hazel Hen	~75 GB/s
WS9	Hazel Hen	~200 GB/s
WS2	laki cluster	6 GB/s

• How to use the tools for managing work space directories is explained here .