- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -

Storage usage policy

From HLRS Platforms
Revision as of 10:24, 11 October 2016 by Hpctobei (talk | contribs)
Jump to navigationJump to search

This page describe how to handle data within HLRS computing environment.

Please be aware, storage ressources are are optimized for bandwidth of large parallel IO operations. This requires lots of components like disks, controllers, network connections, ... and make it expensive. To get the maximum benefit for all Users, following extra short guidlines should be read.

Important Notice! No Backup on any filesystem. Please copy important data into the archive.

Following filesystems are available:

  • Home Directory - this storage type is availabel on all compute ressources within HLRS network. Users should store e.g. profiles, script files for workflow tasks, sources for programm develpment, ... But do not use this dircetory for number crunching (esapecially on large parallel) jobs!
  • The workspace - here users jobs read / write large amount of data. IO has to be optimized to get considerable performance (e.g. use of optimized IO libraries), if unsure, please feel free to contact your project supervisor.
  • TMP directory. This is an "in-memory" filesystem, used for small temporary files. All data will be removed at the end of the job. Each node has its own TMP directory, not shared with other nodes, fast but small.


Usage guidlines

  • The workspace filesystems are expensive ressources, only data which is neccessary for the ongoing work should be held within this directories
  • If a project is suspended for a while, users have to free the disk space. Data could be transfered into the HPSS storage system
  • Storge ressources are overcommitted, this means the disk quota is not equal with a grant of storage space. (This is also true for compute ressources)