- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -

NEC SX ACE Disk Storage: Difference between revisions

From HLRS Platforms
Jump to navigationJump to search
No edit summary
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''There is no backup of data.''' Most important first.
'''There is no backup of data.''' Most important first.


Home filesystem has quotas active, temporary diskspace can  
Home filesystem has quotas active and is shared with other HWW systems.
Temporary diskspace shared between frontend and SX-ACE nodes can only
be requested through [[workspace mechanism]].
be requested through [[workspace mechanism]].


Line 13: Line 14:
of standard buffered I/O.
of standard buffered I/O.


There is NO LOCAL disk space on SX nodes!
There is NO LOCAL disk space on SX-ACE nodes!


To copy large files on SX use
To copy large files on SX use
{{Command|command = cp -b 32768 <source> <target>}}
cp -b 32768 <source> <target>
this speeds up the copy by a factor of 5 vs. the default (<tt>cp -b 1024</tt>), a buffer > 32768 will not work!
this speeds up the copy by a factor of 5 vs. the default (<tt>cp -b 1024</tt>), a buffer > 32768 will not work!


Line 27: Line 28:
An example for an unstriped file:
An example for an unstriped file:


  $scatefs_getfinfo testfile
  $ scatefs_getfinfo testfile
  format    : non stripe format
  format    : non stripe format
  iot count  :                18
  iot count  :                18
Line 37: Line 38:
An example for a striped file:
An example for a striped file:


  $scatefs_getfinfo testfile
  $ scatefs_getfinfo testfile
  format    :    stripe format
  format    :    stripe format
  iot count  :                18
  iot count  :                18
Line 51: Line 52:
Example how to set a stripesize of 4MB for all files in a directory
Example how to set a stripesize of 4MB for all files in a directory


  $scatefs_setdirattr -s 4m testdir
  $ scatefs_setdirattr -s 4m testdir
  $touch testdir/testfile
  $ touch testdir/testfile
  $scatefs_getfinfo testdir/testfile
  $ scatefs_getfinfo testdir/testfile
  format    :    stripe format
  format    :    stripe format
  iot count  :                18
  iot count  :                18
Line 63: Line 64:
Example how to set a stripesize of 1MB for a file in a directory
Example how to set a stripesize of 1MB for a file in a directory


  $scatefs_premap -s 1M 0 testdir/testfile1m
  $ scatefs_premap -s 1M 0 testdir/testfile1m
  $scatefs_getfinfo testdir/testfile1m
  $ scatefs_getfinfo testdir/testfile1m
  format    :    stripe format
  format    :    stripe format
  iot count  :                18
  iot count  :                18

Latest revision as of 17:24, 30 January 2015

There is no backup of data. Most important first.

Home filesystem has quotas active and is shared with other HWW systems. Temporary diskspace shared between frontend and SX-ACE nodes can only be requested through workspace mechanism.

For home and workspace filesystems, best performance is achieved with large I/O in blocks > 4MB. All filestytems can be accessed from SXes and frontends under same path.

Avoid formated I/O on SX, try to do large block, unformated I/O.

You can use the variables F_SETBUF and C_SETBUF to set buffer size of standard buffered I/O.

There is NO LOCAL disk space on SX-ACE nodes!

To copy large files on SX use

cp -b 32768 <source> <target>

this speeds up the copy by a factor of 5 vs. the default (cp -b 1024), a buffer > 32768 will not work!


Tuning IO on ScaTeFS

The ScaTeFS configuration consists of 2 servers and 18 storage targets on 4 RAID units. The filesystems knows two ways of storing files, striped and unstriped.

An example for an unstriped file:

$ scatefs_getfinfo testfile
format     : non stripe format
iot count  :                18
stripesize :         268435456
chunksize  :         268435456
filesize   :       10737418240


An example for a striped file:

$ scatefs_getfinfo testfile
format     :     stripe format
iot count  :                18
stripesize :           4194304
chunksize  :        1073741824
filesize   :       10737418240

Default is to write unstriped files.

You can change this in two ways, by setting an attribute to the directory before creating files, or by setting the attributes for the file before writing to it.

Example how to set a stripesize of 4MB for all files in a directory

$ scatefs_setdirattr -s 4m testdir
$ touch testdir/testfile
$ scatefs_getfinfo testdir/testfile
format     :     stripe format
iot count  :                18
stripesize :           4194304
chunksize  :        1073741824
filesize   :                 0


Example how to set a stripesize of 1MB for a file in a directory

$ scatefs_premap -s 1M 0 testdir/testfile1m
$ scatefs_getfinfo testdir/testfile1m
format     :     stripe format
iot count  :                18
stripesize :           1048576
chunksize  :        1073741824
filesize   :                 0

The 0 in front of filename denotes the file size, if the filesize to be written is known, some information can be preallocated to speedup later processing, if not known, give 0.

On SX-ACE, a stripesize of 4M with an IO size of 16MB gives something like 700MB/s write bandwidth into the filesystem. IO size should be multiple of stripsize, and 4M seems to be a good value for stripsize.