- Infos im HLRS Wiki sind nicht rechtsverbindlich und ohne Gewähr -
- Information contained in the HLRS Wiki is not legally binding and HLRS is not responsible for any damages that might result from its use -
NEC SX ACE Disk Storage: Difference between revisions
No edit summary |
m (Wiki syntax fixed) |
||
| (3 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
'''There is no backup of data.''' Most important first. | '''There is no backup of data.''' Most important first. | ||
Home filesystem has quotas active | Home filesystem has quotas active and is shared with other HWW systems. | ||
Temporary diskspace shared between frontend and SX-ACE nodes can only | |||
be requested through [[workspace mechanism]]. | be requested through [[workspace mechanism]]. | ||
| Line 13: | Line 14: | ||
of standard buffered I/O. | of standard buffered I/O. | ||
There is NO LOCAL disk space on SX nodes! | There is NO LOCAL disk space on SX-ACE nodes! | ||
To copy large files on SX use | To copy large files on SX use | ||
cp -b 32768 source target | |||
this speeds up the copy by a factor of 5 vs. the default (<tt>cp -b 1024</tt>), a buffer > 32768 will not work! | this speeds up the copy by a factor of 5 vs. the default (<tt>cp -b 1024</tt>), a buffer > 32768 will not work! | ||
| Line 27: | Line 28: | ||
An example for an unstriped file: | An example for an unstriped file: | ||
$scatefs_getfinfo testfile | $ scatefs_getfinfo testfile | ||
format : non stripe format | format : non stripe format | ||
iot count : 18 | iot count : 18 | ||
| Line 37: | Line 38: | ||
An example for a striped file: | An example for a striped file: | ||
$scatefs_getfinfo testfile | $ scatefs_getfinfo testfile | ||
format : stripe format | format : stripe format | ||
iot count : 18 | iot count : 18 | ||
| Line 51: | Line 52: | ||
Example how to set a stripesize of 4MB for all files in a directory | Example how to set a stripesize of 4MB for all files in a directory | ||
$scatefs_setdirattr -s 4m testdir | $ scatefs_setdirattr -s 4m testdir | ||
$touch testdir/testfile | $ touch testdir/testfile | ||
$scatefs_getfinfo testdir/testfile | $ scatefs_getfinfo testdir/testfile | ||
format : stripe format | format : stripe format | ||
iot count : 18 | iot count : 18 | ||
| Line 63: | Line 64: | ||
Example how to set a stripesize of 1MB for a file in a directory | Example how to set a stripesize of 1MB for a file in a directory | ||
$scatefs_premap -s 1M 0 testdir/testfile1m | $ scatefs_premap -s 1M 0 testdir/testfile1m | ||
$scatefs_getfinfo testdir/testfile1m | $ scatefs_getfinfo testdir/testfile1m | ||
format : stripe format | format : stripe format | ||
iot count : 18 | iot count : 18 | ||
Latest revision as of 16:05, 20 November 2024
There is no backup of data. Most important first.
Home filesystem has quotas active and is shared with other HWW systems. Temporary diskspace shared between frontend and SX-ACE nodes can only be requested through workspace mechanism.
For home and workspace filesystems, best performance is achieved with large I/O in blocks > 4MB. All filestytems can be accessed from SXes and frontends under same path.
Avoid formated I/O on SX, try to do large block, unformated I/O.
You can use the variables F_SETBUF and C_SETBUF to set buffer size of standard buffered I/O.
There is NO LOCAL disk space on SX-ACE nodes!
To copy large files on SX use
cp -b 32768 source target
this speeds up the copy by a factor of 5 vs. the default (cp -b 1024), a buffer > 32768 will not work!
Tuning IO on ScaTeFS
The ScaTeFS configuration consists of 2 servers and 18 storage targets on 4 RAID units. The filesystems knows two ways of storing files, striped and unstriped.
An example for an unstriped file:
$ scatefs_getfinfo testfile format : non stripe format iot count : 18 stripesize : 268435456 chunksize : 268435456 filesize : 10737418240
An example for a striped file:
$ scatefs_getfinfo testfile format : stripe format iot count : 18 stripesize : 4194304 chunksize : 1073741824 filesize : 10737418240
Default is to write unstriped files.
You can change this in two ways, by setting an attribute to the directory before creating files, or by setting the attributes for the file before writing to it.
Example how to set a stripesize of 4MB for all files in a directory
$ scatefs_setdirattr -s 4m testdir $ touch testdir/testfile $ scatefs_getfinfo testdir/testfile format : stripe format iot count : 18 stripesize : 4194304 chunksize : 1073741824 filesize : 0
Example how to set a stripesize of 1MB for a file in a directory
$ scatefs_premap -s 1M 0 testdir/testfile1m $ scatefs_getfinfo testdir/testfile1m format : stripe format iot count : 18 stripesize : 1048576 chunksize : 1073741824 filesize : 0
The 0 in front of filename denotes the file size, if the filesize to be written is known, some information can be preallocated to speedup later processing, if not known, give 0.
On SX-ACE, a stripesize of 4M with an IO size of 16MB gives something like 700MB/s write bandwidth into the filesystem. IO size should be multiple of stripsize, and 4M seems to be a good value for stripsize.