CRAY XE6 Batch System Layout and Limits

There are different type of queues configured on this System. These will be used to set proper priorities for different jobs, consider user permissions and resource reservations for different user groups. The configuration is laid out so that in default all you need to do is request the number of processes (mppwidth) you need (along with the number of processes per node) and the time (walltime) for your job. The scheduling system will then place the job in the appropriate queue.

At the moment there are only two different type of queues defined:

• mpp, the default job queue for Massive Parallel Processing jobs
• ccm, the job queue for the Cluster Compatibility Mode (CCM)

If you don't specify any queue on your qsub command, then your job will be routed in the mpp queues.

Using the qsub option "-q ccm", then your job will be prepared for the Cluster Compatibility Mode (CCM). The CCM is a software solution that provides the services needed to run most cluster-based independent software vendor (ISV) applications out-of-the-box with some configuration adjustments.

Note:

• CCM will only be available for some users, not for default!
• On workdays (Mo.- Fr.) between 8:00 - 18:00, only (short) jobs with walltime lower than 4h will be started. Other (long) jobs with walltime greater than 4h are only started on night or on weekends.
• The max. walltime for jobs is 24h.

Job Run Limitations

• User limits:
  • 2 Jobs that can run at the same time (hard limit is 4 for an low loaded XE6)
  • in total a user can only allocate 640 cores (hard limit is 1024 for an low loaded XE6)
• User Group limits:
  • 5 jobs of users in the same group that can run at the same time (hard limit 10)
• or CCM jobs the max. mppwidth is 256