CRAY XC40 Batch System Layout and Limits

There are different types of queues configured on this System. These will be used to set proper priorities for different jobs, consider user permissions and resource reservations for different user groups. The configuration is laid out such that usually all you need to do is to request the number of processes you need (along with the number of processes per node) and the time (walltime) for your job. Users should always specify a realistic value for the walltime. Jobs with a shorter walltime get a higher priority and may be used for backfilling (users usually specify 24h, which is the max. time limit on HLRS systems. If your job usually runs in 4h 17min and you specified 5h, your job will be selected if nodes are available for this timeframe while the job-scheduler is collecting more nodes for a larger job).

If you don't specify any queue on your job submit command, then your job will be routed in to a default queue(s).

Note:

• In general the max. walltime for jobs is 24h.

Job Run Limitations

• The maximum time limit for a Job is 24 hours.
• User limits:
  • limited number of jobs of one user that can run at the same time
• User Group limits:
  • limited number of jobs of users in the same group that can run at the same time
• Batch Queue limits of all user jobs:
  • available nodes are 4488
    • single node job queues: max. 200 nodes in total
    • multi node job queues: max. 4488 nodes in total (max node count per job: 2560)
      • small jobs (requesting less than 48 nodes): 400 nodes in total
  • the default wall time is 10 minutes

Queues with extended wall time limits

are not available for general use. This Queue is available for Jobs, which can not run within the 24h timeframe. Access to this queue is only granted by passing an evaluation process. Following rules apply to this queue:

• Jobs may be killed for operation reasons at any time.
• Jobs will be accounted in any case. This is also true if the jab has to be terminated for operational reasons.
• Joblimit per Group
• Joblimit per user
• Low scheduling priority
• Max walltime 96h
• Jobs which are possible to run in a normal queue (by walltime or if the job can be splitted into subjobs have to be processed in a standard queue.

Again this queue is not for convenience but for running jobs which can not produce a result in other ways!