How are task resources mapped in Slurm directives?

[pablogoitia]

Hello! This issue is to ask you about how Autosubmit maps the job resources specified in jobs.conf to Slurm directives in the SlurmHeader module.

While exploring how I should map existing Slurm directives to Flux directives for its integration as a wrapper engine (#2720), I realized that there are some parameters I do not fully understand:

1. PROCESSORS directive: why are processors mapped as ntasks, rather than, for example, cpus-per-task?
2. TASKS directive: why are tasks mapped as ntasks-per-node, rather than just ntasks?
3. THREADS directive: why are they mapped as cpus-per-task? What is the purpose of the "threads"?

Thanks in advance for your answer!

CC @manuel-g-castro

[dbeltrankyl]

Not sure about the origin; we would need to ask the Author. Also, the first platform supported wasn't Slurm ( I think). So, the platform for what was Autosubmit was initially designed; these Slurm directives did not exist or had a different name.

Nowadays, for autosubmit, basically, we have to know this (at least as I understand it)

PROCCESORS: Tells the scheduler the number of cores available to the script in general, but it doesn't map anything.

I find the name straightforward, as ntasks would be in reality the number of cores allocated.

Example in marenostrum5:

Processors: 50 will use 1 node
Processors: 113 will use 2 nodes

But it is unbalanced without machine files, and maybe you don't want that.

You could have 112 cores on one node and 1 core on the other node

**THREADS: Tells the scheduler the number of processors per node so:

THREADS: 50

Will use one node of 50 cores

THREADS: 113

Will use one node of 113 cores. It will fail as the maximum processors allowed is 112 ( in mn5)

You need tasks, processors or the node directive ( if available in the scheduler) to determine the number of nodes

THREADS: 100
PROCESSORS: 200

will use 2 nodes and 100 cores in each one

THREADS: 100
TASKS: 2

Will use 2 nodes and 100 cores in each one

Tasks: Defines the quantity of work (task) to be done in each node

Tasks: 1 -> 1 nodes
Tasks: 50 -> 50 nodes

You need a threads directive to balance it, or processors to use more than one core per node

[pablogoitia]

Thanks a lot, @dbeltrankyl! That was really helpful