Dear All, I 'm trying to understand the meaning of TileSize and ProcessCount elements to describe a MPI + OpenMPI job. For example, in a job submission request for a batch scheduler (LoadLeveler, PBS and so on) I can specify : tasks per job=3 threads per task=4 processors per task=4 in order to get 1 thread per processor ( processors per task = threads per task ). I can set "nodes per job"=2 (with a a configuration of 8 cpus per node) or leave the scheduler sets this value. For what I understand in JSDL there are : [tileSize] tasks (processes) per group [ProcessCount] tasks (processes) per job [CPUCount] processors per job (with ResourceCount=1) [CPUCount + ResourceCount] processors per node + nodes per job from which I can get : tasks per job = [ProcessCount] = 3 threads per task = [ ? ] = 4 processors per task = [ CPUCount * ResourceCount / ProcessCount ] = 4 nodes per job = [ ResourceCount ] = 2 And so what is a "group of processes (tasks)" ? Is there a way to specify "threads per task (process)" ? Best Regards, Claudio Cacciari ========================================================= Claudio Cacciari CINECA - Via Magnanelli 6/3 40033 High Performance System Group Casalecchio di Reno - Bologna - ITALY e-mail: c.cacciari@cineca.it http://www.cineca.it Tel: 051/6171411 - Fax: 051/6132198 =========================================================
Claudio
And so what is a "group of processes (tasks)" ?
TileSize specifies the unit that should be used when partitioning the processes across the nodes. For example, TileSize=2 ProcessCount=8 means that the 8 processes should be partitioned across nodes in tiles (units) of 2; for a total of 4 tiles. If [ResourceCount=4] then each node would receive 1 tile (2 processes). If [ResourceCount=3] then one of the nodes would receive 2 tiles and the other two nodes 1 tile each. (It would not be possible to partition the processes as two nodes receiving 3 processes and one node 2 processes.)
Is there a way to specify "threads per task (process)" ?
Not directly. You can specify the number of cpus to be equal to or more than the threads that your application will use. To use your example: ProcessCount=3 CpuCount=8 ResourceCount=2 See also some examples that Michel send out last December http://www-unix.gridforum.org/mail_archive/jsdl-wg/2004/12/msg00006.html I am working (ever so slowly) on a revision of the ProcessTopology section for the specification. Hopefully it will be ready for review early next week. Thanks. Andreas claudio cacciari wrote:
Dear All,
I 'm trying to understand the meaning of TileSize and ProcessCount elements to describe a MPI + OpenMPI job. For example, in a job submission request for a batch scheduler (LoadLeveler, PBS and so on) I can specify :
tasks per job=3 threads per task=4 processors per task=4
in order to get 1 thread per processor ( processors per task = threads per task ). I can set "nodes per job"=2 (with a a configuration of 8 cpus per node) or leave the scheduler sets this value. For what I understand in JSDL there are :
[tileSize] tasks (processes) per group [ProcessCount] tasks (processes) per job [CPUCount] processors per job (with ResourceCount=1) [CPUCount + ResourceCount] processors per node + nodes per job
from which I can get :
tasks per job = [ProcessCount] = 3 threads per task = [ ? ] = 4 processors per task = [ CPUCount * ResourceCount / ProcessCount ] = 4 nodes per job = [ ResourceCount ] = 2
And so what is a "group of processes (tasks)" ? Is there a way to specify "threads per task (process)" ?
Best Regards, Claudio Cacciari
========================================================= Claudio Cacciari CINECA - Via Magnanelli 6/3 40033 High Performance System Group Casalecchio di Reno - Bologna - ITALY e-mail: c.cacciari@cineca.it http://www.cineca.it Tel: 051/6171411 - Fax: 051/6132198 =========================================================
-- Andreas Savva Fujitsu Laboratories Ltd
Hi all, I am trying to do something in the "process topology" space for Unicore and have been given a use case of mixed OpenMP and MPI applications. These seem to me to require specification of _both_ the number of Processes (distributed memory - MPI) and the number of Threads per process (shared memory - OpenMP). How do the JSDL constructs deal with this? Sven "Andreas Savva" <andreas.savva@jp.fujitsu.com> wrote:
Is there a way to specify "threads per task (process)" ?
Not directly. You can specify the number of cpus to be equal to or more than the threads that your application will use. To use your example:
-- Sven van den Berghe Fujitsu Laboratories of Europe +44 208 606 4651
Perhaps there are naming issues here, or there are too many terms defined here. JSDL is only concerned with specifying the resource requirements for the allocation request, not with how those resources end up being used (i.e. using threads or processes). So the TileSize indicates how many cpus need to be allocated as a unit within one OS image (i.e. a group of cpus), and the CPUCount would tell you how many cpus are needed in total. I don't believe you need any more. If the job chooses to use processes or threads is independent of the allocation. So given this example:
tasks per job=3 threads per task=4 processors per task=4
TileSize = 4 CPUCount = 12 This can translate to the PBS syntax "nodes=3:ppn=4" or the LSF syntax "-n 12 -R 'span[ptile=4]'", and I'm sure others as well. To the entity making the allocation, processes or threads doesn't matter. -- Chris
Hi Chris, Just to give you some background on why this is the way it is. At the last F2F after a lengthy discussion on topology and what it really means the group reached the conclusion that topology is part of the application definition rather than part of the hardware requirements. (That is, it is independent of the hardware resources to be allocated to the job and should be described separately and in terms that are not hardware specific.) So the topology tags describe the number of processes and their desired grouping/tiling and appear in the Application section rather than the Resource section. The Resource section (may) contains tags to describe the hardware resources required: CPUCount and ResourceCount. I have some doubts about the current definition (in particular I think we either overloaded the term 'Process' or we defined TileSize the wrong way). But I think it is easier to bash this out on a call or f2f (hand to hand? :-) at GGF13. Andreas Christopher Smith wrote:
Perhaps there are naming issues here, or there are too many terms defined here.
JSDL is only concerned with specifying the resource requirements for the allocation request, not with how those resources end up being used (i.e. using threads or processes). So the TileSize indicates how many cpus need to be allocated as a unit within one OS image (i.e. a group of cpus), and the CPUCount would tell you how many cpus are needed in total. I don't believe you need any more. If the job chooses to use processes or threads is independent of the allocation.
So given this example:
tasks per job=3 threads per task=4 processors per task=4
TileSize = 4 CPUCount = 12
This can translate to the PBS syntax "nodes=3:ppn=4" or the LSF syntax "-n 12 -R 'span[ptile=4]'", and I'm sure others as well.
To the entity making the allocation, processes or threads doesn't matter.
-- Chris
-- Andreas Savva Fujitsu Laboratories Ltd
Thanks for the clarification Andreas. I guess I need to do some reading up on some of the changes. :-) -- Chris On 1/3/05 18:41, "Andreas Savva" <andreas.savva@jp.fujitsu.com> wrote:
Hi Chris,
Just to give you some background on why this is the way it is.
At the last F2F after a lengthy discussion on topology and what it really means the group reached the conclusion that topology is part of the application definition rather than part of the hardware requirements. (That is, it is independent of the hardware resources to be allocated to the job and should be described separately and in terms that are not hardware specific.) So the topology tags describe the number of processes and their desired grouping/tiling and appear in the Application section rather than the Resource section. The Resource section (may) contains tags to describe the hardware resources required: CPUCount and ResourceCount.
I have some doubts about the current definition (in particular I think we either overloaded the term 'Process' or we defined TileSize the wrong way). But I think it is easier to bash this out on a call or f2f (hand to hand? :-) at GGF13.
Andreas
Christopher Smith wrote:
Perhaps there are naming issues here, or there are too many terms defined here.
JSDL is only concerned with specifying the resource requirements for the allocation request, not with how those resources end up being used (i.e. using threads or processes). So the TileSize indicates how many cpus need to be allocated as a unit within one OS image (i.e. a group of cpus), and the CPUCount would tell you how many cpus are needed in total. I don't believe you need any more. If the job chooses to use processes or threads is independent of the allocation.
So given this example:
tasks per job=3 threads per task=4 processors per task=4
TileSize = 4 CPUCount = 12
This can translate to the PBS syntax "nodes=3:ppn=4" or the LSF syntax "-n 12 -R 'span[ptile=4]'", and I'm sure others as well.
To the entity making the allocation, processes or threads doesn't matter.
-- Chris
participants (4)
-
Andreas Savva -
Christopher Smith -
claudio cacciari -
Sven van den Berghe