Hia guys, I'd like to share some thoughts and discussion points from the SAGA group, in respect to the JSDL SPMD spec. Sorry for the long post... At the moment, SAGA has the following job description attributes to specify job multiplicity, which are derived from the JSDL core and JSDL SPMD extension: - SPMDVariation - MPI type etc. - TotalCPUCount - total number of CPUs required by the job - NumberOfProcesses - number of instances of the Executable that the consuming system MUST start - ProcessesPerHost - number of instances of the Executable that the consuming system MUST start per host. - ThreadsPerProcess - number of threads per process (i.e., per instance of the Executable There is not much discussion about SPMDVariation (anymore), but the others are appearently somewhat inconsistent: - processes can be started by the backend, threads can't, so why specify ThreadsPerProcess? How is it to be used by the backend? - hosts can have multiple CPUs - this is not reflected - CPUs can have multiple cores - this is not reflected - cores can have multiple hardware threads - this is not reflected, unless this is what is meant by 'ThreadsPerProcess'? In general, the attributes seem somewhat inconsistent, and not eay to use. Examples: limit the number of hosts to 2, no matter the number of CPUs NumberOfProcesses = 10 ProcessesPerHost = 5 // specify proc per resource limit the number of CPUs to 2, no matter the number of CPUs per host NumberOfProcesses = 10 TotalCPUCount = 2 // specify total resource num Also, there is no way to ensure that an application instance obtains exactly one CPU, unless one limits ProcessesPerHost to 1, which will waste significant resources on multi-CPU systems. I guess the problem really is that JSDL tries to stay out of the resource description business, and thats probably a wise decision. That is what Glue&Co are about to deal with. Nevertheless, the current specs are cumbersome from an *application* perspective. We are curently considering to change our attributes, to SPMDVariation NumberOfProcesses TotalNodeCount TotalSocketCount TotalCoreCount TotalHWThreadCount ProcessesPerNode ProcessesPerSocket (*) ProcessesPerCore ProcessesPerHWThread The 'ProcessesPerXYZ' attributes are here interpreted as upper limits: the backend MUST NOT start more processes per XYZ, but MAY start less. Those attributes are not fully translatable into the JSDL attributes, but map pretty well to other job descriptions so far. The typical use case for us would then boil down to SPMDVariation = MPI NumberOfProcesses = 32 ProcessesPerHWThread = 1 which seems to be what most users want. Other use cases we would be interested in, for example to start one additional IO process per node, would require more attributes and semantics than we are willing to introduce right now, like additional attribs: HWThreadsPerCore HWThreadsPerSocket HWThreadsPerNode CoresPerSocket CoresPerNode SocketsPerNode specification: SPMDVariation = MPI NumberOfProcesses = 32 + TotalNodeCount ProcessesPerNode = HWThreadsPerNode + 1 So, here is the biggie: Is JSDL at some point considering to revise the SPMD spec? If so, can we expect something along the lines above, or is that, aehem, 'out of scope'? If not, how would you propose to align the SAGA use cases with JSDL/Glue/..., so that we still end up with an implementable standards stack? We don't really want to invent our own schemas, as its most likely that it will not map to JSDL then - but we need to cater our use cases one way or the other... Best, Andre. (*) Socket basically stands for CPU, but is supposed to clarify we are not talking about cores. -- Nothing is ever easy.