On Dec 14, Soonwook Hwang modulated: ...
I am also wondering how these different job types will affect the design of EPS/CSG interface and protocol. The way to describe resource requirements for job execution may be different depending on the type of jobs. Is the resource element part of JSDL v1 spec sufficient to be used to describe requirements of jobs of different types? I suspect that depending on job types, the parameters used for ranking function/mechamism is likely to be different, and that we might end up having different EPS/CSG interfaces for each job type.
I think you could express an MPI job in JSDL v1 resource language, and have a co-allocation aware selection service return different possible mappings where some of the homogeneous resources come from different managers. E.g. a "simple" JSDL job is exploded into a set of concurrent JSDL jobs to be directed to different execution sites. However, I think you would quickly want a more rich resource language extension---beyond JSDL v1---where you could describe network connectivity requirements (possibly hierarchically) and heterogeneous resource requirements. For example, describing a job where you need a certain number of "large memory" nodes with a good interconnect, a certain number of "fast CPU" nodes with a separate good interconnect, and a less stringent WAN connection between the two node sets for some coupled numerical code. This is definitely beyond the scope of the JSDL v1 resource language, though a nice extended language could encapsulate multiple v1 resource clauses, one for each homogeneous node set... I think the big debates here will always return to which way such natural job/resource graphs will be normalized into a document tree. Should there be one job with complex resource requirements? Or some complex workflow with lots of simple (sub-)job documents? In the general case, there can be lots of cross-cutting relationships, e.g. correlating different executable specifications or data-staging requirements with different node types (or node identities), having a mixture of localized per-node, per-nodeset, and global resource allocation requirements, etc. Different normalized document trees will emphasize one job regime over another, making that one convenient to specify using nice, lexically scoped properties while either blocking other kinds of job or at least making them use less convenient cross-references and such to encode the natural graph. Of course, the debates are not just about syntactic style, but assumed document processing algorithms. Out of necessity, people are implementing restricted hueristics; out of convenience, I think they are also relying on document syntax to help enforce representation invariants for their implemented hueristics. I haven't seen anybody favoring a unified "big mess blob in/big mess blob out" interface where all the implementation-specific constraints must be checked internally while mapping to some more prosaic internal format. I have often wondered, and am not sure this would lead to better or worse interop than a bunch of more specialized interface standards... karl -- Karl Czajkowski karlcz@univa.com