HPC Job Delegation Best Practices - Draft Document

Folks, Philipp and I finally managed to compile the ideas that have been collected during the last months since OGF24 for the HPC Job Delegation Best Practices. Although the document is still preliminary (especially the use cases part), it can serve as a basis for further discussion on the topic. Major aspects since last OGF are the following: - Cleanup and restructuring: The last part of the document has been streamlined towards a "discussion of gaps" part and a "recommendation" part. While discussion of gaps is -- excluding GRAAP -- pretty complete, the whole recommendation part is still missing and needs to be done during the upcoming week, partly depending on further discussion in the GSA session on Wednesday. - OGSA-RSS incorporation: After some discussion with Oliver regarding the possibility of using dynamic WS-Agreement Factory sessions and resulting performance issues, we decided to propose a different approach to the offering of jobs to the delegatee scheduler. This protocol heavily borrows ideas from the OGSA-RSS Candidate Set Generator and Basic Execution Planning Service, but swaps the input/output parameters: while CSG and BEPS deliver resources that match a job specification, our requirement is to deliver job specifications matching certain resource constraints (i.e. those which can be fulfilled by the delegatee scheduler). We preliminarily named the corresponding thing "Delegation Candidate Service", pending concrete specification. - Mentioning of the (still BoFing) Activity Instance Schema Although in JSDL-WG everyone agrees that the Activity Instance Schema is an important thing, there is no "official" statement up to now if (and how) specification towards a recommendation will proceed. However, we felt that AI has reached a status where we can use it within GSA-RG in order to bridge the gap of missing status information before submission to a BES container. The problem domain is therefore discussed in detail in the document, pointing to AI already. I attached the current document draft to this message. Flaming is welcome, I have my fireproof underwear with me. If you decide to make modifications, please remember to enable change tracking in Word. Regards, Alexander ps. Apologies for duplicate posts. -- Alexander Papaspyrou alexander.papaspyrou@tu-dortmund.de
participants (1)
-
Alexander Papaspyrou