Hi; I'm going to reply to a bunch of the previous emails in this one. As several people, including Oxana and Karl and Michel and Donal, have pointed out, the fully general problem of describing job requirements and the resources available from various candidate execution systems is awfully hard. Indeed, it is arguably still a research problem and hence not something for defining standards about. The HPC profile work is about defining a standard for those simple situations that are well-understood and common enough that a standard would provide some tangible benefit in terms of allowing multiple implementations of things like job schedulers and job scheduling client libraries to interoperate with each other. From that point-of-view, I need to divide the world of JSDL and RSS solutions into at least two parts: a (very simple) part that we understand well enough to standardize now and a research part that requires further exploration and should NOT be standardized now. This is why I keep banging the drum about structuring things as base cases and extensions, so that I can define a very simple base case and a few simple extensions for purposes of the HPC profile standards work now (i.e. this summer) while allowing a graceful path to the future for all the really interesting and important work yet to come. So, what should be the base case and the few simple extensions that should get standardized by the HPC profile working group now (meaning this summer)? I would love to hear peoples' opinions and feedback -- especially in terms of deltas on the straw man proposal that I posted in my previous email(s). :-) With thanks in advance, Marvin. -----Original Message----- From: Donal K. Fellows [mailto:donal.k.fellows@manchester.ac.uk] Sent: Friday, June 09, 2006 7:53 AM To: Oxana Smirnova Cc: Alexander Papaspyrou; Michel Drescher; Karl Czajkowski; Marvin Theimer; JSDL Working Group; ogsa-bes-wg@ggf.org; Ed Lassettre; Ming Xu (WINDOWS) Subject: Re: [ogsa-bes-wg] Re: [jsdl-wg] Questions and potential changes to JSDL, as seen from HPC Profile point-of-view Oxana Smirnova wrote:
As a user, I strongly disagree. I *am* interested to have my jobs executed as soon as possible, for sure. This means I want them to be sent by a workload management system not just to any site that matches
job requirements, but to the best site - e.g., with the fastest processor, bigger memory, better bandwidth etc. I may also be interested in to send them to a cheapest site, or to a fastest site among the cheap ones. I may prefer to stay away from sites that use afs, and I may need to specify that I need inbound connectivity for a worker node. I perhaps only want to use sites in one specific country, for some licensing reasons. There are so many levels of optimization that users need, one
can write a book about it.
This is certainly a good indication that there cannot be a fixed set of resource selection descriptions. Any truly workable solution to this problem has to be really quite general. I've already designed (and implemented) such a scheme as it happens. ;-)
This is not a hypothetical case: I know many users that schedule jobs by hand to sites that in their experience are better, while the workload management system can not tell this from the available information or job description. This manual scheduling is orthogonal to the Grid idea, I dare say.
It's certainly reflecting the (too common) case where users and admins are in different warring camps. :-(
Instead, job specification should include very explicit attributes, including potentially a preferred sysadmin name :-) As it is pretty difficult to define a boundary between generic service levels description and specific informmation for fine-tuning, I would say it's better to stay with one "language" that covers all.
I should note that this is properly part of the domain of the OGSA-RSS working group, which I happen to chair. :-) I'm currently looking for a co-chair (more than one would be cool!) so that I'm not completely overworked. Volunteers will be able to partake in numerous benefits, such as invitations to speak at OGSA F2F meetings and the chance to go to the Chairs' Appreciation Night... Donal.