- ogsa-bes-wg - lists.ogf.org

Telecon - 22/6/06
by Steven Newhouse 16 Jun '06

16 Jun '06

All, It cannot have escaped your notice that there has been a lot of discussion over the last few weeks. I'd like to try and identify where we have reached consensus, try and resolve any outstanding issues, and clarify agenda items for the upcoming OGSA F2F by holding a telecon: Call details to be circulated: 22/6/06 4pm BST Anybody willing to work with me to identify these items please contact me direct! Regards, Steven -- ---------------------------------------------------------------- Dr Steven Newhouse Mob:+44(0)7920489420 Tel:+44(0)23 80598789 Director, Open Middleware Infrastructure Institute-UK (OMII-UK) c/o Suite 6005, Faraday Building (B21), Highfield Campus, Southampton University, Highfield, Southampton, SO17 1BJ, UK

1 0

Re: [ogsa-bes-wg] Re: [jsdl-wg] Questions and potential changes to JSDL, as seen from HPC Profile point-of-view
by Donal K. Fellows 12 Jun '06

12 Jun '06

Oxana Smirnova wrote: > As a user, I strongly disagree. I *am* interested to have my jobs > executed as soon as possible, for sure. This means I want them to be > sent by a workload management system not just to any site that matches > job requirements, but to the best site - e.g., with the fastest > processor, bigger memory, better bandwidth etc. I may also be interested > in to send them to a cheapest site, or to a fastest site among the cheap > ones. I may prefer to stay away from sites that use afs, and I may need > to specify that I need inbound connectivity for a worker node. I perhaps > only want to use sites in one specific country, for some licensing > reasons. There are so many levels of optimization that users need, one > can write a book about it. This is certainly a good indication that there cannot be a fixed set of resource selection descriptions. Any truly workable solution to this problem has to be really quite general. I've already designed (and implemented) such a scheme as it happens. ;-) > This is not a hypothetical case: I know many users that schedule jobs by > hand to sites that in their experience are better, while the workload > management system can not tell this from the available information or > job description. This manual scheduling is orthogonal to the Grid idea, > I dare say. It's certainly reflecting the (too common) case where users and admins are in different warring camps. :-( > Instead, job specification should include very explicit > attributes, including potentially a preferred sysadmin name :-) As it is > pretty difficult to define a boundary between generic service levels > description and specific informmation for fine-tuning, I would say it's > better to stay with one "language" that covers all. I should note that this is properly part of the domain of the OGSA-RSS working group, which I happen to chair. :-) I'm currently looking for a co-chair (more than one would be cool!) so that I'm not completely overworked. Volunteers will be able to partake in numerous benefits, such as invitations to speak at OGSA F2F meetings and the chance to go to the Chairs' Appreciation Night... Donal.

2 2

RE: [ogsa-bes-wg] Re: [jsdl-wg] Questions and potential changes to JSDL, as seen from HPC Profile point-of-view
by Marvin Theimer 10 Jun '06

10 Jun '06

Hi; If you structure things to explicitly support evolution and extension then I believe you get the desired effects: - Simple base case - Extensions that selectively and optionally add richness while building on the base - An explicit sanction to make early progress on the simple, well-understood parts of a problem without preventing future evolution to richer, more complete solutions. Marvin. -----Original Message----- From: Alexander Papaspyrou [mailto:alexander.papaspyrou@udo.edu] Sent: Friday, June 09, 2006 4:24 AM To: Michel Drescher Cc: Karl Czajkowski; Donal K. Fellows; Marvin Theimer; JSDL Working Group; ogsa-bes-wg(a)ggf.org; Ed Lassettre; Ming Xu (WINDOWS) Subject: Re: [ogsa-bes-wg] Re: [jsdl-wg] Questions and potential changes to JSDL, as seen from HPC Profile point-of-view Michel Drescher wrote >> The discovery ought to be "what types of job are acceptable" and not >> what resources are there. Or rather, the latter is part of some >> administrative interface which is misleading for job-submitting users >> and middleware. > > Yes! Yes! *waves the supporting flag* I also agree. From a user's POV, I pretty much don't care on what resource (within the constraints I specify) the job is run, just that it is at some point. > After reading this mail, we are probably best fitted if we provide *two* > resource models. Which may sound impractical, wasted resources or even > impossible, I think the idea may be worth exploring: I'm unsure whether one needs *two* different resource models here. > While system administrators are interested in making the best use out of > their machines (simple reason: return of investment), job submitters are > interested in having their jobs actually executed rather than optimised I think this observation is correct. The whole idea is that the user does not care about the specific resource, but only about a defined TOS (type of service, for the non-acronym impaired), and even then, only regarding the proper (and potentially early) execution of his job. > A solution to this dilemma may be to provide two "languages", one > fitting each group best: One language that job submitters use to specify > resources they need, which sacrifices accuracy for practicability. This > can be very simple, even name/value pairs. > > The other language, however, aims for maximum accuracy that I envision > to be a feature rich, (strongly?) typed representation of their resources. > > Obviously, these two languages need matching when a job is submitted. > The natural candidate for that is (Donal, forgive me for inaccuracy > here) "something" coming from the RSS WG. Maybe this can be covered by one "language". Then, one would need to specify two, let's say, versions of it: - the provider version, which should be strict to ensure accurate descriptions of their service's capabilities and - the consumer version, which is more relaxed in terms of allowing to leave out certain parts in order to be able to specify "loose" constraints for their service requirements IMHO, this would cover what Michel suggested, but make life easier for the involved WGs (not having to maintain two languages which are more or less identical) and -- later -- for the implementors (not having to provide two libraries for more or less the same functionality). Furthermore, the match maker people wouldn't have to suffer that much... Greetings, Alexander -- Dipl.-Inform. Alexander Papaspyrou | 44221 Dortmund, NRW (Germany) Robotics Research Institute | phone : +49(231)755-5058 Information Technology Section | fax : +49(231)755-3251 Dortmund University | web : http://www.irf.de/

1 0

Updates to document
by Andrew Grimshaw 28 May '06

28 May '06

All, I have done the updates to the document discussed in the last two phone calls, and in the EMS meeting in Japan. At a high level this includes modifications to adopt text/issues from the ESI document and the HPC profile working document. Note that this is NOT a final document. These changes revolve around: 1: Changing CreateActivityFromJSDL to a) take additional optional arguments, specifically notification arguments and an "idempotent" argument lifted from ESI. b) An extension to JSDL suggested in ESI to support libraries 2: Adding a resource information model section. Originally to be from 4.1 of the ESI document - but post discussion with Snelling, Stokes, et al a modified 4.1. 3: State model. This is one of the biggest jobs. At GGF in Japan the HPC profile group introduced a nice, simple, extensible state model that would allow supporting a variety of state machines while still allowing basic clients to understand what they were getting. After discussions with many people I have decided to adopt that simpler state model. CLEARLY THIS WILL NEED DISCUSSION BY INTERESTED PARTIES. I AM NOT TRYING TO PULL A FAST ONE. I have taken the text verbatim from the HPC profile paper. Note that I do not yet have permission from their lawyers - so we may need to yank this and retype the same info. I hope not. 4: Refer to managedjob "type" of EPR, just as we refer to "WS-Name" "type" of EPR. I have not even tried to modify the WSDL and renderings. Once we decide on the rest of the content we can do that. So don't read past page 20! I am not sure of the status of the weekly call this week. I will synch with Darren and Steven and send mail tomorrow. I'm also including some slides from the discussion of BES in the EMS session. A Andrew Grimshaw Professor of Computer Science University of Virginia 434-982-2204 grimshaw(a)cs.virginia.edu

3 4

Additional Input into BES w.r.t ESI document
by Steven Newhouse 24 May '06

24 May '06

All, From the OMII-Europe project (which is engaging with GGF in implementing grid standards) here are some comments from one of the teams that will be looking to implement the BES specification. Please respond to the following comments. The attached document has comments on the proposed ESI state model and information/resource model. Steven About the Job Factory Interface: * The list of the defined operations does not cover administrative operations like “rejectJobSubmissions(Policy)” and “allowJobSubmission (Policy)” useful for disabling/enabling new job submissions based on the policy defined by the CE administrator (e.g. if the CE has to be shutdown for maintenance; disable new submissions if the number of active jobs is > 3000; etc). Do you plan to provide it? * The current proposed JobFactory interface allows users to create new jobs AND (optionally) subscribe for notifications. We believe that the job management service should be better decoupled from the notification service, as they provide different functionalities. We suggest that Figure 1 be extended with a new box (“SubscriptionFactory”) which exposes an interface for creating, modifying and removing notification requests. In this way, notification management can be decoupled from job management, allowing a greater degree of flexibility. For example, it would be possible for users to subscribe to notifications after a job has been created (with the current proposed interface, this would not be possible). Moreover, it would allow users to submit to the notification service requests for “cumulative” subscriptions (i.e., in order to receive notifications related to all jobs submitted by the same user, or by members of the same Virtual Organization). * We propose to include a “JobAssess” operation on the JobFactory Interface. This operation provides the user with an estimate of the start running time, e.g. taking into account the current state of the Computing Element, the number of running/queued jobs and other parameters. * Related with the previous issue, it would be interesting to include an additional operation for estimating the Quality of Service (QoS) for the service instance. The exact meaning of QoS in general depends on user requirements (users may assume different weights for different parameters). * This is probably a “cosmetic” adjustment, but we believe that the name of the “Release” operation makes sense only for reactivating jobs from “Held” states; the transition from “Start Pending” and “Staging In” could probably be called something like “Activate”. * The proposed interface does not provide mechanisms for handling capabilities. With this we refer to the possibility for a user to authorize other user(s) to perform certain operations on his/her jobs. For example, a user may want to allow another user to monitor his/her jobs, or to interrupt and abort jobs and so on. Perhaps this functionality is not strictly related to job management, but is rather a security issue (which, according to the draft, is still to be discussed). We may keep it for future discussions. About the staging of files: * While it is clear that users might explicitly “push” files from their storage space to the Grid while a job is in “Start Pending” state, it is unclear how users might explicitly “pull” by hand resulting files from the Grid after a job competed, and before everything gets cleaned up. About the Job Interface: * The description of the states of Fig. 2 should be expanded with more details, including details on state transitions (basically, we suggest to put a complete description of the states in section 3.2.1). * Section 5.1: The meaning of the “Log” property on Table 3 is not clear: what does it mean? * From Table 3 we see a JobState property which represents the current state of the job. We think that it would be useful to provide the user with the history of all job status changes with the associated timestamp. We also support the need for exitCode and failureReason attributes (see Section 8.2, issue 9) to describe the job return code and job failure reason respectively. * It may be useful to provide an additional property (we may call it “CommandList”) representing the list of all commands issued for a given job. About the Application Interface: * The status of this interface is unclear: is this section going to be discussed? Are you going to consider the problem of user interaction with running jobs? -- Moreno Marzolla INFN Sezione di Padova, via Marzolo 8, 35100 PADOVA, Italy EMail: moreno.marzolla(a)pd.infn.it Phone: +39 049 8277047 WWW : http://www.pd.infn.it/~marzolla Fax : +39 049 8756233

4 6

Re: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs
by Christopher Smith 04 May '06

04 May '06

I agree ... we can discuss this next week. -- Chris On 03/5/06 10:09, "Thilo Kielmann" <kielmann(a)cs.vu.nl> wrote: > Chris and Andre, > > I think we really should resolve this issue face-to-face next week at GGF17. > That will save a lot of email typing time... > > Andre, do we have a slot in one of our sessions? > > > Thilo > > On Wed, May 03, 2006 at 06:08:46PM +0200, Andre Merzky wrote: >> X-Original-To: kielmann@localhost >> Delivered-To: kielmann(a)localhost.cs.vu.nl >> Delivered-To: grdfm-saga-rg-outgoing(a)mailbouncer.mcs.anl.gov >> X-Original-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> Delivered-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> Date: Wed, 3 May 2006 18:08:46 +0200 >> From: Andre Merzky <andre(a)merzky.net> >> To: Christopher Smith <csmith(a)platform.com> >> Cc: Andre Merzky <andre(a)merzky.net>, >> "ogsa-wg(a)ggf.org" <ogsa-wg(a)ggf.org>, ogsa-bes-wg(a)ggf.org, >> SAGA RG <saga-rg(a)ggf.org> >> Subject: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs >> >> Quoting [Christopher Smith] (May 03 2006): >>> >>> On 03/5/06 05:36, "Andre Merzky" <andre(a)merzky.net> wrote: >>> >>>> Right now its absent because only those states which can be >>>> actually reached by SAGA method calls are present. Now, as >>>> you said earlier, it can be assumed a flaw that SAGA has no >>>> means to put a job into 'Pending' state. >>>> >>> >>>> So, if there are some use cases or some agreement that the >>>> API should allow to move a job into 'Pending' state, the >>>> method/attribute will be added, and the state will be added. >>>> >>> The state that a job enters after a create_job operation is 'pending', so >>> the create_job operation is the method that puts a job in this state. >> >> Right, I understand that from the queuing point I think. >> However, from the application level it does not matter, as >> it cannot actively change that state. We have no notion of >> queue states at all. >> >> Well, let me put it differently: what do we gain by moving >> the state to the higher level? It complicates the state >> diagram (not much, that is not the point), but does not >> allow to perform any new state transition from application >> level. >> >> As said, I would compare this to Suspended state: its >> clearly an important state in many systems, but as long as >> the API does not expose means to suspend/resume a job, the >> state is meaningless, and merily informative (e.g. the job >> gut suspended by a 3rd party, or by the backend). That >> information is exposed however, through the job state >> details - same holds for hols (ha!) I think. >> >> >>> This state is required for queuing systems ... I can't even imagine how it >>> got dropped from the state diagram. I think you're trying to map the task >>> state model too closely. ;-) >> >> Nono, that has nothing to do with the task states - they >> could easily leave that state out. Well, its very >> convenient to have the same model here, thats for sure ;-) >> >> >>>> Similar reasoning holds for 'Suspend' - as long as we don't >>>> have suspend/resume methods, we would not like to expose a >>>> 'Suspended' state on the higher level state diagram. >>> >>> This I agree with. >> >> As said, it might well be that SAGA is broken in the respect >> that we doesn't allow job.hold (), or submit into hold. >> Well, it wasn't in the use-cases ... :-P >> >> Cheers, Andre. >> >> >>> -- Chris >> >> >> >> -- >> "So much time, so little to do..." -- Garfield >> > >

2 2

Extensible state model for jobs
by Christopher Smith 03 May '06

03 May '06

Hi all, As Marvin mentioned in his email on extensibility and composition of protocols, I'd like to present this short document describing, in more detail, a model for extending the state model of jobs from a "base profile". The document is based on some discussions that Marvin and I have been having about scheduler interoperability, and is very close to the model that has been proposed within the SAGA working group. Please review Marvin's email about extensibility to get a feel for the "context" of the model. It is my intention to show how the ESI and BES state models can be represented using this approach. I'll try to get that done before Tokyo, but might just need to present some slides at GGF17, assuming I'm given a few minutes to do so. This is very much a work in progress, so I'm looking forward to feedback.... A note for the SAGA group .... I'd still like to see "pending" in the base job state diagram ... don't know why it disappeared. ;-) -- Chris

2 3

Meeting Minutes for Wednesday April 26, 2006
by darren＠pulsipher.org 03 May '06

03 May '06

Here are the minutes from today? Next meeting is the same time same place.

2 1

Re: [ogsa-wg] More comments: HPC Use Cases -- Base Case and Common Cases
by Ian Foster 28 Apr '06

28 Apr '06

Susanne: I'd like to respond to your comments. I believe that the reference to "network partitions" refers to the fact that in a distributed environment, unlike a single machine environment, we cannot be sure that messages will be delivered: a network failure can result in any message being lost. Thus, a job submission may not receive a response, and in that case, we cannot know whether the job was submitted (i.e., the request got through, but the response was lost) or not (i.e., the request was lost). One convenient way of dealing with this problem is to allow users to associate a "unique job id" with a job request. A scheduler that receives a second or subsequent submission with the same jobid should simply return the response it provided to the first request received. It's true that a user can achieve a similar effect by searching for the submitted job in the scheduler queue. However, this approach is more complex to do, and also suffers from the problem that the job might have already completed and thus can't be found that way. In our recently circulated ESI specification, we proposed an optional "unique job id" field in JSDL as a way of addressing this requirement. This notion was discussed on a BES call, and people seemed sympathetic to the idea. Ian. At 12:20 PM 4/28/2006 -0400, Balle, Susanne wrote: >Marvin, > >Enclosed find the remaining of my comments: > >Page 5. (top paragraph) I think I know what you mean by "with the >ambiguity of distinguishing between scheduler crashes and network >partitions ". "scheduler crashes" is obvious. I am assuming that by >"network partitions" you are inferring that various sub-networks are >going to have different response time which will have an effect on the >time it takes to deliver a call-back message. > >Reading further along in the same paragraph I am now not sure I know >what you mean by "network partitions". > >Page 5. Section 3.3 >The topic of this section is clear (described in the first line of the >paragraph) but of the section is a little confusing. > >"possibility that a client cannot directly tell whether its job >submission request has been successful ..." --> Do we expect the client >to re-submit the job if the submission failed or do we expect users to >inspect that their job has in fact been submitted and resubmit if >needed? I am wondering if we assume the later if that wouldn't result in >users re-launching their jobs several time if they do not see their job >listed in some state when pulling the job scheduler for the state of >their job? > >I guess I do not understand why so much emphasis is put on the >"At-Most-Once" or "Exactly-Once". > >Can't the client poll the Job scheduler and ask the JS for a list of >jobs queued, running, terminated, failed, etc.? It might be useful for >the client to be able to submit jobs with a special keyword like >JOB_SUBMITTED_BY since that would reduce the list it gets back. It would >be nice if the value for the keyword was a unique identifier but doesn't >have to be. Most schedulers allows you to name or associate a group to >programs so that feature could be used as special keyword. > >Page 6. Section 3.4 >General question: Are you taking into account that user applications >will require different software? > >1. For example if my executable is compiled for Linux, Intel platform >then I would like to run it on a Linux,Intel system and not a Linux,AMD >system. > >2. Are you assuming that the program will be compiled on the fly on the >allocated system? or pre-compiled and then staged? > >I agree that staging the data is going to be an interesting topic. > >All this is probably out-of-band for the HPC JS Profile but should be >considered somewhere. I am sure it is I just don't know where. > >I like the section on virtual machines and think that they will be used >more and more in the future. > >Page 7. Extended Resource Features >The second approach (arbitrary resource types ...) is the only one that >make sense to me since that approach is extensible. I believe that Moab >is implementing this approach as well. > >Page 8. Extended Client/System Administrator Operations > >Are you assuming that System Administrators will be able to perform sys >admin operations on somebody else's system? I don't think that is right. > >You mention suspend-resume. Are you thinking of suspending a job running >across several clusters that are in different organizations? Or just >suspending a job on a single cluster/server? > >Again I am trying to figure out how this fit in with "One important >aspect, is that the individual clusters should remain under the control >of their local system administrator and/or of their local policies". > >I believe that suspend-resume is a JS operation or an operation to be >performed by the local sys admin, NOT by remote sys admins. > >If we are now talking about a meta-scheduler then yes it makes sense. In >the case of a meta-scheduler it might take over the individual JS and >schedule jobs base on its own policies, on its job reservation system, >etc. In this case I look at it as we have one deciding entity (the >meta-scheduler) and several "slaves". Moab and Maui are the only >meta-scheduler I an familiar with and they do take over the scheduling >decisions/node allocations/etc and just submit jobs to the local job >schedulers. > >This does of course assume that the local system administrators have >agreed on a schedule when their cluster is shared within this greater >infrastructure. This is a different approach than having jobs passed >onto their local scheduler and run on their systems. > >This just seems to be a different approach from the one that is taken in >this paper. >I might be wrong. If I am please educate me. > >Page 9. Section 3.10 > >Don't forget UPC (Unified Parallel C: http://upc.nersc.gov/) This >parallel programming paradigm is getting more and more interest from >several communities. >We'll need to provide support for UPC as well. > >Page 10. Section 3.13 >A meta-scheduler approach that make sense to me is to allow developers >to submit their job to their local cluster using their "favorite" >scheduler commands and then have the meta-scheduler load-balance the >work and forward the job to another system/cluster if needed. Moab from >cluster resources support this approach even if the clusters have >different JSs. They have a list of supported JS such as LSF, PBSpro, >SLURM, etc. and they can "translate" one JS's commands into another >within that supported set. > >Page 11. SLURM is missing. > >Let me know what you think, > >Regards > >Susanne > >--------------------------------------------------------------- >Susanne M. Balle, >Hewlett-Packard >High Performance Computing R&D Organization >110 Spit Brook Road >Nashua, NH 03062 > >Phone: 603-884-7732 >Fax: 603-884-0630 > >Susanne.Balle(a)hp.com _______________________________________________________________ Ian Foster, Director, Computation Institute Argonne National Laboratory & University of Chicago Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439 Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637 Tel: +1 630 252 4619. Web: www.ci.uchicago.edu. Globus Alliance: www.globus.org.

3 2

draft ESI 0.7 OGSA WSRF BP rendering
by Michel Drescher 28 Apr '06

28 Apr '06

Folks, I took a shot and hacked a WSDL that is OGSA WSRF Basic Profile compliant (I hope). It is fairly incomplete with respect to a) the operations, messages and faults taken from WSRF- ResourceProperties, WSRF-ResourceLifetime and WS-Notification specifications. b) the ManagedJob interface completely missing as this in a constant flux at the moment (and will probably merge eventually with the existing BES specification) The documents differ from the draft specification in that the resource properties that define the (POSIX) limits of a Job Factory retain the "Limit" suffix to avoid a naming conflict of the per ManagedJob limit "VirtualMemory" with the capability resource property "VirtualMemory". As I am either not allowed or not capable otherwise to add documents to GridForge, would please one of the project admins do so for me. Cheers, Michel

4 7