November 2005 - ogsa-bes-wg

Fwd: RE: [ogsa-bes-wg] Proposed changes to Spec.
by Ian Foster 09 Dec '05

09 Dec '05

Hi, I wanted to follow up on my email below. There wasn't any response to the concern I raised concerning the proposed operations, and I think it is an important issue that should be discussed. I realize that the BES management might reply that I need to join the phone calls, but that isn't easy for me, and I think that we should be able to discuss issues on the email list also. Regards, Ian. >>Darren: >> >>I would like to argue against your proposed operations on the grounds of >>fragility and scalability. Your operation definitions require that a >>client keep track of the set of activities to be monitored and >>controlled. That's a big burden to place on the client. >> >>I suggest (as we have also suggested in earlier communications to BES-WG) >>that we should instead allow the user to specify a set of activities that >>are to be monitored or controlled in terms of their properties, not >>names. E.g., "activities that belong to me", "activities that involve >>executable Foo", "activities that have been running for over 2 hours." >> >>A natural way of modeling this is to represent the set of activities >>associated with your jobs as a WS-ServiceGroup Service Group, and then >>use queries on the service group to specify the activities to which an >>operation applies. >> >>Regards -- Ian. >> >>At 08:52 AM 10/26/2005 -0600, Darren Pulsipher wrote: >> >>I have spent some considerable time with the document. >>I know far more than I planned, but it was good time spent. >>I think we should discuss these items in our meeting on Thursday. >>I have posted some changes to the document but would like to discuss >>these items before I make these changes to them. >> >>There are some fundamental problems/questions that we need to resolve. >> >>1. How do we resume from where we left off. There is no Resume anymore >>so we are putting the responsibility to the submitter or controller to >>keep track of states. We can only use RequestActivityStateChanges to >>suspend something and if I want to suspend 1000s of activities that >>means I would need to first do a GetActivityState save the states and >>then call RequestActivityStateChanges without any of the activities >>changing state. That is not going to happen. So I propose we do the >>following: >>- Add the following interfaces: >>SuspendActivites(WS-Name[] activityIdentifiers) >>ResumeActivities(WS-Name[] activityIdentifiers) >> >>Or we add one interface >>PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers) >> >>2. Another problem is how do we terminate an activity. The States that >>are available are "Terminated" and "ShuttingDown". According to the >>state net the activity is moved to the "ShuttingDown" and then to >>"Terminated" so requesting RequestActivityStateChanges for activities >>with "ShuttingDown" means it should move to the ShuttingDown state but >>it will move the activities to the "Terminated" state. This can be very >>confusing. So I suggest we add another interface. >>- Add the following interface: >>TerminateActivities(WS-Name[] activityIdentifiers) >> >>Or add another possible argument to the interface. >>PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] >>activityIdentifiers) >> >>3. Another question that is not clear in the spec is if an action >>(staging in or staging out) moves into an exception state does that mean >>that the activity moves into the exception state right away or does it >>mean that it ever moves into the exception state. Does it mean that if >>one of the actions moves into an exception state that all other actions >>are halted and moved to the exception state and then the activity? >>- I proposed that actions state have no impact on the activity state. If >>an action has an exception or is terminated then it should not have any >>impact on the activity what so every. >> >>4. Another not so important change but it might help clarify some >>confusion and allow for extendibility later on would be to get rid of >>the "ExecutionPending" and "ExecutionComplete" and make the "Running" >>state a composite state like the "StagingIn" and "StagingOut" states. >>This will give the ability to handle complex multiple application >>execution definitions that the JSDL group is looking at for the next rev >>of the Job Definition. This will allow for multiple Application actions. >>This is not workflow. Nor should it be considered the start of workflow. >>This just allows for extendibility easily. >> >> >>Darren Pulsipher >> >>_______________________________________________________________ >>Ian Foster www.mcs.anl.gov/~foster >>Math & Computer Science Div. Dept of Computer Science >>Argonne National Laboratory The University of Chicago >>Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. >>Tel: 630 252 4619 Fax: 630 252 1997 >> Globus Alliance, www.globus.org > >_______________________________________________________________ >Ian Foster www.mcs.anl.gov/~foster >Math & Computer Science Div. Dept of Computer Science >Argonne National Laboratory The University of Chicago >Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. >Tel: 630 252 4619 Fax: 630 252 1997 > Globus Alliance, www.globus.org _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org

2 2

BES WSDL
by William Lee 30 Nov '05

30 Nov '05

Hi all, I've had a quick look at the BES WSDL. Here are my comments and modifications (attached). 1) Typos that cause missing referenced elements (see inline mod in the attached word doc.) 2) JSDL namespace has been updated according to the JSDL v1.0 spec. 3) The BES schema references the jsdl:JobDefinitionType complexType (it's actually been renamed to jsdl:JobDefinition_Type in the v1.0 spec), should we use the JSDL defined element "jsdl:JobDefinition" instead of creating our own. This makes life easier for user who's used to write JSDL document. <xsd:element ref="jsdl:JobDefinition" minOccurs="1" maxOccurs="1"/> 4) How are we going to cope with different renderings? The current WSDL contains a lot of WS-RF artifacts. Here's a cleanup WSDL with the WS-RF related artifacts commented out. http://www.doc.ic.ac.uk/~wwhl/bes/wsdl/basic-execution-service9.wsdl You can use the following tool to visualise it. http://www.mindreef.net/tide/scopeit/start.do William --- William Lee @ London e-Science Centre, Imperial College London -- --- Software Coordinator --- A: Room 380, Department of Computing, Imperial College London, Huxley Building, South Kensington campus, London SW7 2AZ, UK E: wwhl(a)doc.ic.ac.uk | william(a)imageunion.com W: www.lesc.ic.ac.uk | www.imageunion.com P: +44(0)20 7594 8251 F: +44(0)20 7581 8024 --- Projects ---------------------------- GridSAM: http://www.lesc.ic.ac.uk/gridsam Markets: http://www.lesc.ic.ac.uk/markets ICENI: http://www.lesc.ic.ac.uk/iceni -----------------------------------------

1 0

BES Meeting
by Darren Pulsipher 30 Nov '05

30 Nov '05

When: Thursday, Dec 1, 2005 7:00 AM (MST) Scheduled to Occur: Once Duration: 1:00 Darren Pulsipher has invited you to attend a Live Meeting. https://www.livemeeting.com/cc/xangoit/join?id=QG4M2D&role=attend Subject: BES Meeting Meeting URL: https://www.livemeeting.com/cc/xangoit/join Meeting ID: QG4M2D Role: Attendee Audio Conferencing (Toll): +1 (770) 765-9436 Audio Conferencing (Toll-free): +1 (866) 765-2460 Participant Code: 8018168214 FIRST TIME USERS: Install the Windows-based Meeting Console before your meeting: http://r.office.microsoft.com/r/rlidLiveMeeting?p1=7&p2=en_US&p3=LMInfo&p4=… This Live Meeting invitation is a personal invitation; it should not be forwarded. For assistance, please visit Live Meeting Help and Support: http://r.office.microsoft.com/r/rlidLiveMeeting?p1=7&p2=en_US&p3=LMInfo&p4=…

1 0

BES Meeting
by Darren Pulsipher 30 Nov '05

30 Nov '05

When: Thursday, Dec 1, 2005 7:00 AM (MST) Where: On Line *~*~*~*~*~*~*~*~*~* -+-----+-----+-----+-----+-----+-----+-----+-----+- Darren Pulsipher has invited you to attend a Live Meeting. https://www.livemeeting.com/cc/xangoit/join?id=QG4M2D&role=attend Subject: BES Meeting Meeting URL: https://www.livemeeting.com/cc/xangoit/join Meeting ID: QG4M2D Role: Attendee Audio Conferencing (Toll): +1 (770) 765-9436 Audio Conferencing (Toll-free): +1 (866) 765-2460 Participant Code: 8018168214 FIRST TIME USERS: Install the Windows-based Meeting Console before your meeting: http://r.office.microsoft.com/r/rlidLiveMeeting?p1=7&p2=en_US&p3=LMInfo&p4=… This Live Meeting invitation is a personal invitation; it should not be forwarded. For assistance, please visit Live Meeting Help and Support: http://r.office.microsoft.com/r/rlidLiveMeeting?p1=7&p2=en_US&p3=LMInfo&p4=…

1 0

RE: naming issues resolution process discussion (Re: [ogsa-wg] Proposed agenda for Nov. 16 call)
by Ian Foster 17 Nov '05

17 Nov '05

Andrew: I'm not ignoring you: either I hadn't heard you express the use cases below (I haven't been on all the calls) or I hadn't internalized you doing so. I appreciate you taking the time to write them down: this information is very helpful. Three comments on the use cases: 1) These use cases, like Chris Smith's, do not imply a need for names that are "globally unique in time and space." The only requirement is that names be unique with respect to the scope of the activities that the user wants to monitor or track. This might seem a minor point, but I think it's an important one: as Chris pointed out, a sensible key to use as a name in #1 could be, in some circumstances, e.g. the LSF jobid. That choice of key is prohibited by the current WS-Name definition. 2) In use case #1, I think you assume that a notification has the form "job with epr E has status S." Certainly, if that was the case, then you'd face the problem of determining which job that EPR referred to, something that you can't do without the ability to compare EPRs. But in WS-Notification, at least, when a client subscribes to notifications it can supply, as the callback address, an EPR to a unique WS-Resource. So the notification delivers the message "status S" to the subscription for job with epr E. Thus, there is no need to include a key in the EPR for the purpose of disambiguation. 3) Use case #2 makes sense to me. I could also do this without an AbstractName, by passing around an XML structure containing an application-specific key and the EPR--or, more likely, embedding the EPR in a larger application-specific structure that I have created for other purposes. But clearly an AbstractName could usefully be applied here. Ian. At 12:03 PM 11/16/2005 -0500, Andrew Grimshaw wrote: >Ian, > >I did give a scenario you just chose to ignore it. Here are two: > >1) As a client of BES I want to be able to keep track of the activities I >start. Suppose I have one hundred going. I want to build a data structure >that keys off of the activity. Id like the handle I get back from the BES >container (which is basically acting like a factory) to have a keyin it I >can use so that when I get notifications about the activity, or some other >services wants to talk about the activity I have some way to associate and >search for the activity information I am keeping. > >2) Using the WS-name (specifically the abstract name) to correlate >information flows about an activity. (This is the example I have used >several times.) > > > >The requirement in both cases is to be able to identify when two EPRs >refer to the same activity. > > > >Andrew > > > >---------- >From: Ian Foster [mailto:foster@mcs.anl.gov] >Sent: Wednesday, November 16, 2005 9:33 AM >To: Hiro Kishimoto; Andrew Grimshaw; Tom Maguire; Frank Siebenlist; Smith >Chris; Mark Morgan >Cc: Savva Andreas >Subject: Re: naming issues resolution process discussion (Re: [ogsa-wg] >Proposed agenda for Nov. 16 call) > > > >Hiro: > >Ahead of the call, I'd like to raise the question again of what >application requirements motivate the use of WS-Names in BES. I asked this >on the last call, and my question wasn't answered. I asked the question >again via email, and Chris replied with what turned out to be a >requirement for something different than WS-Names. I think it would be >really helpful to have a list of requirements as a basis for discussion. > >Ian.

1 0

Next telecon
by Andrew Grimshaw 17 Nov '05

17 Nov '05

All, Many people are at SC this week and cannot make the telecon. As next Thursday is Thanksgiving here in the US (one of the top two holidays) I propose that we skip next weeks telecon as well, and schedule our next telecom for Thursday, December 1, at the usual time. The primary issue on the table is the use of WS-Names in BES. There is a proposal under discussion in the OGSA-WG to prohibit mandatory use of WS-Names in ANY OGSA specification. That would impact BES in that the current specification requires the use of WS-Names as return types and parameters to many of the operations. This has generated a proposal (during discussions) that if WS-Names cannot be mandated, that BES revert to the original draft use of AbstractNames (i.e. strings) as the parameters and return values of the functions. Another issue that needs discussion is some changes to the document itself. As Mark Morgan has begun implementing a prototype of the draft specification several under-specified, or incompletely specified issues have come up. In separate email he discussed some of those. We will need to take actions (yea/nea) on some of his choices. Andrew Grimshaw Professor of Computer Science University of Virginia 434-982-2204 grimshaw(a)cs.virginia.edu

2 1

Re: [ogsa-wg] Abstract names in BES
by Christopher Smith 11 Nov '05

11 Nov '05

> > I'm concerned that the following two notions may be inconsistent: > > a) WS-Names should be globally unique (Andrew) > > b) The WS-Name can be used to pass a LSF jobid (Chris) > > Or is Chris proposing that LSF be changed to generate its jobids with > whatever scheme WS-Naming proposes to ensure global uniqueness? (If it > doesn't, then I don't think that Chris can guarantee that the names that > LSF generates and the names that someone else generates will not collide.) > For me, these names are all about context. Within an LSF environment (including an LSF multicluster environment), the LSF jobid is sufficient, and generating another globally unique identifier doesn't really add much value. If I'm in a context where I'm dealing with multiple systems that will be providing me EPRs for interacting with "jobs", then I'll need a higher level software layer to shield me from the lower level details, and here globally unique (well ... at least contextual globally unique) identifiers are required, and add more value. So let's circle back on this discussion for a moment. You asked for some use cases for the use of abstract names, and I provided one (perhaps you don't agree with my usage of abstract names, but that's this side discussion). This use case is one that we see all the time. Imagine an application "workbench" (e.g. something like Fluent or Sybyl), that exports a special BES service to run and manage domain specific jobs through its own "gateway" to LSF. Imagine then, if you will, a higher level job "viewer" interface that then produces another EPR for watching jobs (maybe so I can aggregate my job views from multiple workbenches). A job submitted through the workbench is viewable through the viewer, but they have different EPRs. The LSF jobid in this case is the abstract name that let's me know they are the same thing. And this jobid is as unique as it needs to be within this context. If I embed this identity information in the EPR, then how do the two gateways share this information? How can they if they are hosted at different SOAP endpoints? -- Chris

2 1

FW: BOUNCE ogsa-bes-wg@ggf.org: Non-member submission from [itf@mcs.anl.gov]
by Darren Pulsipher 10 Nov '05

10 Nov '05

-----Original Message----- From: owner-ogsa-bes-wg(a)ggf.org [mailto:owner-ogsa-bes-wg@ggf.org] Sent: Thursday, November 10, 2005 2:47 PM To: owner-ogsa-bes-wg(a)ggf.org Subject: BOUNCE ogsa-bes-wg(a)ggf.org: Non-member submission from [itf(a)mcs.anl.gov] >From owner-grdfm-ogsa-bes-wg(a)mailbouncer.mcs.anl.gov Thu Nov 10 15:47:13 2005 Return-Path: <owner-grdfm-ogsa-bes-wg(a)mailbouncer.mcs.anl.gov> X-Original-To: grdfm-ogsa-bes-wg(a)mailbouncer.mcs.anl.gov Delivered-To: grdfm-ogsa-bes-wg(a)mailbouncer.mcs.anl.gov Received: from localhost (localhost [127.0.0.1]) by mailbouncer.mcs.anl.gov (Postfix) with ESMTP id 83F5112C96; Thu, 10 Nov 2005 15:47:13 -0600 (CST) Received: from mcs.anl.gov (cliff.mcs.anl.gov [140.221.9.17]) by mailbouncer.mcs.anl.gov (Postfix) with ESMTP id 1592112C94; Thu, 10 Nov 2005 15:47:12 -0600 (CST) Received: from www-unix.mcs.anl.gov (gust.mcs.anl.gov [140.221.9.215]) by mcs.anl.gov (8.11.6/8.9.3) with ESMTP id jAALkqQ14270; Thu, 10 Nov 2005 15:46:52 -0600 Received: from 208.54.72.239 (SquirrelMail authenticated user itf) by www-unix.mcs.anl.gov with HTTP; Thu, 10 Nov 2005 15:46:52 -0600 (CST) Message-ID: <1472.208.54.72.239.1131659212.squirrel(a)www-unix.mcs.anl.gov> In-Reply-To: <43739301.7080507(a)mcs.anl.gov> References: <20051110143143.H67244-100000(a)nessie.mcc.ac.uk> <43739301.7080507(a)mcs.anl.gov> Date: Thu, 10 Nov 2005 15:46:52 -0600 (CST) Subject: Re: [ogsa-naming-wg] Re: [ogsa-wg] Abstract names in BES From: itf(a)mcs.anl.gov To: "Frank Siebenlist" <franks(a)mcs.anl.gov> Cc: "Mark McKeown" <zzalsmm3(a)nessie.mcc.ac.uk>, "Donal K. Fellows" <donal.k.fellows(a)manchester.ac.uk>, ogsa-wg(a)ggf.org, ogsa-naming-wg(a)ggf.org, ogsa-bes-wg(a)ggf.org User-Agent: SquirrelMail/1.4.4 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at mailbouncer.mcs.anl.gov X-Spam-Status: No, hits=-0.8 tagged_above=-1.0 required=5.0 tests=AWL, BAYES_00, NO_REAL_NAME X-Spam-Level: I'm concerned that the following two notions may be inconsistent: a) WS-Names should be globally unique (Andrew) b) The WS-Name can be used to pass a LSF jobid (Chris) Or is Chris proposing that LSF be changed to generate its jobids with whatever scheme WS-Naming proposes to ensure global uniqueness? (If it doesn't, then I don't think that Chris can guarantee that the names that LSF generates and the names that someone else generates will not collide.) Ian. > Mark McKeown wrote: >> Hi Donal, >> >> >>>> Regarding Chris's desire to be able to pass the LSF jobid >>>> back to the client somehow - it could be included in the EPR's >>>> Metdata, possibly RDF could be used to mark up the Metadata >>>> to indicate that it is a LSF jobid. In this way the Address >>>> IRI can be kept opaque. >>>> >>> That would seem to me to be needless disambiguation. Surely it is just >>> up to the service that mints the abstract name to understand it; there >>> is no inherent need for it to explain what that means to anyone else. >>> >> >> I am not sure I understand your comment - you don't seem to >> be disagreeing with me... >> >> Frank wants to use the EPR's wsa:Address IRI as an AbstractName, >> Chris wants to send a LSF jobid to the client and the W3C >> recommends that IRIs should be opaque. One way to include >> the LSF jobid in the EPR is to embed it into the wsa:Address >> IRI, this might help make it unique and the client could extract >> it from the IRI - however the W3C recommends that IRIs >> should be opaque. >> > > Well, I also believe that it's better to keep the IRIs opaque and was > suggesting to use the complete IRI itself as an alternative jobId. > > Whether that could work depends on the use cases we have to consider... > > -Frank. > > -- > Frank Siebenlist franks(a)mcs.anl.gov > The Globus Alliance - Argonne National Laboratory > >

1 0

Re: [ogsa-naming-wg] Re: [ogsa-wg] Abstract names in BES
by Christopher Smith 10 Nov '05

10 Nov '05

On 10/11/05 10:35, "Frank Siebenlist" <franks(a)mcs.anl.gov> wrote: > Mark McKeown wrote: >> Hi Donal, >> >> >>>> Regarding Chris's desire to be able to pass the LSF jobid >>>> back to the client somehow - it could be included in the EPR's >>>> Metdata, possibly RDF could be used to mark up the Metadata >>>> to indicate that it is a LSF jobid. In this way the Address >>>> IRI can be kept opaque. >>>> >>> That would seem to me to be needless disambiguation. Surely it is just >>> up to the service that mints the abstract name to understand it; there >>> is no inherent need for it to explain what that means to anyone else. >>> >> >> I am not sure I understand your comment - you don't seem to >> be disagreeing with me... >> >> Frank wants to use the EPR's wsa:Address IRI as an AbstractName, >> Chris wants to send a LSF jobid to the client and the W3C >> recommends that IRIs should be opaque. One way to include >> the LSF jobid in the EPR is to embed it into the wsa:Address >> IRI, this might help make it unique and the client could extract >> it from the IRI - however the W3C recommends that IRIs >> should be opaque. >> > > Well, I also believe that it's better to keep the IRIs opaque and was > suggesting to use the complete IRI itself as an alternative jobId. > > Whether that could work depends on the use cases we have to consider... > See this is exactly what I want to avoid. Our customers are used to seeing a regular jobID (now if this is generated by a higher level scheduler that understands checkpointing and migration and the fact that lower level jobids might change, that's fine). The use this jobID within command line tools and web portals to access information about their job. The web services interfaces are now an additional interface to these others. Why should I bring WS baggage along (in the form of EPRs and IRIs as identifiers) when I have one that our users understand already. I don't want to have to retool my CLI and other existing interfaces to now take some other structured identifier. The abstract name seems to me like a nice way of providing this identifier in the context of the WS interface. Sure, many of the alternative proposals would do the trick as well, but I had always been under the assumption that clients weren't supposed to interpret the EPR elements. -- Chris

1 0

Re: [ogsa-wg] Re: [ogsa-bes-wg] Abstract names in BES
by Christopher Smith 10 Nov '05

10 Nov '05

I haven't thought too much about it, but it would probably be something like the abstract name being the jobid (maybe plus the submit time for some time uniqueness if desired). I could then possibly use this to resolve to an EPR if needed. The jobid is used for both WS and non-WS interfaces, so that it's easier for clients (i.e. people) to remember them and pass them around. So, say someone gets an email from LSF about the job, which contains the LSF jobid, they can then use their portal (which uses the WS interface) to get more information. As for embedding in the EPR's address, I thought that it was not supposed to be "interpreted" by the people that they are handed to. :-) Of course, as I mentioned, this could also easily be done by embedding this information in the payload of some SOAP messages. -- Chris On 9/11/05 18:24, "Frank Siebenlist" <franks(a)mcs.anl.gov> wrote: > Hi Chris, > > Could you expand a little on the scenario and use case? > > Will the job id be the abstract name itself or embedded in it? > > What kind of resolution is needed and when? > (jobId=>EPR?) > > Could the jobId be embedded inside of the EPR's address? > > Thanks, Frank. > > > Christopher Smith wrote: >> For a mapping of BES to LSF, I would put something like a job id in >> the abstract name, so that it could be used outside the web services >> world (say with a CLI). Sure, it could be put in the payload of >> messages, or retrieved via some explicit call, but it seems reasonable >> to use the abstract name for this. >> >> -- Chris >> >> >> >> On 9/11/05 17:28, "Ian Foster" <foster(a)mcs.anl.gov> wrote: >> >> I found the call today very helpful. >> >> One thing that I wanted to get clarification on is the requirement >> that motivates a desire for "abstract names" in BES. >> >> I've heard one, which is that some entity may be handed two or >> more references (EPRs) to jobs, and want to know which of these >> are (not) unique. >> >> Are there others? >> >> Regards -- Ian. >> >> >>

1 0