- ogsa-bes-wg - lists.ogf.org

Re: [ogsa-naming-wg] Re: [ogsa-wg] Abstract names in BES
by Christopher Smith 10 Nov '05

10 Nov '05

On 10/11/05 10:35, "Frank Siebenlist" <franks(a)mcs.anl.gov> wrote: > Mark McKeown wrote: >> Hi Donal, >> >> >>>> Regarding Chris's desire to be able to pass the LSF jobid >>>> back to the client somehow - it could be included in the EPR's >>>> Metdata, possibly RDF could be used to mark up the Metadata >>>> to indicate that it is a LSF jobid. In this way the Address >>>> IRI can be kept opaque. >>>> >>> That would seem to me to be needless disambiguation. Surely it is just >>> up to the service that mints the abstract name to understand it; there >>> is no inherent need for it to explain what that means to anyone else. >>> >> >> I am not sure I understand your comment - you don't seem to >> be disagreeing with me... >> >> Frank wants to use the EPR's wsa:Address IRI as an AbstractName, >> Chris wants to send a LSF jobid to the client and the W3C >> recommends that IRIs should be opaque. One way to include >> the LSF jobid in the EPR is to embed it into the wsa:Address >> IRI, this might help make it unique and the client could extract >> it from the IRI - however the W3C recommends that IRIs >> should be opaque. >> > > Well, I also believe that it's better to keep the IRIs opaque and was > suggesting to use the complete IRI itself as an alternative jobId. > > Whether that could work depends on the use cases we have to consider... > See this is exactly what I want to avoid. Our customers are used to seeing a regular jobID (now if this is generated by a higher level scheduler that understands checkpointing and migration and the fact that lower level jobids might change, that's fine). The use this jobID within command line tools and web portals to access information about their job. The web services interfaces are now an additional interface to these others. Why should I bring WS baggage along (in the form of EPRs and IRIs as identifiers) when I have one that our users understand already. I don't want to have to retool my CLI and other existing interfaces to now take some other structured identifier. The abstract name seems to me like a nice way of providing this identifier in the context of the WS interface. Sure, many of the alternative proposals would do the trick as well, but I had always been under the assumption that clients weren't supposed to interpret the EPR elements. -- Chris

1 0

Re: [ogsa-wg] Re: [ogsa-bes-wg] Abstract names in BES
by Christopher Smith 10 Nov '05

10 Nov '05

I haven't thought too much about it, but it would probably be something like the abstract name being the jobid (maybe plus the submit time for some time uniqueness if desired). I could then possibly use this to resolve to an EPR if needed. The jobid is used for both WS and non-WS interfaces, so that it's easier for clients (i.e. people) to remember them and pass them around. So, say someone gets an email from LSF about the job, which contains the LSF jobid, they can then use their portal (which uses the WS interface) to get more information. As for embedding in the EPR's address, I thought that it was not supposed to be "interpreted" by the people that they are handed to. :-) Of course, as I mentioned, this could also easily be done by embedding this information in the payload of some SOAP messages. -- Chris On 9/11/05 18:24, "Frank Siebenlist" <franks(a)mcs.anl.gov> wrote: > Hi Chris, > > Could you expand a little on the scenario and use case? > > Will the job id be the abstract name itself or embedded in it? > > What kind of resolution is needed and when? > (jobId=>EPR?) > > Could the jobId be embedded inside of the EPR's address? > > Thanks, Frank. > > > Christopher Smith wrote: >> For a mapping of BES to LSF, I would put something like a job id in >> the abstract name, so that it could be used outside the web services >> world (say with a CLI). Sure, it could be put in the payload of >> messages, or retrieved via some explicit call, but it seems reasonable >> to use the abstract name for this. >> >> -- Chris >> >> >> >> On 9/11/05 17:28, "Ian Foster" <foster(a)mcs.anl.gov> wrote: >> >> I found the call today very helpful. >> >> One thing that I wanted to get clarification on is the requirement >> that motivates a desire for "abstract names" in BES. >> >> I've heard one, which is that some entity may be handed two or >> more references (EPRs) to jobs, and want to know which of these >> are (not) unique. >> >> Are there others? >> >> Regards -- Ian. >> >> >>

1 0

Abstract names in BES
by Ian Foster 10 Nov '05

10 Nov '05

I found the call today very helpful. One thing that I wanted to get clarification on is the requirement that motivates a desire for "abstract names" in BES. I've heard one, which is that some entity may be handed two or more references (EPRs) to jobs, and want to know which of these are (not) unique. Are there others? Regards -- Ian.

2 1

Question on minutes
by Maguire_Tom＠emc.com 27 Oct '05

27 Oct '05

Sorry for the cross post.... * Mark to do WSRF Profile Rendering, Stephen to do WS-I Rendering. So does the 'WS-I Rendering' contemplate anything over and above WS-I BP 1.1? For instance is there a need for eventing? Or is this just really a 'Not OGSA WSRF Basic Profile' in which case isn't it really just the WSDL for the BES portTypes? Don't those just get composed with the WSDLs for the WSRF Basic Profile portTypes? Thanks, Tom Senior Technologist, CTO Office EMC²|SMARTS 44 South Broadway 7th Floor White Plains, NY 10601 Office: +1-914-508-3477 Mobile: +1-845-729-4806 Email: maguire_tom(a)emc.com <mailto:maguire_tom@emc.com> If you want to build a ship, don't drum up people to collect wood and don't assign them tasks and work, but rather teach them to long for the endless immensity of the sea. Antoine de Saint-Exupery

1 0

Re: [ogsa-bes-wg] Proposed changes to Spec.
by Ian Foster 26 Oct '05

26 Oct '05

Darren: I would like to argue against your proposed operations on the grounds of fragility and scalability. Your operation definitions require that a client keep track of the set of activities to be monitored and controlled. That's a big burden to place on the client. I suggest (as we have also suggested in earlier communications to BES-WG) that we should instead allow the user to specify a set of activities that are to be monitored or controlled in terms of their properties, not names. E.g., "activities that belong to me", "activities that involve executable Foo", "activities that have been running for over 2 hours." A natural way of modeling this is to represent the set of activities associated with your jobs as a WS-ServiceGroup Service Group, and then use queries on the service group to specify the activities to which an operation applies. Regards -- Ian. PS: As noted in previous emails and conversations, I also believe that the use of WS-Names rather than vanilla EPRs is inappropriate. I also believe that it is inconsistent with the OGSA policy that WS-Names cannot be mandated in OGSA-related specifications. At 08:52 AM 10/26/2005 -0600, Darren Pulsipher wrote: >I have spent some considerable time with the document. >I know far more than I planned, but it was good time spent. >I think we should discuss these items in our meeting on Thursday. >I have posted some changes to the document but would like to discuss >these items before I make these changes to them. > >There are some fundamental problems/questions that we need to resolve. > >1. How do we resume from where we left off. There is no Resume anymore >so we are putting the responsibility to the submitter or controller to >keep track of states. We can only use RequestActivityStateChanges to >suspend something and if I want to suspend 1000s of activities that >means I would need to first do a GetActivityState save the states and >then call RequestActivityStateChanges without any of the activities >changing state. That is not going to happen. So I propose we do the >following: >- Add the following interfaces: >SuspendActivites(WS-Name[] activityIdentifiers) >ResumeActivities(WS-Name[] activityIdentifiers) > >Or we add one interface >PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers) > >2. Another problem is how do we terminate an activity. The States that >are available are "Terminated" and "ShuttingDown". According to the >state net the activity is moved to the "ShuttingDown" and then to >"Terminated" so requesting RequestActivityStateChanges for activities >with "ShuttingDown" means it should move to the ShuttingDown state but >it will move the activities to the "Terminated" state. This can be very >confusing. So I suggest we add another interface. >- Add the following interface: >TerminateActivities(WS-Name[] activityIdentifiers) > >Or add another possible argument to the interface. >PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] >activityIdentifiers) > >3. Another question that is not clear in the spec is if an action >(staging in or staging out) moves into an exception state does that mean >that the activity moves into the exception state right away or does it >mean that it ever moves into the exception state. Does it mean that if >one of the actions moves into an exception state that all other actions >are halted and moved to the exception state and then the activity? >- I proposed that actions state have no impact on the activity state. If >an action has an exception or is terminated then it should not have any >impact on the activity what so every. > >4. Another not so important change but it might help clarify some >confusion and allow for extendibility later on would be to get rid of >the "ExecutionPending" and "ExecutionComplete" and make the "Running" >state a composite state like the "StagingIn" and "StagingOut" states. >This will give the ability to handle complex multiple application >execution definitions that the JSDL group is looking at for the next rev >of the Job Definition. This will allow for multiple Application actions. >This is not workflow. Nor should it be considered the start of workflow. >This just allows for extendibility easily. > > >Darren Pulsipher _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org

2 3

Meeting on Oct. 26, 2005
by Darren Pulsipher 26 Oct '05

26 Oct '05

Please dial in on Thursday October 27th, at 8:00 AM MDT. Dial-In Number(s): 866.765.2460 or 770.765.9436 Conference Code: 8018168214 Thanks Darren Pulsipher

1 0

RE: [ogsa-bes-wg] Proposed changes to Spec.
by Maguire_Tom＠emc.com 26 Oct '05

26 Oct '05

So this speaks to my comment during the OGSA-F2F; the position that all messages for BES require WS-Names will generate a fair amount of comments during public comment. Further, I believe there was some confusion about where and when OGSA-BES session took place at GGF15; I personally was not there. Nevertheless, the position that Andrew expressed is the 'rough consensus' of the OGSA-BES-WG. WRT, the WS-Names usage policy in OGSA. From the GGF F2F minutes: The group went through the exercise of doing a number of straw polls to identify the weight people put on various approaches. <<Straw polls>> - Agreed that we can choose MUST - 1. MUST EPR (majority yes; no negative votes) - passed - 2. MUST {EPR or URL} (yes:2; no:many) - rejected - 3. MUST {EPR or RNS} (yes:0; no:many) - rejected - 4. MUST EPR be WS-Name (yes:; no:) - rejected - 5. SHOULD EPR be WS-Name (yes:; no:) - ? - 6. RenewableReferences: MUST, SHOULD - rejected - 7. RenewableReferences: MAY - passed In summary: "MUST support EPRs, may support WS-Names or RRs or other things" It seems to depend on the situation whether to use something beyond an EPR. Need more exploratory work to identify which areas would benefit from using WS-Names or RenewableReference. - Proposal to also just say they are EPRs that may be decorated (and hence be WS-Names) In response to the proposal (last item) there was a note that went to OGSA-WG and OGSA-Naming-WG. Fundamentally, the proposal is still outstanding and needs to be settled or closed. Tom _____ From: owner-ogsa-bes-wg(a)ggf.org [mailto:owner-ogsa-bes-wg@ggf.org] On Behalf Of Ian Foster Sent: Wednesday, October 26, 2005 1:44 PM To: Andrew Grimshaw; 'Darren Pulsipher'; ogsa-bes-wg(a)ggf.org Subject: RE: [ogsa-bes-wg] Proposed changes to Spec. Andrew: We should ask the OGSA-WG to speak definitively on the first issue, as there seems to be confusion among participants as to what the policy means. Regarding OGSA-BES's use of WS-Name: I appreciate the information on the decision procedure. More generally, I guess this exchange reflects a common tension between the relative importance of the views of "insiders" vs. "outsiders" in GGF WGs. I know that quite a few people (myself included) started off wanting to engage as insiders in OGSA-BES, but after various unsatisfactory interactions (e.g., discussions like this one, where the response to a concern is to be told that it will be disregarded), ended up becoming outsiders. The OGSA-BES is of course free to work as it wants, but excluding the views of so many people also greatly reduces the likelihood that whatever specifications OGSA-BES produces will be widely adopted. I don't understand the discussion of events: I didn't speak to eventing in my email, but about the right way of specifying the activities to which an operation should apply. Regards, Ian. At 01:05 PM 10/26/2005 -0400, Andrew Grimshaw wrote: Ian, My understanding of the OGSA policy is that OGSA does not mandate WS-Names & instead it encourages them WHERE THEY ARE APPROPRIATE. That decision is up to individual working groups, and perhaps a design team working on a profile. The policy, in my mind, is NOT that WS-Names may not be mandated by OGSA specifications. In the case of OGSA-BES, at the last GGF in Boston the working group discussed at great length the pros and cons of WS-Names, opaque EPRs, and abstract names (i.e., uris with a few properties). The overwhelming majority preferred WS-Names (an explicit, comparable return value) (8 to 2). A further discussion was held as to whether the WS-Names should have both abstract names AND resolvers, and the vote was 4 to 1 for having a resolver, with many people indifferent. Thus, OGSA-BES will have* WS-Names as return values and input parameters. We also discussed eventingat some length. I have not yet typed in my notes, the three high order bits were: 1) The container is the right place to emit the events 2) Clients must explicitly subscribe to events (use filtering from the container, not specific registration& Im not sure what that means, I think we will need to discuss this in a call and remind me.) 3) Events are state transitions of the activities Andrew *Of course the document still has to go into public comment, get revised, and hopefully some day be approved by the GFSG. What is really in the document may change between now and then. _____ From: owner-ogsa-bes-wg(a)ggf.org [mailto:owner-ogsa-bes-wg@ggf.org <mailto:owner-ogsa-bes-wg@ggf.org> ] On Behalf Of Ian Foster Sent: Wednesday, October 26, 2005 11:40 AM To: Darren Pulsipher; ogsa-bes-wg(a)ggf.org Subject: Re: [ogsa-bes-wg] Proposed changes to Spec. Darren: I would like to argue against your proposed operations on the grounds of fragility and scalability. Your operation definitions require that a client keep track of the set of activities to be monitored and controlled. That's a big burden to place on the client. I suggest (as we have also suggested in earlier communications to BES-WG) that we should instead allow the user to specify a set of activities that are to be monitored or controlled in terms of their properties, not names. E.g., "activities that belong to me", "activities that involve executable Foo", "activities that have been running for over 2 hours." A natural way of modeling this is to represent the set of activities associated with your jobs as a WS-ServiceGroup Service Group, and then use queries on the service group to specify the activities to which an operation applies. Regards -- Ian. PS: As noted in previous emails and conversations, I also believe that the use of WS-Names rather than vanilla EPRs is inappropriate. I also believe that it is inconsistent with the OGSA policy that WS-Names cannot be mandated in OGSA-related specifications. At 08:52 AM 10/26/2005 -0600, Darren Pulsipher wrote: I have spent some considerable time with the document. I know far more than I planned, but it was good time spent. I think we should discuss these items in our meeting on Thursday. I have posted some changes to the document but would like to discuss these items before I make these changes to them. There are some fundamental problems/questions that we need to resolve. 1. How do we resume from where we left off. There is no Resume anymore so we are putting the responsibility to the submitter or controller to keep track of states. We can only use RequestActivityStateChanges to suspend something and if I want to suspend 1000s of activities that means I would need to first do a GetActivityState save the states and then call RequestActivityStateChanges without any of the activities changing state. That is not going to happen. So I propose we do the following: - Add the following interfaces: SuspendActivites(WS-Name[] activityIdentifiers) ResumeActivities(WS-Name[] activityIdentifiers) Or we add one interface PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers) 2. Another problem is how do we terminate an activity. The States that are available are "Terminated" and "ShuttingDown". According to the state net the activity is moved to the "ShuttingDown" and then to "Terminated" so requesting RequestActivityStateChanges for activities with "ShuttingDown" means it should move to the ShuttingDown state but it will move the activities to the "Terminated" state. This can be very confusing. So I suggest we add another interface. - Add the following interface: TerminateActivities(WS-Name[] activityIdentifiers) Or add another possible argument to the interface. PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] activityIdentifiers) 3. Another question that is not clear in the spec is if an action (staging in or staging out) moves into an exception state does that mean that the activity moves into the exception state right away or does it mean that it ever moves into the exception state. Does it mean that if one of the actions moves into an exception state that all other actions are halted and moved to the exception state and then the activity? - I proposed that actions state have no impact on the activity state. If an action has an exception or is terminated then it should not have any impact on the activity what so every. 4. Another not so important change but it might help clarify some confusion and allow for extendibility later on would be to get rid of the "ExecutionPending" and "ExecutionComplete" and make the "Running" state a composite state like the "StagingIn" and "StagingOut" states. This will give the ability to handle complex multiple application execution definitions that the JSDL group is looking at for the next rev of the Job Definition. This will allow for multiple Application actions. This is not workflow. Nor should it be considered the start of workflow. This just allows for extendibility easily. Darren Pulsipher _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster <http://www.mcs.anl.gov/~foster> Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/> _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster <http://www.mcs.anl.gov/~foster> Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/>

1 0

Thursday 10 am Eastern call
by Andrew Grimshaw 26 Oct '05

26 Oct '05

Per Darren Pulsipher, =20 Please dial in on Thursday October 27th, at 8:00 AM MST. =20 Dial-In Number(s): 866.765.2460 or 770.765.9436 Conference Code: 8018168214 Agenda: Discuss Darren's document updates.

1 0

Proposed changes to Spec.
by Darren Pulsipher 26 Oct '05

26 Oct '05

I have spent some considerable time with the document. I know far more than I planned, but it was good time spent. I think we should discuss these items in our meeting on Thursday. I have posted some changes to the document but would like to discuss these items before I make these changes to them. There are some fundamental problems/questions that we need to resolve. 1. How do we resume from where we left off. There is no Resume anymore so we are putting the responsibility to the submitter or controller to keep track of states. We can only use RequestActivityStateChanges to suspend something and if I want to suspend 1000s of activities that means I would need to first do a GetActivityState save the states and then call RequestActivityStateChanges without any of the activities changing state. That is not going to happen. So I propose we do the following: - Add the following interfaces: SuspendActivites(WS-Name[] activityIdentifiers) ResumeActivities(WS-Name[] activityIdentifiers) Or we add one interface PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers) 2. Another problem is how do we terminate an activity. The States that are available are "Terminated" and "ShuttingDown". According to the state net the activity is moved to the "ShuttingDown" and then to "Terminated" so requesting RequestActivityStateChanges for activities with "ShuttingDown" means it should move to the ShuttingDown state but it will move the activities to the "Terminated" state. This can be very confusing. So I suggest we add another interface. - Add the following interface: TerminateActivities(WS-Name[] activityIdentifiers) Or add another possible argument to the interface. PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] activityIdentifiers) 3. Another question that is not clear in the spec is if an action (staging in or staging out) moves into an exception state does that mean that the activity moves into the exception state right away or does it mean that it ever moves into the exception state. Does it mean that if one of the actions moves into an exception state that all other actions are halted and moved to the exception state and then the activity? - I proposed that actions state have no impact on the activity state. If an action has an exception or is terminated then it should not have any impact on the activity what so every. 4. Another not so important change but it might help clarify some confusion and allow for extendibility later on would be to get rid of the "ExecutionPending" and "ExecutionComplete" and make the "Running" state a composite state like the "StagingIn" and "StagingOut" states. This will give the ability to handle complex multiple application execution definitions that the JSDL group is looking at for the next rev of the Job Definition. This will allow for multiple Application actions. This is not workflow. Nor should it be considered the start of workflow. This just allows for extendibility easily. Darren Pulsipher

1 0

Call number is incorrect for Oct 13 call in
by Darren Pulsipher 13 Oct '05

13 Oct '05

Please use 866.756.2460 or 770.765.9436 Passcode: 8018168214 Darren

1 0