November 2006 - ogsa-bes-wg

Where to find functional BES WSDL?
by Aleksandr Konstantinov 30 Nov '06

30 Nov '06

Hello, I'm looking for BES WSDLs and schemas. I tried at GridForge but was able to find only those in ogsa-bes-draft-v26. Unfortunately they are unusable due to significant amount of errors. Anywhere else I could look at? Regards, A.K. --

2 1

[Fwd: RE: [Fwd: BES Comments]]
by Hiro Kishimoto 30 Nov '06

30 Nov '06

Forwarding Paul's email. -------- Original Message -------- Subject: RE: [OGSA-BES-WG] [Fwd: BES Comments] Date: Wed, 29 Nov 2006 14:31:40 -0700 From: Strong, Paul <pstrong(a)ebay.com> To: Christopher Smith <csmith(a)platform.com>, Karl Czajkowski <karlcz(a)univa.com>, Hiro Kishimoto <hiro.kishimoto(a)jp.fujitsu.com> CC: <ogsa-bes-wg(a)ggf.org> Thanks to both of you for helping me understand the reasoning behind the current description. I'd still suggest that clearly modeling the life-cycle of an activity, irrespective of one's perspective (i.e. the development of a specific interface), should provide a foundation for what control and monitoring capabilities you want the BES container to expose. Then one can attack the "what do I want to monitor?" question. That model should avoid the use of states as descriptions of past activity. Entangling states with transitions is fraught. This approach can typically lead to a great proliferation in the number of essentially equivalent states that result from different transitions, thus reducing overall clarity in the diagram. It also prevents the implementer from clearly identifying common elements, states etc. Modeling states and transitions separately does not mean that a client can't easily query the BES container to determine the state and last transition of an activity or a collection of activities. Anyway I will let this lie. You guys are more familiar with your problem space than I and as a group you have reached a consensus. All I will urge you to do is to provide clearer annotation and labeling of transitions so that your reasoning is clearly understood :o) Thanks for affording me the opportunity to comment and helping me gain a greater understanding of BES. Regards Paul -----Original Message----- From: Christopher Smith [mailto:csmith@platform.com] Sent: Wednesday, November 29, 2006 7:58 AM To: Karl Czajkowski; Hiro Kishimoto Cc: ogsa-bes-wg(a)ggf.org; Strong, Paul Subject: Re: [OGSA-BES-WG] [Fwd: BES Comments] Thanks for the clarification, Karl. I was thinking that I didn't express this very well. -- Chris On 28/11/06 19:37, "Karl Czajkowski" <karlcz(a)univa.com> wrote: > If I am not mistaken, I think there is a mismatch between Paul's > expected use of the state machine and the (perhaps underspecified) > assumptions of the BES authors. > > I think the BES authors are taking for granted an idiomatic "Grid > monitoring" viewpoint which emphasizes steady-state conditions as > descriptive summaries of past activity, while downplaying the > transitions or transient events. The BES implementation might have > actions associated with transitions, but the main monitoring view is > meant to be the conditions between events. > > In other words: we do not usually assume that the client viewed all > transitions, but that he wants to be able to determine the relevant > actionable state from a single view of the "current steady > state". There are a variety of reasons for this, including a more > self-healing distributed system (observer and observed can converge to > stable conditions) and more efficient aggregation and indexing > (observer can export a meaningful merged model of many resources' > conditions). > > I think this condition/event dichotomy is the gut reasoning behind > both wanting multiple container-level termination states and not > wanting self-transitions on a state. As a condition, if you started > in the state and ended in the state with no intervening state, then > you never left the state, as it is impossible to not be in some state! > > As a client, I will not get a stream of "self transitioned" events > with any descriptive information. What Chris suggested is to indicate > sub-conditions to indicate, in a more domain-specific way, how the > system really did change temporarily to another observable > steady-state condition, one that could still be interpreted as Pending > in the overall condition model. > > > karl -- Hiro Kishimoto

1 0

BES Faults
by Christopher Smith 29 Nov '06

29 Nov '06

Hi all, Just wanted to start some discussion on how to render faults for BES. The first thing to state is that I'm assuming the use of SOAP 1.1 for messaging, as this is the version referenced in the WS-I BP version 1.1. Please correct me if I'm wrong.... The second is that for now I'm ignoring WS-BaseFaults, as I think it would be very difficult to reach consensus on it's use in a short amount of time. The BES specification defines a few faults: (contained in SOAP Fault element in response) - NotAcceptingNewActivities - UnsupportedJSDLVersionFault - UnsupportedFeatureFault (contained in a response body) - ActivityDoesNotExistFault - CantApplyOperationToCurrentState <-- implied in doc as a todo We can render these in one of two ways: 1. We define new faultcode elements for these faults, and the detail can provide a higher level of information if needed. I'm not clear on how easy this is for most people from the tooling point of view. It would also require the WSDL to reflect the new fault codes (is this true?). 2. We define a set of fault elements that can either be used in the detail element, or on their own in some response elements. These fault elements wouldn't be "anyType", but would just be empty complexTypes. I prefer number 2 as this is quite straightforward from a tooling support point of view. Here are some examples of the various fault messages. Actual SOAP faults: <soap:Fault xmlns:soap="http://schemas.xmlsoap.org/soap/envelope"> <faultcode>soap:Server</faultcode> <faultstring>This BES is not accepting new activites now</faultstring> <detail> <bes:NotAcceptingNewActivities xmlns:bes="..."/> </detail> </soap:Fault> <soap:Fault xmlns:soap="http://schemas.xmlsoap.org/soap/envelope"> <faultcode>soap:Client</faultcode> <faultstring>Sorry ... I don't get it</faultstring> <detail> <bes:UnsupportedFeatureFault xmlns:bes="..."> <bes:UnsupportedElement name="jsdl-posix:POSIXApplication"/> </bes:UnsupportedFeatureFault> </detail> </soap:Fault> Faults in responses would be done using a type like: <xsd:complexType name="TerminateActivityResponseType"> <xsd:sequence> <xsd:element ref="bes-factory:ActivityIdentifier"/> <xsd:element name="Cancelled" type="xsd:boolean"/> <xsd:element ref="soap:Fault" minOccurs="0"/> <-- SOAP 1.1 fault </xsd:sequence> </xsd:complexType> So you'd get: <bes:TerminateActivityResponse xmlns:bes="..."> <bes:ActivityIdentifier>EPR</bes:ActivityIdentifier> <bes:Cancelled>false</bes:Cancelled> <soap:Fault> <faultcode>soap:Client</faultcode> <faultstring>That activity doesn't exist</faultstring> <detail><bes:ActivityDoesNotExistFault/></detail> </soap:Fault> </bes:TerminateActivityResponse> <bes:TerminateActivityResponse xmlns:bes="..."> <bes:ActivityIdentifier>EPR</bes:ActivityIdentifier> <bes:Cancelled>false</bes:Cancelled> <soap:Fault> <faultcode>soap:Client</faultcode> <faultstring>That activity doesn't exist</faultstring> <detail><bes:CantApplyOperationToCurrentState/></detail> </soap:Fault> </bes:TerminateActivityResponse> Comments? -- Chris

2 2

HPC Profile Call Minutes - 11/29
by Richard Ciapala 29 Nov '06

29 Nov '06

Attendees * Luigi * Andrew * Mark * Duane * Marty * Glenn * Chris * Joel * Vivian * Stephen * Peter * Rich Notes * We decided to work through the BES issues in Marty's email over the next couple of weeks with the goal of submitting BES to public comment before Christmas * Andrew will send Chris v27 of the BES specification which will include the changes from the last face-to-face. This is the version we will work from. * We will work from the Axis 1.4 BES WSDL from wiki * We discussed having FactoryResourceAttributesDocumentType encapsulate BasicFactoryResourceAttributesDocumentType instead of derive from it. No one had any issues with this. * We discussed having GetFactoryAttributesDocument take any type of filter to limit the amount of data returned. Rich will write up more detail on this and send it out to mailing list. If we go down this route, there should also be a way for the service to publish the filters it supports. * Chris start writing up solutions to the other BES issues mentioned and send them out to the mailing list * Discussed use of WS-BaseFaults. Need to research what the previous issues were. Thanks

1 0

[Fwd: BES Comments]
by Hiro Kishimoto 29 Nov '06

29 Nov '06

Hi all, I am forwarding this email on behalf of Paul Strong. -------- Original Message -------- Comments in line... Regards Paul -----Original Message----- From: Christopher Smith [mailto:csmith@platform.com] Sent: Tuesday, November 28, 2006 2:25 PM To: ogsa-bes-wg(a)ggf.org; Strong, Paul Cc: Hiro Kishimoto Subject: Re: [OGSA-BES-WG] [Fwd: [ogsa-wg] BES Comments] > > ii) Why does the state model not end in one terminated > state? Are there any real differences between the 3 enumerated end > states other than how the activity ended up in the state? > > iii) Also it seems somewhat counter intuitive that if > one is going to have three termination states, one should end up in the > one state which reflects both success and failure (Finished) and a > separate one for failures that prevent the activity from exiting in a > controlled fashion. It seems like the transitions to the terminated > state should be something like - > > a. Completed Successfully > > b. Failed > > c. Cancelled > The three states are intended to convey how the activity completes. The finished state indicates that the activity itself completed (without outside intervention), but makes no statement about whether the computation that the activity was engaged in was successful or not. This was envisioned to be extended through some extension sub-states. One of the reasons is that it's hard (generically) to determine whether an activity completes successfully or not. Using things like (say) exit code is traditional, but doesn't always help you distinguish between failure ("file doesn't exist") or a problem in the running of the activity ("my calculated result is wrong"). Failed means that some "outside event" (e.g. a host failure) caused the activity to finish prematurely, so that you can't say whether it completed or not. And Cancelled is used to indicate a manual intervention that stops the activity from completing. At one point we discussed having one terminal state, but we felt that three was quite useful. [PDS] I think this raises an important point with respect to defining the nature of an activity as a unit of work and defining its boundaries, pre-conditions and post-conditions in terms of the interaction between the client and the BES container. [PDS] If I were to define an activity then I would say that from the perspective of the BES container it is a unique instance of self contained unit of work. The client creates the activity, can monitor it and can store its result upon termination. That unit of work can be repeated but it becomes a new activity from the perspective of the BES container. I think you need something that says something like (though obviously not necessarily the same as) this. You obviously need to decide whether the container will keep a log of past activities or whether that is the responsibility of the client. [PDS] Three terminal states is obviously fine if there really is a substantive difference in the state of the activity at termination. Otherwise three state transitions to one termination state would typically allow you to capture the three (or more) modes of termination you desire in a simpler form (see attached GIF). And of course the terminated state of the activity can be queried for additional information. Termination implies nothing about whether the activity ceases to exist, merely that there are no state transitions from that state, only to it. Also a single terminated state makes it easy to understand the flow across the interface between the client and the BES container. [PDS] Anyway, whatever route you go requires greater clarity about the transitions and the states. > iv) Is there any notion of a retry, given that the > state machine captures failure? Or perhaps rescheduling? For example > one could imagine additional useful transitions - > > a. Pending to Pending - i.e. rescheduling > > b. Running to Pending - i.e. failed/retrying > > Having these in the base profile or basic model would enable greater > flexibility in implementing future profiles where one could request that > the BES container implement retries or allow rescheduling. Or are you > categorically stating that users will never be able to reschedule or > request automatic retries. Again if this is the case, it should > probably be explicitly stated. I would actually urge you to make the > base specification as broad as possible in terms of the acceptable state > changes at this level to avoid unnecessarily constraining yourself down > the line. I think this is especially important given what is stated in > 4.2. > I don't think pending to pending is necessary, as you can sub-state this to indicate any internal pending state transitions. [PDS] Perhaps you can, but perhaps there are no sub-states, merely a transition, in which case you have to capture it this way :o) Good point on running to pending ... maybe we should consider this one? How about failed to pending, or finished to pending? [PDS] I think finished to pending may be problematic unless you refine what you mean by finished. Again I guess we're back to what you define an activity to be. If finished is a terminal state then you should not be able to transition from it. The important point here is that you have to very clearly define what an activity is and what are and are not terminal states. > > vi) 4.2 - State Specialization really seems to be > indicating that this specification makes no assumptions with respect to > sub-states that may be incorporated within an implementation but that > such sub-states are acceptable as long as they do not introduce new > state transitions between the standard, specified states. It would be > very helpful to simply state this before all of the examples. > Yes ... this should be made very clear. It's definitely time for another BES conference call..... -- Chris -- Hiro Kishimoto

3 2

Reminder: HPC Profile conf call Wed Nov 29 8am Pacific time: post-SC2006 discussion
by Marty Humphrey 29 Nov '06

29 Nov '06

Folks, A reminder: we'll have a call tomorrow at 8am Pacific to discuss what we learned from the implementations of the HPC Profile for SC2006. We'll shortly send out telephone call info... -- Marty Quoting Marty Humphrey <humphrey(a)cs.virginia.edu>: > Folks, > > First, I want to THANK EVERYBODY for their hard work leading up to and > including the SC2006 interop demofest! I think we made tremendous progress > and I want to continue the momentum inside the BES, JSDL, and HPC Profile > working groups!!!! > > I propose that we have a conference call on Wed Nov 29 at 8am US Pacific > time to discuss what we each learned while we implemented the HPC Profile, > and where we as a group go from here. Please let me know if you can make > this call (a few people said that Fridays were not good, so we're trying a > different day -- but note: this is not necessarily proposed as a weekly > call, this is just a "special" call to discuss things while SC is still in > our minds). > > Rich Ciapala, Chris Smith and I tried to identify the unresolved issues that > we either directly encountered or someone raised on the mailing list. We > have certainly tried to capture everything but we probably missed a few > things. > > > Audience > > Topic > > > HPCP WG > > Finalize security additions to profile > > > BES WG > > Finalize faults for all operations > > > BES WG > > Limit amount of data that can be returned from GetAttributesDocument > > > HPCP WG > > Tools for which the schemas should work cleanly with > > > JSDL WG > > RangeValue_Type element clarifications > > * If a single min value and a single max value are to be specified, > should it be done with Boundary_Type's or a single Range_Type? > > * I assume Exact takes precedence over any specified Boundary_Type > or Range_Type's since Exact is a MUST and the others are optional. > > > BES WG > > Finalize Fault element's type for multiple response operations. We changed > this from soap:Fault to xsd:anyType for the demo. > > > HPCP WG > > Finalize list of dependent WS-I specs > > > BES WG > > Use of QNAMEs in GetAttributesDocument response > > > HPCP WG > > Data staging support > > > BES WG > > Clarify operations semantics (i.e. TerminateActivity) > > So, please let me know if you can make this call on Wed Nov 29. Also, let's > start email discussions on the appropriate mailing lists to attempt to > resolve some of these (or at least prioritize these and identify new ones) > before the call next Wednesday. > > Thanks again -- I think everything's coming together really well!!! > > -- Marty > > > > > > > > > > > > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.

2 1

[Fwd: [ogsa-wg] BES Comments]
by Hiro Kishimoto 28 Nov '06

28 Nov '06

Hi all, Paul Strong, you may remember him as GGF18 keynote speaker, asks me to forward his comments on BES spec to OGSA-BES-WG list. Thanks, -- Hiro Kishimoto -------- Original Message -------- Subject: [ogsa-wg] BES Comments Date: Tue, 21 Nov 2006 11:18:25 -0700 From: Strong, Paul <pstrong(a)ebay.com> To: <ogsa-bes-wg(a)ogf.org>, <ogsa-wg(a)ogf.org> All, I have been attempting to catch up on BES and was reading the v1.0 document. I have some questions and comments. Please note that I have no context from meetings or discussions about BES so please forgive my ignorance, I may be missing important context or details :o) i) Why are there 2 client focused interfaces (3 interfaces in total)? It is very natural for there to be 2 interfaces to a service/component, one focused on delivering client functionality and one for manageability. It is also natural to minimize the number of interfaces and the calls across them so as to aid maintainability. It would appear to me that the difference between the two BES client interfaces is that one allows activities to be created and for collections of activities to be monitored and managed. The other is focused on individual activities, but includes very similar functionality. There seems to be a lot of commonality. What are the reasons for them not being combined? Indeed as I read further I note that BES-Activity exposes no operations, so perhaps this interface is being deprecated and the document is in the process of being edited? ii) Why does the state model not end in one terminated state? Are there any real differences between the 3 enumerated end states other than how the activity ended up in the state? iii) Also it seems somewhat counter intuitive that if one is going to have three termination states, one should end up in the one state which reflects both success and failure (Finished) and a separate one for failures that prevent the activity from exiting in a controlled fashion. It seems like the transitions to the terminated state should be something like - a. Completed Successfully b. Failed c. Cancelled iv) Is there any notion of a retry, given that the state machine captures failure? Or perhaps rescheduling? For example one could imagine additional useful transitions - a. Pending to Pending - i.e. rescheduling b. Running to Pending - i.e. failed/retrying Having these in the base profile or basic model would enable greater flexibility in implementing future profiles where one could request that the BES container implement retries or allow rescheduling. Or are you categorically stating that users will never be able to reschedule or request automatic retries. Again if this is the case, it should probably be explicitly stated. I would actually urge you to make the base specification as broad as possible in terms of the acceptable state changes at this level to avoid unnecessarily constraining yourself down the line. I think this is especially important given what is stated in 4.2. v) Labeling of the state transitions would be most helpful, together with the adoption of standard UML statechart representation. Included is a state transition diagram which includes a few additional transitions which may, or may not be helpful/appropriate. :o) vi) 4.2 - State Specialization really seems to be indicating that this specification makes no assumptions with respect to sub-states that may be incorporated within an implementation but that such sub-states are acceptable as long as they do not introduce new state transitions between the standard, specified states. It would be very helpful to simply state this before all of the examples. vii) Do you expect to extend the BES model to activities that are not binaries running on operating systems? Regards Paul -- ogsa-wg mailing list ogsa-wg(a)ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg

2 1

Re: [OGSA-BES-WG] [ogsa-wg] BES Comments
by Donal K. Fellows 24 Nov '06

24 Nov '06

Strong, Paul wrote: > vii) Do you expect to extend the BES model to activities > that are not binaries running on operating systems? Such activity extensions do fit some of the ideas that have been floating around in the JSDL group for a few years now. In particular, activities might include things like database queries, webservice invokations, and data transfers. None of those three have had any serious effort into design of a real JSDL extension though, and there are some complexities in the understanding of the meaning of the concept of a resource in some of the cases. Given that, and the fact that BES is a JSDL enactment engine spec, I'd say that extending in the way you suggest is distinctly possible. The priority of what gets done when depends on who turns up to do the work, as always... :-) Donal.

1 0

Paul Strong's HPC usecase DAG
by Hiro Kishimoto 24 Nov '06

24 Nov '06

Hi all, I've upload Paul's DAG and State Transition UML diagram to the GridForege which was reviewed on Nov. 20th call. https://forge.gridforum.org/sf/go/doc14034 > 1) Powerpoint of one (of several) possible DAGs for > HPC. This is one of potentially several different renderings. > > 2) HPC State Transition In UML - UML representation of > the diagram in the BES v1.0 doc > > 3) HPC State Transition 2 In UML - UML pulling this > together with EGA Ref Model showing an implied state (Defined) and > implied transitions. Thanks, -- Hiro Kishimoto

1 0

HPC Profile conf call Wed Nov 29 8am Pacific time: post-SC2006 discussion
by Marty Humphrey 22 Nov '06

22 Nov '06

Folks, First, I want to THANK EVERYBODY for their hard work leading up to and including the SC2006 interop demofest! I think we made tremendous progress and I want to continue the momentum inside the BES, JSDL, and HPC Profile working groups!!!! I propose that we have a conference call on Wed Nov 29 at 8am US Pacific time to discuss what we each learned while we implemented the HPC Profile, and where we as a group go from here. Please let me know if you can make this call (a few people said that Fridays were not good, so we're trying a different day -- but note: this is not necessarily proposed as a weekly call, this is just a "special" call to discuss things while SC is still in our minds). Rich Ciapala, Chris Smith and I tried to identify the unresolved issues that we either directly encountered or someone raised on the mailing list. We have certainly tried to capture everything but we probably missed a few things. Audience Topic HPCP WG Finalize security additions to profile BES WG Finalize faults for all operations BES WG Limit amount of data that can be returned from GetAttributesDocument HPCP WG Tools for which the schemas should work cleanly with JSDL WG RangeValue_Type element clarifications . If a single min value and a single max value are to be specified, should it be done with Boundary_Type's or a single Range_Type? . I assume Exact takes precedence over any specified Boundary_Type or Range_Type's since Exact is a MUST and the others are optional. BES WG Finalize Fault element's type for multiple response operations. We changed this from soap:Fault to xsd:anyType for the demo. HPCP WG Finalize list of dependent WS-I specs BES WG Use of QNAMEs in GetAttributesDocument response HPCP WG Data staging support BES WG Clarify operations semantics (i.e. TerminateActivity) So, please let me know if you can make this call on Wed Nov 29. Also, let's start email discussions on the appropriate mailing lists to attempt to resolve some of these (or at least prioritize these and identify new ones) before the call next Wednesday. Thanks again -- I think everything's coming together really well!!! -- Marty

4 3