Karl Czajkowski wrote:

On Mar 21, Toshiyuki Nakata loaded a tape reading:

...

Thank you very much for the announcement. I'll be atttending but would have to leave early (7:40..) I can supplement what Wolfgang wrote in the minutes of GGF13, but most of it's already there so not much to add.

The other big issue would be the OGSA-BES and EMS Best Regards Toshi

My impression is that it is an uphill battle to convince the OGSA-BES group to consider seriously using WS-Agreement, while they all seem clear on the idea of using JSDL. I think this is due to several facts:

I talked with several people @GGF13. 1)If you loook at the two layered picture of Agreeement Services and the lower level, Real services (Such as Job submission etc), My understanding is BES has to deal with the Serive layer beneath the Agreement Factory layer. 2)On the other hand EMS has to look into what is the relationship between the Service Layer and the Agreement Layer. Eg. Is the Job defined first in the Service Layer and an Agreement Layer has to encompass this or, Is the Job defined between the Service Layer and the Agreement layer as a result of the CreateAgreement Operation. There *ARE* open questions as these which someone has to address...

4) BES is supposed to fit EMS, and it is not clear EMS really has an understanding where WS-Agreement could fit.

From my undersatnding, EMS had been waiting for WS-Ag to get fixed. I've also heard several semntimets from other projects...

5) Most groups are unclear on how we intend WS-Agreement to be used, not really seeing the "generalized GRAM/GARA" history and perhaps being confused by all the generic agreement, SLA, etc. terminology into thinking it has to be a heavyweight process involving human administrators.

I think these last issues are the biggest concern for GRAAP-WG besides, obviously, getting the technical issues ironed out. We need to figure out how to market all these nifty ideas or we run the risk of producing a specification that will gather dust on the shelf.

Actually as I said before, several WGs / projects are really waiting for output from GRAAP-WG. Best Reagards Toshi..

karl

-- Toshiyuki Nakata t-nakata@cw.jp.nec.com +81-44-431-7653 (NEC Internal 8-22-60210)

On Mar 21, 2005, at 5:49 AM, Karl Czajkowski wrote:

On Mar 21, Toshiyuki Nakata loaded a tape reading: ...

...

1)If you loook at the two layered picture of Agreeement Services and the lower level, Real services (Such as Job submission etc), My understanding is BES has to deal with the Serive layer beneath the Agreement Factory layer.

I am not 100% certain that I understood your comment after this one, because I want to read it as stating something close to what I think I have been saying, but as I said at the BES BoF, this "Agreement + job submission" view is a misunderstanding of WS-Agreement's service abstraction. "Job submission" is not the service layer we would envision beneath the Agreement management layer, because the submission pattern is exactly what WS-Agreement is meant to address.

Specifically, agreements about "generic execution jobs" or "specific kinds of job" are what we intended the basic agreement creation pattern to handle. Creation==submission. It is hard to express this without using the word agreement in a circular manner, but we took GRAM/GARA and said, "these are agreements about jobs or reservations" and tried to generalize it just a bit. These generalizations, while important, are not very intuitive for people to grasp; if they were, the GRAM and GARA systems would probably have started off more general. ;-)

Are you saying that agreement creation is equivalent to job submission? (More comments below)

WS-Agreement, through the GRAAP-WG charter, was scoped from the very beginning to address the "submission" part in its management model; our two explicit example use cases were job submission and advance reservation for jobs! That's the domain area brought in by the founding participants.

The GRAAP-WG charter does not address job submission. It addresses resource allocation. We used advance reservation for computational jobs as a use case. However, we make clear that the claiming of the resources, i.e. the job submission part of that use case, is not something that the GRAAP-WG would address.

This is why I frame this as a misunderstanding/public relations problem. There are not open questions for how WS-Agreement SHOULD handle jobs except that we haven't publicized the the answers well enough. :-)

I can understand that the decoupling of the resource allocation from the job submission is only one architectural viewpoint. In private discussions that I've had with non-GGF participants, they have wanted to view the resource allocation, job submission, and job execution as a single "business transaction". That's a completely valid viewpoint. However, it is different from the one described in the charter, where we view these as separate concerns. Perhaps this is where people's misconceptions come from. Perhaps you could clarify exactly where you view a GRAM-type protocol fitting in with WS-Agreement? Perhaps some sort of sequence diagram. Do you envisage the user have a separate interaction with GRAM to create/submit/launch the underlying computational job? Jon.

Hi Karl, Ok, I wasn't at the face-to-face meetings - I should have read the minutes. But then I was out of the loop for a while during those times. But where would someone new to the group pick up this information? It is likely that a new user would read the group charter, the use cases, and then the example in the specification. Lets look at each of these in turn. 1. If you look at the charter, it only mentions doing advance reservation of the resources - doesn't mention job submission. 2. The use cases talk in terms of SNAP, with the reservation that is being made by GRAAP being an RSLA. It also states that TSLAs (i.e. the running job) are created when something is submitted (something like) a batch queue. 3. And even if you look in Appendix 2 of the specification, you don't state whether or not the user needs to do something else to make the job "happen". (The only hint that this might automatically happen is that the user specifies many things here, far more than a system would need merely to reserve resources - but it's totally implicit.) So no wonder people are confused... Why don't you put together a simple informational document that shows the process of a user creating a job using WS-Agreement, the staging in of files, the execution, and staging out of results. Something that shows all the interactions between the user, the agreement provider, the resource manager, etc. The authors of the spec obviously have very clear intentions about this - they should be clearly stated somewhere! I think that such a document would be very valuable at this point. It could also be a valuable input to the BES group, to show you their intent - I didn't pick up on your viewpoint from your presentation in Seoul, and I think I was pretty awake at that point (you were lucky :-). The group might also want to consider updating the charter. Jon. On Mar 21, 2005, at 4:13 PM, Karl Czajkowski wrote:

Wow! I thought we'd discussed this ad nauseum in GRAAP face-to-faces, so I neglected to mention it.

I see GRAM as being obsolete once WS-Agreement is available to form agreements bearing JSDL terms. The only bits of GRAM that might need to remain are some extended operations or RPs on the Agreement itself.

karl

On Mar 21, Jon MacLaren loaded a tape reading: ...

...

The GRAAP-WG charter does not address job submission. It addresses resource allocation. We used advance reservation for computational jobs as a use case. However, we make clear that the claiming of the resources, i.e. the job submission part of that use case, is not something that the GRAAP-WG would address.

...

This is why I frame this as a misunderstanding/public relations problem. There are not open questions for how WS-Agreement SHOULD handle jobs except that we haven't publicized the the answers well enough. :-)

I can understand that the decoupling of the resource allocation from the job submission is only one architectural viewpoint. In private discussions that I've had with non-GGF participants, they have wanted to view the resource allocation, job submission, and job execution as a single "business transaction". That's a completely valid viewpoint. However, it is different from the one described in the charter, where we view these as separate concerns.

Perhaps this is where people's misconceptions come from.

Perhaps you could clarify exactly where you view a GRAM-type protocol fitting in with WS-Agreement? Perhaps some sort of sequence diagram. Do you envisage the user have a separate interaction with GRAM to create/submit/launch the underlying computational job?

Jon.

-- Karl Czajkowski karlcz@univa.com

On Mar 21, 2005, at 5:44 PM, Karl Czajkowski wrote:

...

I think that such a document would be very valuable at this point. It could also be a valuable input to the BES group, to show you their intent - I didn't pick up on your viewpoint from your presentation in Seoul, and I think I was pretty awake at that point (you were lucky :-).

Really?? I feel like I am in the Twilight Zone...

I thought I said explicitly that WS-Agreement is a generalization of the GRAM pattern for "creating a job" and could use JSDL instead of our GRAM job language. I then added that this had to be weighed against their milestones to decide whether to do it or take a more traditional approach, and gave due warning about the difficulty of rendering either type of interface on a short schedule, i.e. the devil is in the details, and the traditional approach is not necessarily the conservative approach since it can lead to repetition of mistakes.

Fair enough - I expect that I was wrong about my state of awakeness. (This leads me to wonder what WG sessions I *was* awake for...) I still like the idea of an information document, though. Cheers, Jon.

On Mar 22, Toshiyuki Nakata loaded a tape reading:

...

Specifically, agreements about "generic execution jobs" or "specific kinds of job" are what we intended the basic agreement creation pattern to handle. Creation==submission.

Am I correct in understanding that (Agreement ) Creation == (Job) submission?

Yes, or to put it more concretely: wsag:createAgreement(...jsdl:job...) should be a reasonable replacement for gram:createManagedJob(...gram:job...). If we (Globus) do this and there are job-related mechanisms we wish to keep, we might augment the Agreement portType with those operations and/or RPs. Given the way JSDL has gone, it will be up to the context of the JSDL document in the Agreement to indicate that the agreement is about executing the job. They have the word "submission" in the JSDL name, but I think that is more about scoping the set of job-related information that is captured in the JSDL specification. A lot of what/why the JSDL document exists is left to the messaging context that makes use of the specification. For example, one could easily describe two JSDL-bearing agreement idioms: 1) Advance reservation for job, indicating that the initiator wants a promise that such a job can be accepted later. Additional schedule/deadline terms may be introduced since JSDL leaves them out of scope. One can imagine omitting parts of the JSDL or having some other markup/annotations to indicate that some parts are variable and will be fixed later in the "submission" agreement. 2) Job execution, indicating that the initiator wants a promise that such a job should be executed. One can imagine optionally referencing a reservation Agreement in this Agreement, either through the JSDL extensibility or through some other WS-Agreement extensibility. I would be interested in others' opinions on whether an extra bit of wrapping XML is needed around the JSDL in the service description to distinguish these kinds of agreement, versus some other guarantee term that distinguishes "execute job J" from "be able to execute job J". This is the whole subtle RSLA/TSLA/BSLA distinction from SNAP, and I believe it is largely an aesthetic issue to perfer it rendered one way or another... however, if we do not provide guidance on expected use, I expect all the early adopters of WS-Agreement will do wildly different things because there are too many combinatorial avenues for composing terms and concepts here. karl -- Karl Czajkowski karlcz@univa.com

On Mar 22, Toshiyuki Nakata loaded a tape reading: ...

Hmmm. in business oriented jobs we would have a life-cycle of

Job reservation Deployment of the application (including data-bases) Starting the Application Stopping the Application Undeploying the Application

And had assumed that the ageement provider which would understand this kind of job would be able to understand this without having it explicitly stated..

(Also, except for Jobreservation, Deploying/Starting/Stopping/Undeploying would be interfaces of the service provider and not the Agreement Provider..

(I am afraid I am getting off the WS-Agreement topic and so would stop here..)

Best regards Toshi

The job life-cycle you describe is more along the lines of "job submission or service deployment with guarantees" than "advance reservation". All we need to do is add some performance guarantee terms together with the JSDL and the job host can handle all the planning in one step. This is an interesting area where job submission (or generically, service deployment) can be extended with guarantee terms without fundamentally changing the signalling protocol. I think the starting/stopping etc. are additional (optional) management operations on the newly deployed service. We have something similar in GRAM, where we support "hold states" that can pause the otherwise automatic lifecycle. Another rendering choice is whether these management operations are part of the service or part of the "container". We take the latter view in GRAM, so they would extend the Agreement interface which already represents our new transient container for jobs. The actual user job (deployed service) may or may not have any meaningful message interfaces of its own (web or otherwise). I think this is the way to generalize for BES in the way Andrew has advocated, where "jobs" may be of different types, e.g. a web service, a JVM, a fortran code, etc.! You can easily see how a broker might satisfy an abstract web service obligation by establishing more "concrete" agreements to run the underlying implementation components. Advance reservation, in the sense described in the GARA and SNAP work on which this was based, is the explicit decoupling of the "resource promise" from the binding of the resource to a service. It is a 2-phase protocol where one negotiates for the promise of capability and then later binds or "claims" the resource. It is not strictly necessary unless one needs to defer the binding decision or parameterization or deal with differences in cardinality, but it is what I was referring to when I described it separately from job-submission. This is a capability that multiple current batch schedulers possess, and I expect we should be able to expose it via WS-Agreement. karl -- Karl Czajkowski karlcz@univa.com

On Mar 21, Jon MacLaren loaded a tape reading: ...

A strawman interface was presented, but this was (I felt) little more than "these are the things that BES should support". Pre-conceived is a little harsh - everything presented can be changed.

Well, I didn't mean it with the negative connotation you are reading... I guess I should have chosen a better adjective. I meant that the only real consensus I have heard so far seems to center on some very traditional rendering. I am slowly learning to be conservative in assuming there is agreement where it hasn't been explicitly recognized. :-) I think this traditionalist approach relates directly to the aggressive schedule, and is not necessarily a bad thing for a starting point. However, to get everyone interested in BES to work through and understand the WS-Agreement rendering well enough to accept or reject it would put the remaining BES activity on a pace that I have yet to see happen in GGF! I'm just trying to share my view on it to set expectations in GRAAP... we need to make sure we get WS-Agreement right and on the next editing cycle; focusing too much on BES when things are this uncertain may not be productive in bringing WS-Agreement to closure. (I would love to see it adopted for BES, but I would not be willing to put that on the critical path unless BES is willing to delay its milestones if necessary. If their schedules are really so tight, having a "BES 2.0" that uses WS-Agreement might be wiser.)

...

3) There was a bit of the lingering anti-wsrf sentiment floating around Seoul.

There has been some excellent discussion on the OGSA-WG list just prior to this meeting on whether or not WS-RF is necessary, and if it is, what is it necessary for? But to be fair, it's not "sentiment" - there are some very astute people saying that the Grid community doesn't need WS-RF. Unfortunately, in Seoul, there was practically no debate on this subject (at least, not inside of the WG sessions!). I would expect to see this being played out in the OGSA-BES list. It's a debate that needs to happen.

Well, I would hope BES only considers its applicability to job submission/management! If there needs to be a larger architectural debate like whether "the Grid community [needs] WS-RF", it needs to happen in a broader context. I only brought it up because WS-Agreement is obviously WSRF-based, so in order to adopt WS-Agreement soon, the BES group would have to agree on the underlying technologies almost immediately.

Jon.

karl -- Karl Czajkowski karlcz@univa.com