- ogsa-wg - lists.ogf.org

RE: [ogsa-wg] RE: Modeling State: Technical Questions
by Ian Foster 05 Apr '05

05 Apr '05

Savas: I think your email misses the point of the current discussion. Of course, as you point out, if I have a handle of some sort for a job (a WS-Addressing EPR, as in WSRF/WS-Transfer, or a job-id, as you advocate below), then I can include several of these in a request to the job factory asking for some operation to be performed on them all. I don't believe that anyone's debating that. But that's not the topic of the recent thread. What Mark and Karl are discussing is how you would express more abstract requests such as "kill all jobs that have consumed more than one hour's CPU time." This is a question of query languages, I presume, not modeling of job ids. Ian. At 01:10 PM 4/5/2005 +0100, Savas Parastatidis wrote: >Dear all, > >I think something needs to be clarified with regards to handling >multiple jobs with one message. The beauty of document-oriented >interactions is that you can do things like... > ><job-details-request> > <job-id>urn:ogsa:job:guid:bla-bla-bla-001</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-010</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-029</job-id> ></job-details-request> > >Or > ><job-suspend-request> > <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-005</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-008</job-id> ></job-suspend-request> > >The schema for the above document can allow anything from 0 to N number >of <job-id> elements. > >What WS-RF and WS-Transfer and REST are doing is require that each >message be directed to only one resource. As a result, when it comes to >defining groups of resources, additional resources (representing >collections) have to be created. Populating and managing the collections >require additional messages. > >The WS-RF/Ws-Transfer/REST model is a special case of the >document-oriented model I described above... > > ><job-suspend-request> > <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> ></job-suspend-request> > >In the WS-RF/WS-Transfer case the job ID will have to be part of the >wsa:To (wsa -> WS-Addressing) header. In REST, it is the URI on which >the operations are called (if we are using HTTP and the HTTP verbs, then >the URI usually has the 'http' prefix). > >An example, of a WS-Addresing EPR... > ><my:MyEndpointReference> > <wsa:Address> urn:ogsa:job:guid:bla-bla-bla-002</wsa:Address> ></my:MyEndpointReference> > >Please note that the address doesn't have to carry transport/transfer >specific semantics (i.e. it doesn't have to be an 'http' URI). The above >would require a registry look up if that's necessary or perhaps a P2P >network that will know how to direct the message to its destination >based only on the above information. The sender of the message may never >actually see the transport-specific address of the receiving service. > >This means that a SOAP msg like the one bellow will have to be sent... > ><soap:Envelope> > <soap:Header> > <wsa:To>urn:ogsa:job:guid:bla-bla-bla-002</wsa:To> > </soap:Header> > <soap:Body> > <job:job-suspend-request /> > </soap:Body> ></soap:Envelope> > >Well... it turns out that this can by the special case of a message that >looks like this... > ><soap:Envelope> > <soap:Header> > <wsa:To>urn:ogsa:job:service:Newcastle-Job-Service</wsa:To> >  > </soap:Header> > <soap:Body> > <job:job-suspend-request> > <job:job-id>urn:ogsa:job:guid:bla-bla-bla-002</job:job-id> > <job:job-id>urn:ogsa:job:guid:bla-bla-bla-003</job:job-id> > <job:job-id>urn:ogsa:job:guid:bla-bla-bla-004</job:job-id> > </job:job-suspend-request> > </soap:Body> ></soap:Envelope> > >What WS-RF and WS-Transfer seem to be doing is to expose to the wire the >programming abstraction that most of us are used to (i.e. calling >methods on an object). As a result, systems based on a special case are >designed rather than the more general case. It's been our argument all >along that this may not be the most efficient way of designing systems >in general (perhaps in certain application domains the WS-RF/WS-Transfer >approach may be appropriate) but I am prepared to be corrected on this >:-) > >Best regards, >-- >Savas Parastatidis >http://savas.parastatidis.name > _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org

2 1

RE: [ogsa-wg] RE: Modeling State: Technical Questions
by Ian Foster 05 Apr '05

05 Apr '05

[I'm feeling increasingly bad about sending email to all of the people CCed here, who may not be interested in these issues at all but got addressed by Tony long ago...] Savas: It seems that we are in agreement, then, that we want the ability to: a) Request operations on individual jobs identified by some sort of "jobid" b) Request operations on sets of jobs identified by a user-supplied list of "jobids" c) Request operations on sets of jobs identified by more abstract criteria We also agree that (as I expressed in the email that started this discussion) such requests can be expressed in a few different ways, with somewhat different characteristics. That's progress I hope. Ian. At 02:44 PM 4/5/2005 +0100, Savas Parastatidis wrote: >Dear Ian, > > > >I dont think that the approach I proposed forces the user to do more than >they would have to do anyway if EPRs were used. It is still the case that >someone has to manage the EPRs to the resources in WSRF. This is similar >to what happens in the real world. The online bookstore will ask for my >credit card number (a URI), or the book store will as for an ISBN (another >URI) or multiple ISBNs if I want to buy multiple books. The banking >service will ask for my bank account number (another URI perhaps). > > > >Also, there is no reason why a kill all my jobsmessage couldnt also be >supported. But please note that this message is now addressed to the >service (the container of resources) and not, as in the case of WSRF, to a >specific resource. This is no different from what I am advocating. > > > >Also& to Steves point about partial failure. If one wishes atomic >transaction semantics, I dont see the difference from the two approaches& > > > >Atomic > > Msg -> resource 1 > > Msg -> resource 2 > > Msg -> resource 3 > >End Atomic > > > >Vs > > > >Msg > > Atomic > > Resource 1 > > Resource 2 > > Resource 3 > > End Atomic > > > >In fact, I would argue that the latter is better because: > > > >1. It uses fewer messages (and, Steve, I am not assuming only HTTP and the >optimisations that may be supported) > > > >2. I can more easily deal with the failures in an application >specific-manner since my atomic TX semantics do not span multiple msgs. > > > >(Anyway& who wants to do atomic TXs over the Web anyway? :-) > > > >Regards, > >-- >Savas Parastatidis ><http://savas.parastatidis.name>http://savas.parastatidis.name > > >From: Ian Foster [mailto:foster@mcs.anl.gov] >Sent: Tuesday, April 05, 2005 2:22 PM >To: Steve Loughran; Savas Parastatidis >Cc: Mark McKeown; Karl Czajkowski; Dennis Gannon; Samuel Meder; ogsa-wg; >dave.pearson(a)oracle.com; gray(a)microsoft.com; humphrey(a)cs.virginia.edu; >grimshaw(a)virginia.edu; aherbert(a)microsoft.com; gcf(a)indiana.edu; >mark.linesch(a)hp.com; Frank Siebenlist; Tony Hey; Dave Berry >Subject: Re: [ogsa-wg] RE: Modeling State: Technical Questions > > > >Steve's note raises a key point for me: do we really want to force the >user (as Savas seems to be advocating) to keep track of jobs running at a >remote site? > >I'd rather send a request "kill all my jobs" or "kill all my jobs that >have run for more than a day" to the factory than carefully keep track of >all jobs that I have active, and how long they have been running, so that >I can send the big document (or stream) discussed below. > >Ian. > > >At 02:10 PM 4/5/2005 +0100, Steve Loughran wrote: > >Savas Parastatidis wrote: > >Dear all, >I think something needs to be clarified with regards to handling >multiple jobs with one message. The beauty of document-oriented >interactions is that you can do things like... ><job-details-request> > <job-id>urn:ogsa:job:guid:bla-bla-bla-001</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-010</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-029</job-id> ></job-details-request> >Or ><job-suspend-request> > <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-005</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-008</job-id> ></job-suspend-request> >The schema for the above document can allow anything from 0 to N number >of <job-id> elements. > > >the trouble with any bulk operation is you have to handle partial failure. >You need either atomic operations (not long lived transactions over HTTP >Savas, I wouldn't be that daft), or a way of indicating that only a bit >went wrong > >Hence the 207 Multi-Status response in WebDav, the "something failed, look >in the message". WebDav is still single instance (here a RESTy URL), but >you can set >1 property and so have partial failure. > >SOAP just has SOAPFault and extensions; no explicit multiple failure >response. WS-RF-ResourceProperties has a similar problem with >SetResourceProperties, but a different failure model in which any failure >to set can result in a WS-BaseFault, indicating which failed, but >providing no apparent information on which worked. > >It seems to me that if you want to bulk stuff, you do need ways of (a) >handling partial failure and (b) declaring what happens on partial >failure. For the curions, WebDav's failure mode on file operations (MOVE, >COPY) is explicitly declared to be that of failed file operations of Win98 >on a FAT32 filesystem [1,2] > >Alternatively, you dont go for bulk operations, neither on a multiple >jobs, or on multiple properties of a job (remember, WS-RF doesn't declare >atomic/transacted property operations, so all you do here is increase the >window of instability, a window that already exists). Instead you just >stream a series of operations over the same HTTP1.1 connection -assuming >that everything is accessible at the same far-end host, and get a series >of (potentially out of order, we are talking HTTP1.1) responses. > >This could be efficient, and you could do better handling of failure. But >you do need a SOAP stack that can keep an HTTP1.1 channel open for >multiple requests. Axis doesnt, even if you get httpclient to do the HTTP >work; I don't know about .NET/WSE. You also need developers to model the >communication correctly. Manipulating JAXRPC proxies as if they represent >remote objects is *clearly* the wrong way to do it. You'd almost want to >model a queue of requests waiting to be POSTed, a queue you can fill up >then push out. Something like this, in your Java-era language of choice :- > >//different queues for SOAP, REST >Queue q=new Soap12RequestQueue(); > >q.add(new StatePut(job1.uri,Job.LIFECYCLE,Job.SUSPENDED)); >//let the queue reorder stuff if it wants to >q.add(new >StatePut(job2.uri,Job.LIFECYCLE,Job.SUSPENDED),Queue.POSITION_OPTIMAL); >q.add(new StatePut(job3.uri,Job.LIFECYCLE,Job.SUSPENDED),Queue.POSITION_LAST); > >q.setEventHandler(this); >q.nonBlockingSubmit(); > >No, there is no code behind this example, and I am avoiding any hints as >to what the even handler would look like. I think the key point is that >once you embrace remote operations as async actions, then you can model >the manipulations differently. Note also that I am representing job >suspension not as an explicit suspend() operation, but as a request to put >a job into the suspended state. This API could work with our friend REST >just as easily as with WS-RF... > >Anyway Savas, to conclude: do you have any evidence that a single document >is suboptimal compared to a sequences of requests over an open HTTP/1.1 >connection? That is, assuming we ignore the SHOULD in the HTTP1.1 >specification " Clients SHOULD NOT pipeline requests using non-idempotent >methods or non-idempotent sequences of methods" [3] > >-Steve > > >[1] WebDav http://www.ietf.org/rfc/rfc2518.txt S8.9.2 > >"after encountering an error moving a non-collection > resource as part of an infinite depth move, the server SHOULD try to > finish as much of the original move operation as possible." > >[2] http://lists.w3.org/Archives/Public/w3c-dist-auth/1997JulSep/0177.html > >[3] RFC2616 HTTP1.1 > >_______________________________________________________________ >Ian Foster www.mcs.anl.gov/~foster >Math & Computer Science Div. Dept of Computer Science >Argonne National Laboratory The University of Chicago >Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. >Tel: 630 252 4619 Fax: 630 252 1997 > Globus Alliance, www.globus.org _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org

1 0

RE: [ogsa-wg] Telcon for OGSA ByteIO WG charter
by Dave Berry 05 Apr '05

05 Apr '05

Sorry about this... We are postponing to next week. Dave. > -----Original Message----- > From: Dave Berry > Sent: 05 April 2005 16:28 > To: Dave Berry; ogsa-wg > Cc: Dave Pearson; Mario Antonioletti; Martin Westhead; > gtr(a)ast.cam.ac.uk; chappella(a)battelle.org; arun(a)sdsc.edu; > jxu(a)us.ibm.com; o.tatebe(a)aist.go.jp; Yves.Denneulin(a)imag.fr; > ppadala(a)cise.ufl.edu; ivm(a)fnal.gov; jalameda(a)ncsa.uiuc.edu; > annc(a)isi.edu; madsen(a)us.ibm.com; allcock(a)mcs.anl.gov; Allen > Luniewski; Malcolm Atkinson; Susan Malaika; Peter Clarke > (UCL); David E Martin; Peter Kunszt; Ian.Bird(a)cern.ch; > Erwin.Laure(a)cern.ch; Gavin.Mccance(a)cern.ch; Scott Oster; > Stephen Langella; Shannon Hastings; Manuel Pereira; Ted > Anderson; Leo Luan; Mark M. Morgan > Subject: RE: [ogsa-wg] Telcon for OGSA ByteIO WG charter > > There was a mixup with the telcon booking - it's sorted now. > If you were trying to call in, please try again. > > Dave > > > -----Original Message----- > > From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On > > Behalf Of Dave Berry > > Sent: 01 April 2005 11:42 > > To: ogsa-wg > > Cc: Dave Pearson; Mario Antonioletti; Martin Westhead; > > gtr(a)ast.cam.ac.uk; chappella(a)battelle.org; arun(a)sdsc.edu; > > jxu(a)us.ibm.com; o.tatebe(a)aist.go.jp; Yves.Denneulin(a)imag.fr; > > ppadala(a)cise.ufl.edu; ivm(a)fnal.gov; jalameda(a)ncsa.uiuc.edu; > > annc(a)isi.edu; madsen(a)us.ibm.com; allcock(a)mcs.anl.gov; Allen > > Luniewski; Malcolm Atkinson; Susan Malaika; Peter Clarke > > (UCL); David E Martin; Peter Kunszt; Ian.Bird(a)cern.ch; > > Erwin.Laure(a)cern.ch; Gavin.Mccance(a)cern.ch; Scott Oster; > > Stephen Langella; Shannon Hastings; Manuel Pereira; Ted > > Anderson; Leo Luan; Mark M. Morgan > > Subject: [ogsa-wg] Telcon for OGSA ByteIO WG charter > > > > Dear all, > > > > The ByteIO group plans on having it's first telecon next > > Tuesday at 11AM > > Eastern (8AM Pacific, 4PM UK and midnight in Japan). Our > > agenda for that > > telecon includes the group's first discussion of the > charter. Please > > see the attachments. > > > > > > UK Dial In: 0845 245 0224 > > US Dial In: 1866 838 0046 > > International Dial In: +44 1452 542309 > > > > PIN Number: 632 065 > > > > Best wishes, > > > > > > Dave Berry > > Research Manager, National e-Science Centre > > 15 South College St., Edinburgh > > Tel: +44 131 6514039 > > >

1 0

RE: [ogsa-wg] Telcon for OGSA ByteIO WG charter
by Dave Berry 05 Apr '05

05 Apr '05

There was a mixup with the telcon booking - it's sorted now. If you were trying to call in, please try again. Dave > -----Original Message----- > From: owner-ogsa-wg(a)ggf.org [mailto:owner-ogsa-wg@ggf.org] On > Behalf Of Dave Berry > Sent: 01 April 2005 11:42 > To: ogsa-wg > Cc: Dave Pearson; Mario Antonioletti; Martin Westhead; > gtr(a)ast.cam.ac.uk; chappella(a)battelle.org; arun(a)sdsc.edu; > jxu(a)us.ibm.com; o.tatebe(a)aist.go.jp; Yves.Denneulin(a)imag.fr; > ppadala(a)cise.ufl.edu; ivm(a)fnal.gov; jalameda(a)ncsa.uiuc.edu; > annc(a)isi.edu; madsen(a)us.ibm.com; allcock(a)mcs.anl.gov; Allen > Luniewski; Malcolm Atkinson; Susan Malaika; Peter Clarke > (UCL); David E Martin; Peter Kunszt; Ian.Bird(a)cern.ch; > Erwin.Laure(a)cern.ch; Gavin.Mccance(a)cern.ch; Scott Oster; > Stephen Langella; Shannon Hastings; Manuel Pereira; Ted > Anderson; Leo Luan; Mark M. Morgan > Subject: [ogsa-wg] Telcon for OGSA ByteIO WG charter > > Dear all, > > The ByteIO group plans on having it's first telecon next > Tuesday at 11AM > Eastern (8AM Pacific, 4PM UK and midnight in Japan). Our > agenda for that > telecon includes the group's first discussion of the charter. Please > see the attachments. > > > UK Dial In: 0845 245 0224 > US Dial In: 1866 838 0046 > International Dial In: +44 1452 542309 > > PIN Number: 632 065 > > Best wishes, > > > Dave Berry > Research Manager, National e-Science Centre > 15 South College St., Edinburgh > Tel: +44 131 6514039 >

1 0

RE: Telcon for OGSA ByteIO WG charter
by Allen Luniewski 05 Apr '05

05 Apr '05

I am getting the same message. Allen "Mark Morgan" <mmm2a(a)cs.virginia.edu> 04/05/2005 08:01 AM To "'Dave Berry'" <daveb(a)nesc.ac.uk>, "'ogsa-wg'" <ogsa-wg(a)gridforum.org> cc "'Dave Pearson'" <Dave.Pearson(a)oracle.com>, "'Mario Antonioletti'" <mario(a)epcc.ed.ac.uk>, "'Martin Westhead'" <M.Westhead(a)epcc.ed.ac.uk>, <gtr(a)ast.cam.ac.uk>, <chappella(a)battelle.org>, <arun(a)sdsc.edu>, Jane Xu/San Jose/IBM@IBMUS, <o.tatebe(a)aist.go.jp>, <Yves.Denneulin(a)imag.fr>, <ppadala(a)cise.ufl.edu>, <ivm(a)fnal.gov>, <jalameda(a)ncsa.uiuc.edu>, <annc(a)isi.edu>, Cecile Madsen/Santa Teresa/IBM@IBMUS, <allcock(a)mcs.anl.gov>, "'Allen Luniewski'" <luniew(a)almaden.ibm.com>, "'Malcolm Atkinson'" <mpa(a)nesc.ac.uk>, Susan Malaika/Santa Teresa/IBM@IBMUS, "'Peter Clarke \(UCL\)'" <clarke(a)hep.ucl.ac.uk>, David E Martin/Chicago/IBM@IBMUS, "'Peter Kunszt'" <pkunszt(a)mail.cern.ch>, <Ian.Bird(a)cern.ch>, <Erwin.Laure(a)cern.ch>, <Gavin.Mccance(a)cern.ch>, "'Scott Oster'" <oster(a)bmi.osu.edu>, "'Stephen Langella'" <langella(a)bmi.osu.edu>, "'Shannon Hastings'" <hastings(a)bmi.osu.edu>, Manuel Pereira/Almaden/IBM@IBMUS, Ted Anderson/Almaden/IBM@IBMUS, "'Leo Luan'" <luan(a)almaden.ibm.com> Subject RE: Telcon for OGSA ByteIO WG charter I'm getting a message indicating that this passcode is not recognized. Is anyone else having any luck? -Mark > -----Original Message----- > From: Dave Berry [mailto:daveb@nesc.ac.uk] > Sent: Friday, April 01, 2005 5:42 AM > To: ogsa-wg > Cc: Dave Pearson; Mario Antonioletti; Martin Westhead; > gtr(a)ast.cam.ac.uk; chappella(a)battelle.org; arun(a)sdsc.edu; > jxu(a)us.ibm.com; o.tatebe(a)aist.go.jp; Yves.Denneulin(a)imag.fr; > ppadala(a)cise.ufl.edu; ivm(a)fnal.gov; jalameda(a)ncsa.uiuc.edu; > annc(a)isi.edu; madsen(a)us.ibm.com; allcock(a)mcs.anl.gov; Allen > Luniewski; Malcolm Atkinson; Susan Malaika; Peter Clarke > (UCL); David E Martin; Peter Kunszt; Ian.Bird(a)cern.ch; > Erwin.Laure(a)cern.ch; Gavin.Mccance(a)cern.ch; Scott Oster; > Stephen Langella; Shannon Hastings; Manuel Pereira; Ted > Anderson; Leo Luan; Mark M. Morgan > Subject: Telcon for OGSA ByteIO WG charter > > Dear all, > > The ByteIO group plans on having it's first telecon next > Tuesday at 11AM Eastern (8AM Pacific, 4PM UK and midnight in > Japan). Our agenda for that telecon includes the group's > first discussion of the charter. Please see the attachments. > > > UK Dial In: 0845 245 0224 > US Dial In: 1866 838 0046 > International Dial In: +44 1452 542309 > > PIN Number: 632 065 > > Best wishes, > > > Dave Berry > Research Manager, National e-Science Centre > 15 South College St., Edinburgh > Tel: +44 131 6514039 >

1 0

RE: [ogsa-wg] RE: Modeling State: Technical Questions
by Savas Parastatidis 05 Apr '05

05 Apr '05

Dear all, I think something needs to be clarified with regards to handling multiple jobs with one message. The beauty of document-oriented interactions is that you can do things like... <job-details-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-001</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-010</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-029</job-id> </job-details-request> Or <job-suspend-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-005</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-008</job-id> </job-suspend-request> The schema for the above document can allow anything from 0 to N number of <job-id> elements. What WS-RF and WS-Transfer and REST are doing is require that each message be directed to only one resource. As a result, when it comes to defining groups of resources, additional resources (representing collections) have to be created. Populating and managing the collections require additional messages. The WS-RF/Ws-Transfer/REST model is a special case of the document-oriented model I described above...  <job-suspend-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> </job-suspend-request> In the WS-RF/WS-Transfer case the job ID will have to be part of the wsa:To (wsa -> WS-Addressing) header. In REST, it is the URI on which the operations are called (if we are using HTTP and the HTTP verbs, then the URI usually has the 'http' prefix). An example, of a WS-Addresing EPR... <my:MyEndpointReference> <wsa:Address> urn:ogsa:job:guid:bla-bla-bla-002</wsa:Address> </my:MyEndpointReference> Please note that the address doesn't have to carry transport/transfer specific semantics (i.e. it doesn't have to be an 'http' URI). The above would require a registry look up if that's necessary or perhaps a P2P network that will know how to direct the message to its destination based only on the above information. The sender of the message may never actually see the transport-specific address of the receiving service. This means that a SOAP msg like the one bellow will have to be sent... <soap:Envelope> <soap:Header> <wsa:To>urn:ogsa:job:guid:bla-bla-bla-002</wsa:To> </soap:Header> <soap:Body> <job:job-suspend-request /> </soap:Body> </soap:Envelope> Well... it turns out that this can by the special case of a message that looks like this... <soap:Envelope> <soap:Header> <wsa:To>urn:ogsa:job:service:Newcastle-Job-Service</wsa:To>  </soap:Header> <soap:Body> <job:job-suspend-request> <job:job-id>urn:ogsa:job:guid:bla-bla-bla-002</job:job-id> <job:job-id>urn:ogsa:job:guid:bla-bla-bla-003</job:job-id> <job:job-id>urn:ogsa:job:guid:bla-bla-bla-004</job:job-id> </job:job-suspend-request> </soap:Body> </soap:Envelope> What WS-RF and WS-Transfer seem to be doing is to expose to the wire the programming abstraction that most of us are used to (i.e. calling methods on an object). As a result, systems based on a special case are designed rather than the more general case. It's been our argument all along that this may not be the most efficient way of designing systems in general (perhaps in certain application domains the WS-RF/WS-Transfer approach may be appropriate) but I am prepared to be corrected on this :-) Best regards, -- Savas Parastatidis http://savas.parastatidis.name

4 3

RE: [ogsa-wg] RE: Modeling State: Technical Questions
by Savas Parastatidis 05 Apr '05

05 Apr '05

Dear Ian, My apologies... I should have made it clear that I was referring to the beginning of today's thread when the issue of how to address multiple jobs was discussed. Regards, -- Savas Parastatidis http://savas.parastatidis.name ________________________________ From: Ian Foster [mailto:foster@mcs.anl.gov] Sent: Tuesday, April 05, 2005 2:08 PM To: Savas Parastatidis; Mark McKeown; Karl Czajkowski Cc: Dennis Gannon; Samuel Meder; ogsa-wg; dave.pearson(a)oracle.com; savas.parastatidis(a)ncl.ac.uk; gray(a)microsoft.com; humphrey(a)cs.virginia.edu; grimshaw(a)virginia.edu; aherbert(a)microsoft.com; gcf(a)indiana.edu; mark.linesch(a)hp.com; Frank Siebenlist; Tony Hey; Dave Berry Subject: RE: [ogsa-wg] RE: Modeling State: Technical Questions Savas: I think your email misses the point of the current discussion. Of course, as you point out, if I have a handle of some sort for a job (a WS-Addressing EPR, as in WSRF/WS-Transfer, or a job-id, as you advocate below), then I can include several of these in a request to the job factory asking for some operation to be performed on them all. I don't believe that anyone's debating that. But that's not the topic of the recent thread. What Mark and Karl are discussing is how you would express more abstract requests such as "kill all jobs that have consumed more than one hour's CPU time." This is a question of query languages, I presume, not modeling of job ids. Ian. At 01:10 PM 4/5/2005 +0100, Savas Parastatidis wrote: Dear all, I think something needs to be clarified with regards to handling multiple jobs with one message. The beauty of document-oriented interactions is that you can do things like... <job-details-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-001</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-010</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-029</job-id> </job-details-request> Or <job-suspend-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-005</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-008</job-id> </job-suspend-request> The schema for the above document can allow anything from 0 to N number of <job-id> elements. What WS-RF and WS-Transfer and REST are doing is require that each message be directed to only one resource. As a result, when it comes to defining groups of resources, additional resources (representing collections) have to be created. Populating and managing the collections require additional messages. The WS-RF/Ws-Transfer/REST model is a special case of the document-oriented model I described above...  <job-suspend-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> </job-suspend-request> In the WS-RF/WS-Transfer case the job ID will have to be part of the wsa:To (wsa -> WS-Addressing) header. In REST, it is the URI on which the operations are called (if we are using HTTP and the HTTP verbs, then the URI usually has the 'http' prefix). An example, of a WS-Addresing EPR... <my:MyEndpointReference> <wsa:Address> urn:ogsa:job:guid:bla-bla-bla-002</wsa:Address> </my:MyEndpointReference> Please note that the address doesn't have to carry transport/transfer specific semantics (i.e. it doesn't have to be an 'http' URI). The above would require a registry look up if that's necessary or perhaps a P2P network that will know how to direct the message to its destination based only on the above information. The sender of the message may never actually see the transport-specific address of the receiving service. This means that a SOAP msg like the one bellow will have to be sent... <soap:Envelope> <soap:Header> <wsa:To>urn:ogsa:job:guid:bla-bla-bla-002</wsa:To> </soap:Header> <soap:Body> <job:job-suspend-request /> </soap:Body> </soap:Envelope> Well... it turns out that this can by the special case of a message that looks like this... <soap:Envelope> <soap:Header> <wsa:To>urn:ogsa:job:service:Newcastle-Job-Service</wsa:To>  </soap:Header> <soap:Body> <job:job-suspend-request> <job:job-id>urn:ogsa:job:guid:bla-bla-bla-002</job:job-id> <job:job-id>urn:ogsa:job:guid:bla-bla-bla-003</job:job-id> <job:job-id>urn:ogsa:job:guid:bla-bla-bla-004</job:job-id> </job:job-suspend-request> </soap:Body> </soap:Envelope> What WS-RF and WS-Transfer seem to be doing is to expose to the wire the programming abstraction that most of us are used to (i.e. calling methods on an object). As a result, systems based on a special case are designed rather than the more general case. It's been our argument all along that this may not be the most efficient way of designing systems in general (perhaps in certain application domains the WS-RF/WS-Transfer approach may be appropriate) but I am prepared to be corrected on this :-) Best regards, -- Savas Parastatidis http://savas.parastatidis.name <http://savas.parastatidis.name/> _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/>

1 0

RE: Modeling State: Technical Questions
by Ian Foster 05 Apr '05

05 Apr '05

Paul: My apologies for the slow reply: I left on vacation soon after sending my email. Your notes all look good to me. I'm putting together a revised version of my discussion document that trys to capture these and other points. Ian. At 12:14 PM 3/11/2005 +0000, Paul Watson wrote: >Dear Ian, > >Having faithfully promised myself that I wouldn't spend more time on >discussions about low-level WS specs, I found myself drawn in as this is >a very interesting comparison (it reminded me of the excellent work done >in the DAIS WG to compare different approaches to database access >services). In particular, it gives me a concrete example that I can use >to ask some questions about the use of WSRF. Given that I come from a >non-WSRF background (my experience is with services that use the >approach in your option #3) much of what I've written below is about >understanding the motivation for switching to the WSRF approach. But, >I'd like to stress that this isn't intended to be hostile ... it's meant >to improve my understanding and (hopefully) move on the technical >debate. > >I was going to send it to you only, as you offered to collate feedback, >but as Dave Berry is proposing to discuss these options next week (and I >won't be there) and has posted on it, I thought that I'd send it to Dave >and others too as input to these discussions. > > > OPTIONS > > > > 1) (WSRF) > > > > The job handle is an EPR to a WS-Resource, in which case the case the > > WSRF operations can be used to get status and delete: > > > > send "GetResourceProperty(status)" to job-handle > > send "Destroy" to job-handle > > > >If there are other operations that need to be supported on jobs, e.g. >"suspend, resume, checkpoint, change priority"... then am I right in >thinking that these would be provided by application-specific operations >on resources that were not defined by WSRF? > >e.g. send "Suspend" to job-handle, >send "changePriority" to job-handle etc. > > > In addition, one will probably also want to be able to send requests > > to the job-factory, which is straightforward as there is no strong > > encapsulation in WSRF. E.g.: > > > > send "destroy(all-jobs-owned-by-ian)" to job-factory-web-service > >Given previous misunderstandings, it is good to see that your WSRF >example also includes some operations on the service. > >I think that the motivation for introducing operations on the service is >that these can then operate over a set of resources (e.g. >"all-jobs-owned-by-ian"). This reduces the number of messages (e.g. one >interaction with the service vs. one to each resource in the set of >"all-jobs-owned-by-ian"), and also allows the set of resources to be >selected using information encapsulated within the service (e.g. >"jobs-owned-by-ian", "jobs that have nearly reached their quota", "jobs >that have run for more than 24 hours"). Therefore, for similar reasons, >it would, I think, also be beneficial to define a "check-jobs-status" >operation on the service. This would allow, for example: > >check-jobs-status("all-jobs-owned-by-ian") > >Also, as, "all-jobs-owned-by-ian" defines a set of jobs, it would seem >reasonable to allow the consumer to specify the set of jobs by name: > >"destroy(job1,job27,job35)" >"check-jobs-status(job1,job27,job35)" > >(I may be wrong, but I think that the current naming proposal would >allow this by giving the consumer the abstract name for a resource as >well as the epr). > >These operations on the service are the sort I'd expect to see in option >#3 (as I'll explain below). So, I think that one could view option #1 as >taking the operations from option #3 and then adding to them a set of >operations on individual resources. > >These operations on the service allow a consumer to (I think) do >everything without using the operations on individual resources. This is >because these operations address the general case for a job submission >service... they provide operations over the set of resources >encapsulated by the service. They appear to have the advantage of >supporting and encouraging larger-grained operations, e.g. in one >message you can ask about a set of resources rather than just one. I >expect that will be important for performance, given the costs of >sending messages between web services. The operations on individual >resources seem to be a special case of the general case, and this raises >the question (to me) that if the job management service provides the >general case, then when is it better to use the special case? > > > 3) (Job id) The job-handle is a job-factory-specific string, > >I'd think that it could be an "abstract name" in OGSA naming terms >(though the naming proposal isn't fixed yet, so I may be wrong). > > > and we define > > job-factory-specific operations for manipulating the job, e.g.: > > > > send "check-job-status(job-handle)" to job-factory > > send "terminate-job(job-handle)" to job-factory > > > > #3 is very similar to #1 and #2: the message sent over the wire in > > each case contains the same information (operation name + Web service > > address + job-id-info). > >As described in my comments on option #1 above, I'd suggest that both >these operations should be defined to operate on a set of jobs (c.f. >option #1's "destroy(all-jobs-owned-by-ian)). > >e.g. the operations on the service would allow: > >"check-jobs-status(job1, job14,job27)" >"terminate-jobs(job2, job35)" > >Allowing multiple job names in a message is, I think, no more difficult >to implement, and naturally scales up to larger granularity operations >(e.g. one interaction with a service is enough to carry out actions on >1,2 or 500 jobs). > >Option #3 also differs from #1 and #2 as it does not rely on >ws-addressing (though it could use it). [A personal view is that I see >this as an advantage for those who want to design and build services now >or in the near future. This is because the service can use only >currently stable specifications found in the set "blessed" by >WS-Interoperability, and these are provided in an interoperable manner >by a wide range of existing WS platforms.] > > > What is different about #3 is that you have to manage this job-handle > > separately from the job-factory EPR. Also, you don't use the > > standardized WSRF/WSN operations. > > > > Have I captured all of the options that people think we should be > > considering? > > > >I note that Dave & Karl have now posted about the possible use of >ResourceGroups. I based the above on your message which did not discuss >their use, and so I wonder if it would therefore be helpful for someone >to produce a version of option #1 that creates and uses resourcegroups. > >This was a very good start for discussion, and I'd be interested on your >views on my comments above. > >Regards > >Paul. > > >-----Original Message----- > >From: Ian Foster [mailto:foster@mcs.anl.gov] > >Sent: 07 March 2005 04:47 > >To: Tony Hey; Frank Siebenlist > >Cc: Dennis Gannon; Samuel Meder; ogsa-wg; > >paul.watson(a)ncl.ac.uk; dave.pearson(a)oracle.com; > >savas.parastatidis(a)ncl.ac.uk; gray(a)microsoft.com; > >humphrey(a)cs.virginia.edu; grimshaw(a)virginia.edu; > >aherbert(a)microsoft.com; gcf(a)indiana.edu; mark.linesch(a)hp.com > >Subject: Modeling State: Technical Questions > > > >Dear All: > > > >In an effort to move forward this debate about "OGSA > >plumbing", I thought > >I'd attempt to describe the different positions that I've seen > >expressed. > >Perhaps people can read this quickly, and provide feedback as > >to where I > >have not captured things correctly. I'll be happy to collate > >responses sent > >directly to me and forward them, if there is interest. > > > > > >CONTEXT > > > >The context is that OGSA-WG needs to develop some low-level > >specifications, > >such as execution management services, that require modeling > >interactions > >that involve server-side state. E.g., in the following > >pseudo-code example, > >we request the creation of a "job": > > > > send "create(jsdl-specification)" to job-factory-web-service > > receive "job-handle" from job-factory-web-service > > > >I think that everyone accept that this sort of interaction is > >needed. (Let > >me know if not.) > > > >Where people differ in their views is with respect to what the > >"job-handle" > >should be and how we should represent operations on the job. > >I've heard > >three views expressed, as follows. > > > > > >OPTIONS > > > >1) (WSRF) > > > >The job handle is an EPR to a WS-Resource, in which case the > >case the WSRF > >operations can be used to get status and delete: > > > > send "GetResourceProperty(status)" to job-handle > > send "Destroy" to job-handle > > > >In addition, one will probably also want to be able to send > >requests to the > >job-factory, which is straightforward as there is no strong > >encapsulation > >in WSRF. E.g.: > > > > send "destroy(all-jobs-owned-by-ian)" to job-factory-web-service > > > >This approach exploits the standardized syntax and semantics for the > >various operations (GetResourceProperty, Destroy, etc.) defined by the > >WSRF/WSN specifications. > > > > > >2) (WS-Transfer) > > > >The job-handle is an EPR to a WS-Transfer resource, in which case the > >appropriate WS-Transfer operations can be used to get status > >and delete. > > > > send "Get" to job-handle > > send "Delete" to job-handle > > > >Assuming that there is no strong encapsulation in WS-Transfer > >either, one > >can also contact the job-factory directly when required: > > > > send "destroy(jobs-owned-by-ian)" to job-factory > > > >This approach exploits the syntax and semantics for the > >various operations > >(GetResourceProperty, Destroy, etc.) defined by the > >WS-Transfer specification. > > > >#2 is very similar to #1. One can argue the pros and cons of the two > >approaches. I'm concerned that you have to return the whole > >resource state, > >not just a single component (e.g., the status field). But that's a > >relatively minor issue. > > > > > >3) (Job id) The job-handle is a job-factory-specific string, > >and we define > >job-factory-specific operations for manipulating the job, e.g.: > > > > send "check-job-status(job-handle)" to job-factory > > send "terminate-job(job-handle)" to job-factory > > > >#3 is very similar to #1 and #2: the message sent over the > >wire in each > >case contains the same information (operation name + Web > >service address + > >job-id-info). What is different about #3 is that you have to > >manage this > >job-handle separately from the job-factory EPR. Also, you > >don't use the > >standardized WSRF/WSN operations. > > > >Have I captured all of the options that people think we should > >be considering? > > > > > >WHAT OGSA-WG SHOULD DO > > > >Given this characterization, I'd also be interested in > >people's views as to > >how OGSA-WG should proceed when considering specifications > >that involve > >this sort of interaction pattern. > > > >The majority of active participants OGSA-WG participants seem to favor > >option #1, but I've also heard people argue in favor of > >approaches #2 and #3. > > > >Tony Hey proposed, I think, that we should define > >specifications like job > >management in terms of some more abstract interface that can > >then be mapped > >to each of the approaches above, as desired. > > > >Frank Seibenlist observed that we can use WSRF/WSN as such an abstract > >interface: given an interface defined in terms of WSRF/WSN > >specifications, > >one can derive, via straightforward transformations, a concrete > >specification in the style of approach #2 or #3. > > > >Regards -- Ian. > > > > > > _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org

3 5

Re: [ogsa-wg] RE: Modeling State: Technical Questions
by Hiro Kishimoto 05 Apr '05

05 Apr '05

Roger's email bounced. ---- Hiro Kishimoto Subject: Re: [ogsa-wg] RE: Modeling State: Technical Questions From: Roger Menday <r.menday(a)fz-juelich.de> To: Karl Czajkowski <karlcz(a)univa.com> Cc: ogsa-wg(a)gridforum.org, Mark McKeown <zzalsmm3(a)nessie.mcc.ac.uk> Date: Tue, 05 Apr 2005 12:48:07 +0200 On Tue, 2005-04-05 at 17:09 +0700, Karl Czajkowski wrote: > On Apr 05, Mark McKeown loaded a tape reading: > ... > > I am not sure why people think there should only be a one-to-one > > mapping of a job to a resource. > > I do not think anyone is saying there should ONLY be a one-to-one > mapping, but rather that there is a one-to-one mapping of job to > resource for the mythical 80% solution. In fact, it is this mapping > that gives us an inutitive sense of there being a job at all! I have > come to think that part of the appeal of the WSRF approach for many is > to be able to more clearly codify which operations are applicable to a > specific resource for these 80% cases. It does not preclude the other > 20% solutions, but gives us more firm ground upon which to think about > interoperability for the straightforward scenarios. > > As for the rest of your message (sorry, no pun intended), I think you > underestimate the difficulty: you need a consensus on how to normalize > the concept space around all the different job metadata that might be > useful to reference jobs in different usage scenarios. This > normalization is inherent in having the hierarchical URI space in > which you've framed your simple examples. My experience leads me to > have a flight response when I hear "consensus", "normalization", and > "concept modeling" in the same paragraph... > > As an example, I find it exceedingly strange that the job's transient > state is part of its structured name below, but its unique numeric > identifier is used as a reference parameter. To me, it should clearly > be the other way around. So, each job has it's own URI, eg, http://thegrid/users/ian/jobs/435 Then, doing a GET on this returns some document of job metadata (??) Roger > I don't think you can really avoid the > adoption of a query language for the 20% cases, whether it is some > ad-hoc URI+reference parameter strategy like this, or > XPath/XQuery/etc. against a structured group abstraction. I think a > fruitful standards activity must start with a flat per-job reference > model and then add (optional) grouping abstractions to handle these > more exotic cases. > > > karl > > > >... Applying REST: > > > > GET http://thegrid/user/Ian/jobs > > > > returns a representation of all of Ian's jobs > > (or maybe just Xlinks to resources that represent those jobs). > > > > To suspend all the active jobs send a suspend message to the > > resource that represents all the Active jobs: > > > > POST http://thegrid/user/Ian/jobs/Active 'Suspend' > > > > > > Suspending specific jobs of Ian's, send a suspend message to the > > resource that represents the specific jobs: > > > > POST http://thegrid/user/Ian/jobs/Active?job=27&job=35 'Suspend' > > > > or > > > > DELETE http://thegrid/user/Ian/jobs/Active?job=27&job=35 > > > > > > Suspend all jobs that have used over 1 hours of CPU time owned by > > Ian: > > > > POST http://thegrid/user/Ian/jobs/Active?MinCPUTimeUsed=3600 > > 'Suspend' > > > > > > > > Using conventional SOAP operation this looks like: > > > > suspend('All-jobs-owned-by-Ian + CPUtime > 36000' ) > > > > Doing it this way requires coming up with a suitable query language. > > Access control could be more difficult, in the REST case only Ian > > should be able to access resources starting with > > http://thegrid/user/Ian/. > > > > In theory the above could be done with WS-Resources using WSRF > > - I suspect it would be more difficult/complex though. > > > > > > Perhaps I am very wrong though... > > > > cheers > > Mark > > > > >

1 0

Re: [ogsa-wg] RE: Modeling State: Technical Questions
by Karl Czajkowski 05 Apr '05

05 Apr '05

On Apr 05, Roger Menday loaded a tape reading: > On Tue, 2005-04-05 at 17:09 +0700, Karl Czajkowski wrote: ... > > As an example, I find it exceedingly strange that the job's transient > > state is part of its structured name below, but its unique numeric > > identifier is used as a reference parameter. To me, it should clearly > > be the other way around. > > So, each job has it's own URI, eg, http://thegrid/users/ian/jobs/435 > > Then, doing a GET on this returns some document of job metadata > > (??) > > Roger > I suppose, or really http://server.somewhere.grid/jobsXYZ/TlkUjg13DunN8ie2 I think the URI is really just an opaque dispatch path that makes sense to the endpoint designer and not necessarily anyone else. As soon as you start trying to organize or decompose this space according to conceptual interface rather than implementation, you run headlong into the community modeling tar-pit. My starting assumption is that there will be different communities or sects with different domain models. The danger with using URIs and reference parameters for this is that it is very poorly typed, so there is very little tooling support to express how the concept space has been modeled nor to perform meaningful validation at runtime. There is also very little support for building discovery systems to catalog and organize information about different resources using such an implied model. Using something like XPath or XQuery to select resources from a virtual XML "group" document seems a lot more disciplined to me, as does having XML representation of the "valid" operations and metadata for those resources (WSDL and WS-ResourceProperties, in the WSRF approach). As to whether GET on resource URIs actually does anything useful, I'll leave that for people who care about such things. :-) Similarly, whether DNS is sufficient for practical, efficient, resilient naming of URI namespace "authorities" is also something I'll leave to others. Personally, I hope that the resource URI is more than a transient service address. It seems to me that some of the WS-Addressing debate relates to whether this is the case in practice or not. karl -- Karl Czajkowski karlcz(a)univa.com

1 0