April 2005 - ogsa-wg - lists.ogf.org

RE: [ogsa-wg] RE: Modeling State: Technical Questions
by Mark McKeown 07 Apr '05

07 Apr '05

Hi Paul, Moving the question from can I suspend multiple jobs by sending a single message to a resource (either REST or WS-Resource) to weither this is a good thing. There is a balance between simplicity and efficiency - using a single message intoduces more complexities, as Steve Loughran illustrated, but is potentially more efficient than sending mutliple messages. Remembering that "Early optimisation is the root of all evil" (Knuth) - is adding support for suspending mutiple jobs using a single message an example of early optimisation? I would imagine that this should be a straight forward question since there is already considerable experience in using computational grids. Are users demanding the ability to suspend mutliple jobs using a single message? Is it for improved efficiency reasons? From my experience no, but others on this list will have considerably more experience. Could this be a case of "worse is better", simplicity is more important than efficiency? Perhaps there are other reasons for using a single message to interact with multiple jobs? cheers Mark > Ian, > > > > I agree that this is good progress. So let's bank that and see if we can > we can agree on one more thing, and then I'll ask a question. > > > > Considering your list of abilities (a, b & c) below, do we agree that in > terms of expressiveness, the ordering is: > > > > c>b>a > > > > i.e. using approach c, a client can request operations on: > > a) single jobs: "where (jobid = urn:guid:364)" > > b) sets of jobs: "where (jobid = urn:guid:364) or (jobid = > urn:guid:401)" > > > > If there is agreement on this, then we could move on to discussing why > it is felt necessary to provide more than just c for the job submission > service. > > > > Regards > > Paul > > > > Ian wrote... > > >Savas: > > > > > >It seems that we are in agreement, then, that we want the ability to: > > > > > >a) Request operations on individual jobs identified by some sort of > "jobid" > > > > > >b) Request operations on sets of jobs identified by a user-supplied > list of "jobids" > > > > > >c) Request operations on sets of jobs identified by more abstract > criteria > > > > > >We also agree that (as I expressed in the email that started this > discussion) such >requests can be expressed in a few different ways, > with somewhat different >characteristics. > > > > > >That's progress I hope. > > > > > >Ian. > > > > ________________________________ > > From: Ian Foster [mailto:foster@mcs.anl.gov] > Sent: 05 April 2005 17:59 > To: Savas Parastatidis; Steve Loughran > Cc: Mark McKeown; Karl Czajkowski; Dennis Gannon; Samuel Meder; ogsa-wg; > dave.pearson(a)oracle.com; gray(a)microsoft.com; humphrey(a)cs.virginia.edu; > grimshaw(a)virginia.edu; aherbert(a)microsoft.com; gcf(a)indiana.edu; > mark.linesch(a)hp.com; Frank Siebenlist; Tony Hey; Dave Berry; Paul Watson > Subject: RE: [ogsa-wg] RE: Modeling State: Technical Questions > > > > [I'm feeling increasingly bad about sending email to all of the people > CCed here, who may not be interested in these issues at all but got > addressed by Tony long ago...] > > Savas: > > It seems that we are in agreement, then, that we want the ability to: > > a) Request operations on individual jobs identified by some sort of > "jobid" > > b) Request operations on sets of jobs identified by a user-supplied list > of "jobids" > > c) Request operations on sets of jobs identified by more abstract > criteria > > We also agree that (as I expressed in the email that started this > discussion) such requests can be expressed in a few different ways, with > somewhat different characteristics. > > That's progress I hope. > > Ian. > > At 02:44 PM 4/5/2005 +0100, Savas Parastatidis wrote: > > > > > Dear Ian, > > > > I dont think that the approach I proposed forces the user to do more > than they would have to do anyway if EPRs were used. It is still the > case that someone has to manage the EPRs to the resources in WSRF. This > is similar to what happens in the real world. The online bookstore will > ask for my credit card number (a URI), or the book store will as for an > ISBN (another URI) or multiple ISBNs if I want to buy multiple books. > The banking service will ask for my bank account number (another URI > perhaps). > > > > Also, there is no reason why a kill all my jobsmessage couldnt also be > supported. But please note that this message is now addressed to the > service (the container of resources) and not, as in the case of WSRF, to > a specific resource. This is no different from what I am advocating. > > > > Also& to Steves point about partial failure. If one wishes atomic > transaction semantics, I dont see the difference from the two > approaches& > > > > Atomic > > Msg -> resource 1 > > Msg -> resource 2 > > Msg -> resource 3 > > End Atomic > > > > Vs > > > > Msg > > Atomic > > Resource 1 > > Resource 2 > > Resource 3 > > End Atomic > > > > In fact, I would argue that the latter is better because: > > > > 1. It uses fewer messages (and, Steve, I am not assuming only HTTP and > the optimisations that may be supported) > > > > 2. I can more easily deal with the failures in an application > specific-manner since my atomic TX semantics do not span multiple msgs. > > > > (Anyway& who wants to do atomic TXs over the Web anyway? :-) > > > > Regards, > > -- > Savas Parastatidis > http://savas.parastatidis.name > > > > > From: Ian Foster [mailto:foster@mcs.anl.gov] > Sent: Tuesday, April 05, 2005 2:22 PM > To: Steve Loughran; Savas Parastatidis > Cc: Mark McKeown; Karl Czajkowski; Dennis Gannon; Samuel Meder; ogsa-wg; > dave.pearson(a)oracle.com; gray(a)microsoft.com; humphrey(a)cs.virginia.edu; > grimshaw(a)virginia.edu; aherbert(a)microsoft.com; gcf(a)indiana.edu; > mark.linesch(a)hp.com; Frank Siebenlist; Tony Hey; Dave Berry > Subject: Re: [ogsa-wg] RE: Modeling State: Technical Questions > > > > Steve's note raises a key point for me: do we really want to force the > user (as Savas seems to be advocating) to keep track of jobs running at > a remote site? > > I'd rather send a request "kill all my jobs" or "kill all my jobs that > have run for more than a day" to the factory than carefully keep track > of all jobs that I have active, and how long they have been running, so > that I can send the big document (or stream) discussed below. > > Ian. > > > At 02:10 PM 4/5/2005 +0100, Steve Loughran wrote: > > Savas Parastatidis wrote: > > Dear all, > I think something needs to be clarified with regards to handling > multiple jobs with one message. The beauty of document-oriented > interactions is that you can do things like... > <job-details-request> > <job-id>urn:ogsa:job:guid:bla-bla-bla-001</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-010</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-029</job-id> > </job-details-request> > Or > <job-suspend-request> > <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-005</job-id> > <job-id>urn:ogsa:job:guid:bla-bla-bla-008</job-id> > </job-suspend-request> > The schema for the above document can allow anything from 0 to N number > of <job-id> elements. > > > the trouble with any bulk operation is you have to handle partial > failure. You need either atomic operations (not long lived transactions > over HTTP Savas, I wouldn't be that daft), or a way of indicating that > only a bit went wrong > > Hence the 207 Multi-Status response in WebDav, the "something failed, > look in the message". WebDav is still single instance (here a RESTy > URL), but you can set >1 property and so have partial failure. > > SOAP just has SOAPFault and extensions; no explicit multiple failure > response. WS-RF-ResourceProperties has a similar problem with > SetResourceProperties, but a different failure model in which any > failure to set can result in a WS-BaseFault, indicating which failed, > but providing no apparent information on which worked. > > It seems to me that if you want to bulk stuff, you do need ways of (a) > handling partial failure and (b) declaring what happens on partial > failure. For the curions, WebDav's failure mode on file operations > (MOVE, COPY) is explicitly declared to be that of failed file operations > of Win98 on a FAT32 filesystem [1,2] > > Alternatively, you dont go for bulk operations, neither on a multiple > jobs, or on multiple properties of a job (remember, WS-RF doesn't > declare atomic/transacted property operations, so all you do here is > increase the window of instability, a window that already exists). > Instead you just stream a series of operations over the same HTTP1.1 > connection -assuming that everything is accessible at the same far-end > host, and get a series of (potentially out of order, we are talking > HTTP1.1) responses. > > This could be efficient, and you could do better handling of failure. > But you do need a SOAP stack that can keep an HTTP1.1 channel open for > multiple requests. Axis doesnt, even if you get httpclient to do the > HTTP work; I don't know about .NET/WSE. You also need developers to > model the communication correctly. Manipulating JAXRPC proxies as if > they represent remote objects is *clearly* the wrong way to do it. You'd > almost want to model a queue of requests waiting to be POSTed, a queue > you can fill up then push out. Something like this, in your Java-era > language of choice :- > > //different queues for SOAP, REST > Queue q=new Soap12RequestQueue(); > > q.add(new StatePut(job1.uri,Job.LIFECYCLE,Job.SUSPENDED)); > //let the queue reorder stuff if it wants to > q.add(new > StatePut(job2.uri,Job.LIFECYCLE,Job.SUSPENDED),Queue.POSITION_OPTIMAL); > q.add(new > StatePut(job3.uri,Job.LIFECYCLE,Job.SUSPENDED),Queue.POSITION_LAST); > > q.setEventHandler(this); > q.nonBlockingSubmit(); > > No, there is no code behind this example, and I am avoiding any hints as > to what the even handler would look like. I think the key point is that > once you embrace remote operations as async actions, then you can model > the manipulations differently. Note also that I am representing job > suspension not as an explicit suspend() operation, but as a request to > put a job into the suspended state. This API could work with our friend > REST just as easily as with WS-RF... > > Anyway Savas, to conclude: do you have any evidence that a single > document is suboptimal compared to a sequences of requests over an open > HTTP/1.1 connection? That is, assuming we ignore the SHOULD in the > HTTP1.1 specification " Clients SHOULD NOT pipeline requests using > non-idempotent methods or non-idempotent sequences of methods" [3] > > -Steve > > > [1] WebDav http://www.ietf.org/rfc/rfc2518.txt S8.9.2 > > "after encountering an error moving a non-collection > resource as part of an infinite depth move, the server SHOULD try to > finish as much of the original move operation as possible." > > [2] > http://lists.w3.org/Archives/Public/w3c-dist-auth/1997JulSep/0177.html > > [3] RFC2616 HTTP1.1 > > _______________________________________________________________ > Ian Foster www.mcs.anl.gov/~foster > Math & Computer Science Div. Dept of Computer Science > Argonne National Laboratory The University of Chicago > Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. > Tel: 630 252 4619 Fax: 630 252 1997 > Globus Alliance, www.globus.org <http://www.globus.org/> > > _______________________________________________________________ > Ian Foster www.mcs.anl.gov/~foster > Math & Computer Science Div. Dept of Computer Science > Argonne National Laboratory The University of Chicago > Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. > Tel: 630 252 4619 Fax: 630 252 1997 > Globus Alliance, www.globus.org <http://www.globus.org/> > >

7 6

ByteIO charter: ready for GFSG?
by Dave Berry 07 Apr '05

07 Apr '05

Dear all, I'd like, if possible, to send the draft charter for the OGSA ByteIO group to the GFSG before their next meeting on Tuesday. See the attachment for a version that I've updated in the light of comments received. Please send any comments to the co-chairs and me by Sunday. If there are no unresolved issues I'll submit it to the GFSG on Monday, for discussion at their meeting on Tuesday. They will then make it open for comment for another 7 days. If there are unresolved issues, we can discuss them on next Tuesday's telcon (details to follow). Best wishes, Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

New tracker (for Profile template document)
by Andreas Savva 07 Apr '05

07 Apr '05

>From the Apr 6 minutes: > Action: Andreas to create a tracker for the template document I have created a new tracker for issues relating to the OGSA profile template document.(http://tinyurl.com/5fxfd/draft-ggf-ogsa-profile-template/en/) The new tracker is called "Profile template" https://forge.gridforum.org/tracker/index.php?func=browse&group_id=42&atid=… and is set to send mail to the list on new artifact submissions only. The status fields are set up in the same way as for the Basic Profile Tracker. -- Andreas Savva Fujitsu Laboratories Ltd

1 0

DRAFT OGSA Evolution Policy Statement (call for review)
by Hiro Kishimoto 07 Apr '05

07 Apr '05

Hi all, Since the OGSA is GGF's flagship architecture, the GFSG has been working on a policy surrounding the creation of profiles for OGSA. David Snelling, our Area Director, has done a FIRST DRAFT ("The OGSA Evolution Policy Statement"). At the OGSA call today, attendants reviewed and discussed on this first draft (see meeting minutes also) and agreed to send it out to the mailing list to open discussion. Please read this draft and give your feedback. David and I will bring your feedback to GFSG to improve the policy statement. Thanks, ---- Hiro Kishimoto

1 0

Teleconference minutes - 6 April 2005
by Andreas Savva 07 Apr '05

07 Apr '05

Minutes attached. Gridforge: https://forge.gridforum.org/projects/ogsa-wg/document/minutes-20050406/en/1 aka http://tinyurl.com/5fxfd/minutes-20050406/en/1 -- Andreas Savva Fujitsu Laboratories Ltd OGSA Teleconference - 6 April 2005 ================================== * Participants Mike Behrens (R2AD, LLC) Dave Berry (NeSC) Ian Foster (ANL) Soonwook Hwang (NII) Hiro Kishimoto (Fujitsu) Fred Maciel (Hitachi) Tom Maguire (IBM) Mark Morgan (UVa) Takuya Mori (NEC, ANL) Kazushige Saga (NII) Andreas Savva (Fujitsu) Dave Snelling (Fujitsu) Ravi Subramaniam (Intel) Jem Treadwell (HP) Peter Ziu (Northrop Grumman) Minutes: Andreas Savva * Teleconference minutes April 4 Minutes approved with changes Corrections: (Mark) - May F2F: EMS was confirmed for Monday - Allen's name is mispelled * Byte IO charter During an earlier review it was proposed that since the OGSA Basic Profile 1.0 is not yet published the charter should mention that the WG will start work with a draft version of the Profile and move to the final version later on. To minimize synchronization work it was suggested to pick a draft version to work from and only when the WG's specification gets close to publication synchronize with a later or final version of the BP. There is no BoF ML set up yet. So it was agreed to mail a final draft charter to the ogsa-wg-ml and other interested parties and ask for comments. There is no problem with not setting up a call specifically to discuss the draft charter. Once that is done, incorporate any feedback and submit a final version of the charter to the AD. Since the GFSG call is on Tuesday it is important to get this to the AD fairly soon. (It was also suggested that important emails from the Byte IO WG should also be sent to ogsa-wg-ml as part of liaison work. Obviously there is no need to cross-post everything.) * OGSA evolution policy statement This is a new document written by Dave S for the GGF (GFSG). "Drawer statement": A (M.Linesch) term describing documents that may not be completely public (but not secret) and are used for reference when needed (kept in a drawer). The intention is, however, to send this document out to the ogsa-wg-ml for more comments. OGSA is high profile especially after GGF's adoption of OGSA as a "flagship architecture." There are some concerns raised with OGSA adopting one specific approach. This document tries address those concerns based on the consensus of the December OGSA F2F. Three levels of interoperability - At infrastructure layer (based on WS standards) - (OGSA version 1.0 (aka GFD.30) definition of infrastructure also includes WSRF and WSDM. This definition is inconsistent with the definition in this document. Needs clarification.) - At the highest level based on GFD.30 - (From a technical perspective it is not really meaningful to talk about interoperability at this level) - At the middle level based on profile definitions - Consistency between GFD.30 and profiles is mentioned but is this a vacuous statement at the moment? - Consistency is a work in progress. - There is a plan/desire to produce more detailed architectural documents per capability - Concerns that the way this document is written any profile can be ogsa so resulting in a tower of babel. - Is it possible that profiles can be inconsistent with each other? - Inconclusive discussion on the consensus of the Dec or Feb meeting. - From a technical point of view it is not meaningful to allow inconsistent profiles (so this statement seems to be politically motivated) - Technical vs branding issues - Who controls the OGSA name/brand and who can state that a profile is part of ogsa - If the OGSA 'brand' is important then it should not be 'diluted' by allowing anything to be OGSA. - E.g., is it possible to issue BP and then state that all other profiles should be consistent/build on top of this profile? - And a stronger statement could be that the higher profiles are unique for individual GFD.30 features/components - Might also need to work from top down to articulate the architecture further (something that could be done later) - Is there such a thing as a middle level profile (?architectural profile?) and what does it mean for interoperability - Profiles are concrete and talk about specific conformance targets so such a profile is not possible - Even though potentially this document allows for a tower of babel situation logistically it would be difficult to find the people to define the profiles/specifications currently being worked on. (So it seems like a small risk.) - Proposal for defining a process/policy by which only certain documents can use the OGSA name: - At the moment once a WG is formed it is accountable to the GFSG only; there is no formal mechanism for one WG to affect another except by individual member participation. - So some extra layer to allow OGSA-WG to 'control' such documents from other WGs would be needed. - The current process allows for feedback to documents during the public comment period and that could be the place to raise objections to inappropriate usage of the OGSA term. Action: - Send the document out to the list and continue this discussion there. (Hiro) - Dave S will feedback this discussion back to the GFSG * OGSA Profile definition (profile template) review - Existing issue with copyright affected text (text copied from WS-I basic profile): - Hiro has confirmed that it is not allowed to copy-paste this text. Even if GGF becomes a WS-I associate member, membership only provides early access to WS-I materials. Action: Delete the copyrighted text from the OGSA basic profile and template documents. (Dave S and Tom) - Reference the WS-I document instead - The status and adoption levels are the main contributions of the template document: - It is expected that profiles categorize specs used in this way - Level of adoption: for a profile it affects the transition from proposed recommendation to recommendation. Action: Make this kind of information for each referenced spec a required appendix for profiles and include an example in this document. (Dave S) - "Security considerations" is typically the last section in GGF documents. Action: Fix order of the "Profile type distinction" section and the "Security considerations" section. (Dave S) - Section 5.1: Should the term "informational profile" be changed to the more commonly used "candidate profile" - Dave S prefers "informational" but maybe "candidate" is already well established. No change for now. - Section 5.3: - It is incorrect to include "evolving standard" as an allowed status here. - "Published recommendation" should be "Grid recommendation" Action: Dave S to update the document with the above corrections - Section 5.2: - It was proposed to also delete 'Evolving standard' from this section. - A proposed recommendation becomes a recommendation after implementation and an experience report so the standards it is based on should not be 'evolving'. - The downside is that a lot of referred work may be in progress making this requirement too restrictive - No decision reached. Need to revisit in order to clarify which 'milestones' are being assumed. E.g., is it acceptable to use evolving specs up to submission but make sure they are updated before submission or not. - The adoption levels of section 5.2 and 5.3 should probably be the other way round. Dave S to correct. - Agreed that this document should be submitted as an information document by GGF14 Action: Andreas to create a tracker for the template document * AOB - Next call (Monday) is new WG status call - It is not clear if there is a Naming BoF list. - A BES BoF list seems to be setup (OGSA-BES-bof(a)ggf.org)

1 0

Re: [ogsa-wg] RE: Modeling State: Technical Questions
by Steve Loughran 06 Apr '05

06 Apr '05

I have this theory, it's #2 of my "silly but possibly valid theories":- Any technology other than an RDBMS that implements a SELECT statement is inherently over-complex I would cite SLPv2 and that EJB query language [1] as two examples to back up my claim, with UDDI lurking nearby. I also note that it is badly escaped SQL select statements that are a primary attack point into custom HTTP apps, and even more often, custom SOAP endpoints [2]. There is something about the "let's do a SELECT" operation that gets people excited, but you soon end up with something that is hard to implement, test and interop test against, or you have to hand it off to a real DB underneath. Yes, bits of XPath fall in to this category too. If we are going to have select statements, at least be rigorous about requiring quotes around strings, for security reasons: “where (jobid = 'urn:guid:364') or (jobid = 'urn:guid:401')” -steve [1] http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/EJBQL5.html [2] http://ws.apache.org/axis/java/security.html Paul Watson wrote: > > > Ian, > > > > I agree that this is good progress. So let’s bank that and see if we can > we can agree on one more thing, and then I’ll ask a question. > > > > Considering your list of abilities (a, b & c) below, do we agree that in > terms of expressiveness, the ordering is: > > > > c>b>a > > > > i.e. using approach c, a client can request operations on: > > a) single jobs: “where (jobid = urn:guid:364)” > > b) sets of jobs: “where (jobid = urn:guid:364) or (jobid = urn:guid:401)” > > > > If there is agreement on this, then we could move on to discussing why > it is felt necessary to provide more than just c for the job submission > service. > > > > Regards > > Paul > > > > Ian wrote… > >>Savas: > >> > >>It seems that we are in agreement, then, that we want the ability to: > >> > >>a) Request operations on individual jobs identified by some sort of "jobid" > >> > >>b) Request operations on sets of jobs identified by a user-supplied list > of "jobids" > >> > >>c) Request operations on sets of jobs identified by more abstract criteria > >> > >>We also agree that (as I expressed in the email that started this > discussion) such >requests can be expressed in a few different ways, > with somewhat different >characteristics. > >> > >>That's progress I hope. > >> > >>Ian. > > > > * From: * Ian Foster [mailto:foster@mcs.anl.gov] > *Sent:* 05 April 2005 17:59 > *To:* Savas Parastatidis; Steve Loughran > *Cc:* Mark McKeown; Karl Czajkowski; Dennis Gannon; Samuel Meder; > ogsa-wg; dave.pearson(a)oracle.com; gray(a)microsoft.com; > humphrey(a)cs.virginia.edu; grimshaw(a)virginia.edu; aherbert(a)microsoft.com; > gcf(a)indiana.edu ; mark.linesch(a)hp.com; Frank Siebenlist; Tony Hey; Dave > Berry; Paul Watson > *Subject:* RE: [ogsa-wg] RE: Modeling State : Technical Questions > > > > [I'm feeling increasingly bad about sending email to all of the people > CCed here, who may not be interested in these issues at all but got > addressed by Tony long ago...] > > Savas: > > It seems that we are in agreement, then, that we want the ability to: > > a) Request operations on individual jobs identified by some sort of "jobid" > > b) Request operations on sets of jobs identified by a user-supplied list > of "jobids" > > c) Request operations on sets of jobs identified by more abstract criteria

2 1

London F2F for data, May 25th - who's coming
by Dave Berry 06 Apr '05

06 Apr '05

All, We are holding a F2F meeting for OGSA data on May 25th, following the main OGSA WG F2F on May 23rd-24th. Most of the day will be on the Data Architecture document - we plan to have a significant revision ready at least a week beforehand. There will also be a session on ByteIO as well, either as a parallel session or for part of the time. Please can you let me know if you can attend this day? It would be great to have some of the main OGSA WG people there as well. Best wishes, Dave Berry Research Manager, National e-Science Centre 15 South College St., Edinburgh Tel: +44 131 6514039

1 0

[ogsa-wg | Basic Profile - 1355] Basic Discovery Properties
by Sourceforge Tracker Monitor 06 Apr '05

06 Apr '05

A new artifact has been added: 1355 Submitted by: David Snelling Submitted date: 2005-04-06 10:39:21 Respond by visiting: https://forge.gridforum.org/tracker/?func=detail&atid=780&aid=1355&group_id… (https://forge.gridforum.org/tracker/?func=detail&atid=780&aid=1355&group_id…) Summary: Basic Discovery Properties Project: Open Grid Services Architecture Tracker: Basic Profile Artifact ID: 1355 Category: <None> Group: <None> Status: Open Priority: - Last Modified By: David Snelling Last Modified: 2005-04-06 10:39:21 Submitted By: David Snelling Submit Date: 2005-04-06 10:39:21 Assigned To: &lt;None&gt; File(s): <None> Description: In addition to the ResourcePropertyNames RP, we should also require other basic discovery mechanisms. FinalResourceInterface - QName of the most derived portType of the resource 0..1 ResourceInterface - QName of a portTypes mixed into the FinalResourceInterface 0..n ResourceEndPointReference - An EPR to this Resource 0..n View the Basic Profile : https://forge.gridforum.org/tracker/index.php?func=browse&group_id=42&atid=… (https://forge.gridforum.org/tracker/index.php?func=browse&group_id=42&atid=…)

1 0

Minutes of April 4th Telecon
by Hiro Kishimoto 06 Apr '05

06 Apr '05

Hi all, As you may already know, Mark has uploaded his excellent minutes on the GridForge. Have a look and approve it at the call tomorrow. https://forge.gridforum.org/projects/ogsa-wg/document/minutes-20050404/en/1 Many thanks Mark! ---- Hiro Kishimoto

1 0

RE: [ogsa-wg] RE: Modeling State: Technical Questions
by Hiro Kishimoto 06 Apr '05

06 Apr '05

Paul's email bounced. ---- Hiro Kishimoto Subject: RE: [ogsa-wg] RE: Modeling State: Technical Questions Date: Tue, 5 Apr 2005 20:20:09 +0100 From: "Paul Watson" <Paul.Watson(a)newcastle.ac.uk> To: "Ian Foster" <foster(a)mcs.anl.gov>, "Savas Parastatidis" <Savas.Parastatidis(a)newcastle.ac.uk>, "Steve Loughran" <steve_loughran(a)hpl.hp.com> Ian, I agree that this is good progress. So let's bank that and see if we can we can agree on one more thing, and then I'll ask a question. Considering your list of abilities (a, b & c) below, do we agree that in terms of expressiveness, the ordering is: c>b>a i.e. using approach c, a client can request operations on: a) single jobs: "where (jobid =3D urn:guid:364)" b) sets of jobs: "where (jobid =3D urn:guid:364) or (jobid =3D urn:guid:401)" If there is agreement on this, then we could move on to discussing why it is felt necessary to provide more than just c for the job submission service. Regards Paul Ian wrote... >Savas: > >It seems that we are in agreement, then, that we want the ability to: > >a) Request operations on individual jobs identified by some sort of "jobid" > >b) Request operations on sets of jobs identified by a user-supplied list of "jobids" > >c) Request operations on sets of jobs identified by more abstract criteria > >We also agree that (as I expressed in the email that started this discussion) such >requests can be expressed in a few different ways, with somewhat different >characteristics. > >That's progress I hope. > >Ian. ________________________________ From: Ian Foster [mailto:foster@mcs.anl.gov] Sent: 05 April 2005 17:59 To: Savas Parastatidis; Steve Loughran Cc: Mark McKeown; Karl Czajkowski; Dennis Gannon; Samuel Meder; ogsa-wg; dave.pearson(a)oracle.com; gray(a)microsoft.com; humphrey(a)cs.virginia.edu; grimshaw(a)virginia.edu; aherbert(a)microsoft.com; gcf(a)indiana.edu; mark.linesch@hp. com; Frank Siebenlist; Tony Hey; Dave Berry; Paul Watson Subject: RE: [ogsa-wg] RE: Modeling State: Technical Questions [I'm feeling increasingly bad about sending email to all of the people CCed here, who may not be interested in these issues at all but got addressed by Tony long ago...] Savas: It seems that we are in agreement, then, that we want the ability to: a) Request operations on individual jobs identified by some sort of "jobid" b) Request operations on sets of jobs identified by a user-supplied list of "jobids" c) Request operations on sets of jobs identified by more abstract criteria We also agree that (as I expressed in the email that started this discussion) such requests can be expressed in a few different ways, with somewhat different characteristics. That's progress I hope. Ian. At 02:44 PM 4/5/2005 +0100, Savas Parastatidis wrote: Dear Ian, I dont think that the approach I proposed forces the user to do more than they would have to do anyway if EPRs were used. It is still the case that someone has to manage the EPRs to the resources in WSRF. This is similar to what happens in the real world. The online bookstore will ask for my credit card number (a URI), or the book store will as for an ISBN (another URI) or multiple ISBNs if I want to buy multiple books. The banking service will ask for my bank account number (another URI perhaps). Also, there is no reason why a kill all my jobsmessage couldnt also be supported. But please note that this message is now addressed to the service (the container of resources) and not, as in the case of WSRF, to a specific resource. This is no different from what I am advocating. Also& to Steves point about partial failure. If one wishes atomic transaction semantics, I dont see the difference from the two approaches& Atomic Msg -> resource 1 Msg -> resource 2 Msg -> resource 3 End Atomic Vs Msg Atomic Resource 1 Resource 2 Resource 3 End Atomic In fact, I would argue that the latter is better because: 1. It uses fewer messages (and, Steve, I am not assuming only HTTP and the optimisations that may be supported) 2. I can more easily deal with the failures in an application specific-manner since my atomic TX semantics do not span multiple msgs. (Anyway& who wants to do atomic TXs over the Web anyway? :-) Regards, -- Savas Parastatidis http://savas.parastatidis.name From: Ian Foster [mailto:foster@mcs.anl.gov] Sent: Tuesday, April 05, 2005 2:22 PM To: Steve Loughran; Savas Parastatidis Cc: Mark McKeown; Karl Czajkowski; Dennis Gannon; Samuel Meder; ogsa-wg; dave.pearson(a)oracle.com; gray(a)microsoft.com; humphrey(a)cs.virginia.edu; grimshaw(a)virginia.edu; aherbert(a)microsoft.com; gcf(a)indiana.edu; mark.linesch@hp. com; Frank Siebenlist; Tony Hey; Dave Berry Subject: Re: [ogsa-wg] RE: Modeling State: Technical Questions Steve's note raises a key point for me: do we really want to force the user (as Savas seems to be advocating) to keep track of jobs running at a remote site? I'd rather send a request "kill all my jobs" or "kill all my jobs that have run for more than a day" to the factory than carefully keep track of all jobs that I have active, and how long they have been running, so that I can send the big document (or stream) discussed below. Ian. At 02:10 PM 4/5/2005 +0100, Steve Loughran wrote: Savas Parastatidis wrote: Dear all, I think something needs to be clarified with regards to handling multiple jobs with one message. The beauty of document-oriented interactions is that you can do things like... <job-details-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-001</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-010</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-029</job-id> </job-details-request> Or <job-suspend-request> <job-id>urn:ogsa:job:guid:bla-bla-bla-002</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-005</job-id> <job-id>urn:ogsa:job:guid:bla-bla-bla-008</job-id> </job-suspend-request> The schema for the above document can allow anything from 0 to N number of <job-id> elements. the trouble with any bulk operation is you have to handle partial failure. You need either atomic operations (not long lived transactions over HTTP Savas, I wouldn't be that daft), or a way of indicating that only a bit went wrong Hence the 207 Multi-Status response in WebDav, the "something failed, look in the message". WebDav is still single instance (here a RESTy URL), but you can set >1 property and so have partial failure. SOAP just has SOAPFault and extensions; no explicit multiple failure response. WS-RF-ResourceProperties has a similar problem with SetResourceProperties, but a different failure model in which any failure to set can result in a WS-BaseFault, indicating which failed, but providing no apparent information on which worked. It seems to me that if you want to bulk stuff, you do need ways of (a) handling partial failure and (b) declaring what happens on partial failure. For the curions, WebDav's failure mode on file operations (MOVE, COPY) is explicitly declared to be that of failed file operations of Win98 on a FAT32 filesystem [1,2] Alternatively, you dont go for bulk operations, neither on a multiple jobs, or on multiple properties of a job (remember, WS-RF doesn't declare atomic/transacted property operations, so all you do here is increase the window of instability, a window that already exists). Instead you just stream a series of operations over the same HTTP1.1 connection -assuming that everything is accessible at the same far-end host, and get a series of (potentially out of order, we are talking HTTP1.1) responses. This could be efficient, and you could do better handling of failure. But you do need a SOAP stack that can keep an HTTP1.1 channel open for multiple requests. Axis doesnt, even if you get httpclient to do the HTTP work; I don't know about .NET/WSE. You also need developers to model the communication correctly. Manipulating JAXRPC proxies as if they represent remote objects is *clearly* the wrong way to do it. You'd almost want to model a queue of requests waiting to be POSTed, a queue you can fill up then push out. Something like this, in your Java-era language of choice :- //different queues for SOAP, REST Queue q=3Dnew Soap12RequestQueue(); q.add(new StatePut(job1.uri,Job.LIFECYCLE,Job.SUSPENDED)); //let the queue reorder stuff if it wants to q.add(new StatePut(job2.uri,Job.LIFECYCLE,Job.SUSPENDED),Queue.POSITION_OPTIMAL); q.add(new StatePut(job3.uri,Job.LIFECYCLE,Job.SUSPENDED),Queue.POSITION_LAST); q.setEventHandler(this); q.nonBlockingSubmit(); No, there is no code behind this example, and I am avoiding any hints as to what the even handler would look like. I think the key point is that once you embrace remote operations as async actions, then you can model the manipulations differently. Note also that I am representing job suspension not as an explicit suspend() operation, but as a request to put a job into the suspended state. This API could work with our friend REST just as easily as with WS-RF... Anyway Savas, to conclude: do you have any evidence that a single document is suboptimal compared to a sequences of requests over an open HTTP/1.1 connection? That is, assuming we ignore the SHOULD in the HTTP1.1 specification " Clients SHOULD NOT pipeline requests using non-idempotent methods or non-idempotent sequences of methods" [3] -Steve [1] WebDav http://www.ietf.org/rfc/rfc2518.txt S8.9.2 "after encountering an error moving a non-collection resource as part of an infinite depth move, the server SHOULD try to finish as much of the original move operation as possible." [2] http://lists.w3.org/Archives/Public/w3c-dist-auth/1997JulSep/0177.html [3] RFC2616 HTTP1.1 _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/> _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org <http://www.globus.org/>

1 0