January 2006 - saga-rg - lists.ogf.org

Queries and minor points
by Graeme Pound 23 Feb '06

23 Feb '06

Please find below a couple of queries, and diverse minor points, which arose during my preparation of a Java implementation of the API. I anticipate that my comments on the semantics of the API will become a little more substantive in due course. Thanks, Graeme ######### Queries - I am little unclear of the purpose and operation of Attribute.getVectorAttribute() and Attribute.setVectorAttribute(). What is returned if a vector attribute is queried with getAttribute()? Why are a vector of values associated with a key required? - contextType does not appear to be very extensible, how would different security models be specified? ######### Minor points - Exceptions are not described in the SIDL interface, however Exception types are listed in long description of the methods (for example ReadOnlyAttribute and NoSuchKey). This information should be in the SIDL description as it is fundamental to the interface. - capitalise ivec, openDirFlags, openFlags? - JobService.runJob() returns Job and stdin, stdout, stderr. However stdin, stdout, stderr are available from Job.getJobInfo()? Can runJob() simply return an object Job? - enum values are not specified by JobState - LogicalFile argument types not specified for addLocation, removeLocation, replicate. These should be 'string' - The enumerations openDirFlags and openFlags exist in the following packages SAGA.Namespace, SAGA.File and SAGA.LogicalFile (although SAGA.Namespace is imported by SAGA.File and SAGA.LogicalFile). Is this duplication required? - the interfaces Stream and StreamServer implements Attributes, however Attributes does not exist. Is this Attribute (perhaps Attributes is a better name for this class)? - File.readP() and File.writeP() define input argument of type 'pattern'. This type is not defined by the SIDL API, is 'pattern' an undefined class or primitive type? - in Stream void getContext (out context info); should read void getContext (out Context info); - strawman_error.txt: "All objects in SAGA implement ErrorHandler...", the SIDL for these objects should indicate that they implement the ErrorHandler interface. The following objects: Directory File Job JobService LogicalDirectory LogicalFile Multiplexer Stream StreamServer Task TaskContainer ... any others ?

4 5

Fwd (s.newhouse@omii.ac.uk): Re: SAGA - WG?
by Andre Merzky 03 Feb '06

03 Feb '06

Dear group, as you know, we are currently in transition from a GGF Research Group to a GGF Working Group, which will enable us to submit documents into the standardization track. The last action from our side was to submit the proposed WG charter to our Area Directors (Steven and Dieter), and wait for the last step in the process, the GFSG approval of that charter. Below you find the answer we got from the GFSG. I know people will have strong opinions about that, both positive and negative (well, certainly I do anyway :-P ), so we would like to discuss the GFSG answer on this list. It would be favourable to come to a group internal conclusion, and a solid opinion, about the groups future before GGF16 - that means within the next two weeks. Best regards, your friendly group chairs ;-) ----- Forwarded message from Steven Newhouse <s.newhouse(a)omii.ac.uk> ----- > Date: Mon, 23 Jan 2006 07:38:45 +0000 > From: Steven Newhouse <s.newhouse(a)omii.ac.uk> > To: Andre Merzky <andre(a)merzky.net>, Shantenu Jha <s.jha(a)ucl.ac.uk>, > Tom Goodale <goodale(a)cct.lsu.edu> > CC: Dieter Kranzlmueller <dk(a)gup.jku.at> > Subject: Re: SAGA - WG? > > Dear Andre, Shantenu & Tom, > > At the GFSG meeting last week, there was a general discussion as to how > GFSG should/could steer the standards areas to increase the impact of > GGF. One of the discussions related to the Applications area and how we > (as Area Directors) could help to structure the activity to align work > with activities in the Architecture (i.e. OGSA) area. > > There was considerable interest from the rest of the GFSG in the SAGA > activities and the potential uptake that the generation of stable > client-side interfaces (and potentially command line tools that build on > these interfaces) could provide. The GFSG saw SAGA-RG as an important > step forward for grids being adopted by the wider community. > > That's the good news! > > We mentioned the pending SAGA-WG charter and that this was the next step > to move things forward. Some concern was expressed about focus and broad > scope. Especially as other domains would like to bring forward their own > domains (data access, data movement, etc) for client side API > standardisation. > > One proposed solution to this is that SAGA-RG stays as it is. It is > doing very valuable work collecting use cases, developing the strawman > API that supports these use cases and discussing implementation issues > through real experience. However, clearly there are elements within the > strawman that are ready to move to the next level. > > It is proposed that these aspects should be developed as standalone WG's > starting with a common look and feel, and then picking up on (say) jobs > & file movement to drive some domain specific applications of the common > look and feel. The result would be an umbrella-RG (SAGA) with a set of > coupled WGs for the different aspects. > > So there are two ways forward - you have _our_ support which ever way > _you_ choose to go forward. > > If you go forward with then the current charter then you will need to be > explicit as to which areas you will be doing (to allow space for other > WG's to come forward), i.e. you need to define your API scope. Elements > of the API will change at different rates and putting this all into one > specification adds to its complexity. Small tightly focussed > specifications have had much greater success within GGF. This may be > something else to consider. > > As a conclusion we hope that you will think about this great opportunity > to take the responsibility for the bigger picture, and that you will > adapt your plans accordingly from this feedback. We would certainly be > available to support you in this quest. At the same time, it has also > been agreed to continue the regular bit-flipping procedure with your > charter, should you insist on your currently proposed approach. > > Steven & Dieter -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

6 7

Fwd (s.jha@ucl.ac.uk): terry sloan/malcolm
by Andre Merzky 31 Jan '06

31 Jan '06

Dear Terry, our excuses that responding to your mail took us so long - that does not at all mean that we don't find that interesting! :-) Actually, we are very excited that somebody is taking up parts of the SAGA API and is implementing is - we wish we'd see more of that :-D Anyway, nowadays most emails seem to start with excuses for delay *sigh* The zip file available at the pointer you prvided contains a design document. We read that, and do have a couple of questions. First of all, we are happy that the API you implemented stayed so close to the SAGA spec. Well, positively thinking as we are, we think that this reflects the validity of the API. OTOH, we don't know how painful that was for you ;-) Well, was it painful? Further questions: - Can we add your document to our informal list of UC, i.e. put it on our wiki for future references? That would probably imply GGF IP policy, which means public domain - however, Copyright is of course not affected. - Roctopus sound very interesting: is fair to say it is to unicore what CoG is to Globus? - What security mechanisms are you using on middleware layer? - would it be possibly for you to document the problems you had while implementing in detail? We figured some of them from the docs (see below) - are there others? - The docs mention the lack of resource discovery in SAGA - is that a huge problem for you? - changes from interface to class, e.g. NSDir: we figured out the same in the C++ implementastion: it is much more convenient to have NSDir etc as classes. - get_name and get_URL: We have fixed this I think, get_URL now exists - pg 18, add "doesAttributeExist" , "isAttributeReadOnly" Right, introspection for the attributes was missing, we added that. Thanks! - pg 18 mentions inconsistencies - could you elaborate on that, please? - pg 20, suggestion to clean-up is a good point of course. Its now on the TODO/Isuse list (not sure what that means though ;-) - pg 5 mentiosn your future OGSA activity - is there a timeline for that? Cheers, Andre & Shantenu. Quoting [Terry Sloan] (Nov 24 2005): > Date: Thu, 24 Nov 2005 14:29:08 +0000 > From: Terry Sloan <tms(a)epcc.ed.ac.uk> > To: Simple API for Grid Applications WG <saga-rg(a)ggf.org> > Subject: [saga-rg] Update on the DEISA SAGA-based tool > > Hi all, > > Just thought you might be interested to know the JRA7 team from the > DEISA project have just released the latest version of their command > line tool that is based on parts of NSDir and JobManagement. > > You can download the software > http://forge.nesc.ac.uk/projects/deisa-jra7/ under Latest file releases. > > Regards, > > Terry Sloan -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

2 2

Details for call 03/02/06
by Tom Goodale 30 Jan '06

30 Jan '06

Hi, here are the details for this week's call Date: Fri 3 February 2006 Time: 1500 Central European Time 1400 Greenwich Mean Time 0800 Central Time 0600 Pacific Time Number: 0844 888 8888 (UK) 01805 004 102 (Germany) +44 870 088 5706 (Others) Code: 808044 Proposed Agenda: Agenda Bashing Discussion of meeting with BES and Byte-IO and matters arising Status of charter and bitflipping, wg/rg Miscellaneous Issue list Cheers, Tom

2 1

Constraints for conference call
by Tom Goodale 30 Jan '06

30 Jan '06

Hi, so far I've received constraints from one person. Is anyone else at all interested in having one ? Tom

2 1

Next conference call
by Tom Goodale 25 Jan '06

25 Jan '06

Hi, over the last couple of months, the conference call has been a bit of a disaster, but we now have lots to discuss, especially as to whether we complete our recharting as a working group, or stay as a research group. Will it be possible for us to have a conference call next week ? Perhaps on Wednesday, Thursday or Friday ? If you are interested in attending, please send your constraints so we can work out a mutually convenient, or at least not too inconvenient time for the call. I will collate the constraints on Monday and post day, time, conference number, and proposed agenda then. We will definitely discuss the RG/WG issue, and I'd like to discuss the issues which came out of my meeting with the OGSA-BES and the Byte-IO people. It would probably be worthwhile discussing the requirements document as well, if we get time. Please send other agenda items. To start the ball rolling on times, I will be in Baton Rouge next week, so would rather not have a call between 0600 and 1400 GMT, but am prepared to have have one as late as 0700 or as early as 1200 GMT if really necessary. I don't know of any other constraints for those days yet. Next ? Cheers, Tom

2 1

BOUNCE saga-rg@ggf.org: Non-member submission from ["Rosa M. Badia" <rosab@ac.upc.edu>] (fwd)
by Shantenu Jha 25 Jan '06

25 Jan '06

Bounced message from Rosa below. Interesting perspective. Shantenu PS: My own $0.02 coming soon.... Date: Wed, 25 Jan 2006 18:23:11 +0100 From: "Rosa M. Badia" <rosab(a)ac.upc.edu> To: Craig Lee <craig(a)rushg.aero.org> Cc: Simple API for Grid Applications WG <saga-rg(a)ggf.org> Subject: Re: [saga-rg] Fwd (s.newhouse(a)omii.ac.uk) Re: SAGA - WG? Last November I acted as reviewer in a project, and they stated that they have not taken into account SAGA because it is a RG and there was very few (or no?) industry involved. They also claimed that as RG the results were coming to slowly. Becoming a WG can of course attract users from projects and industry, I do not know if it will attract more effort at the implementation level too. Just to give a use-case :-) Best regards, Rosa Craig Lee wrote: > > All, > > First of all, allow me to say that this is a good problem to have. > The fact that the GFSG is considering whether SAGA should > be one WG or an umbrella RG for multiple WGs means that > people view it as important. > > My personal opinion is that the SAGA-RG could continue as > this umbrella organization, but to follow through with the > original SAGA intent, an initial WG should be formed that is > focused on _S_AGA. To emphasize the _simple_ in > SAGA I keep coming back to the analogy of "the six calls > in MPI". Whether the current SAGA API is simple enough > is separate discussion. In any case, the API should be > (a) minimally complete, (b) easy for implementers to implement > and (c) easy for new grid users to use. > > As long as the umbrella SAGA-RG persists, it can consider > issues of enhancing the API in various functional areas, > e.g., data movement, and also establishing a common > look-and-feel across them all. WGs could be spun-off > as necessary. > > Just my 2 cents. > > --Craig > > At 10:13 AM 1/24/2006, Andre Merzky wrote: > >> Dear group, >> >> as you know, we are currently in transition from a GGF >> Research Group to a GGF Working Group, which will enable >> us to submit documents into the standardization track. >> The last action from our side was to submit the proposed >> WG charter to our Area Directors (Steven and Dieter), and >> wait for the last step in the process, the GFSG approval of >> that charter. >> >> Below you find the answer we got from the GFSG. I know >> people will have strong opinions about that, both positive >> and negative (well, certainly I do anyway :-P ), so we would >> like to discuss the GFSG answer on this list. It would be >> favourable to come to a group internal conclusion, and a >> solid opinion, about the groups future before GGF16 - that >> means within the next two weeks. >> >> Best regards, >> >> your friendly group chairs ;-) >> >> >> >> ----- Forwarded message from Steven Newhouse <s.newhouse(a)omii.ac.uk> >> ----- >> >> > Date: Mon, 23 Jan 2006 07:38:45 +0000 >> > From: Steven Newhouse <s.newhouse(a)omii.ac.uk> >> > To: Andre Merzky <andre(a)merzky.net>, Shantenu Jha <s.jha(a)ucl.ac.uk>, >> > Tom Goodale <goodale(a)cct.lsu.edu> >> > CC: Dieter Kranzlmueller <dk(a)gup.jku.at> >> > Subject: Re: SAGA - WG? >> > >> > Dear Andre, Shantenu & Tom, >> > >> > At the GFSG meeting last week, there was a general discussion as to >> how >> > GFSG should/could steer the standards areas to increase the impact of >> > GGF. One of the discussions related to the Applications area and >> how we >> > (as Area Directors) could help to structure the activity to align work >> > with activities in the Architecture (i.e. OGSA) area. >> > >> > There was considerable interest from the rest of the GFSG in the SAGA >> > activities and the potential uptake that the generation of stable >> > client-side interfaces (and potentially command line tools that >> build on >> > these interfaces) could provide. The GFSG saw SAGA-RG as an important >> > step forward for grids being adopted by the wider community. >> > >> > That's the good news! >> > >> > We mentioned the pending SAGA-WG charter and that this was the next >> step >> > to move things forward. Some concern was expressed about focus and >> broad >> > scope. Especially as other domains would like to bring forward >> their own >> > domains (data access, data movement, etc) for client side API >> > standardisation. >> > >> > One proposed solution to this is that SAGA-RG stays as it is. It is >> > doing very valuable work collecting use cases, developing the strawman >> > API that supports these use cases and discussing implementation issues >> > through real experience. However, clearly there are elements within >> the >> > strawman that are ready to move to the next level. >> > >> > It is proposed that these aspects should be developed as standalone >> WG's >> > starting with a common look and feel, and then picking up on (say) >> jobs >> > & file movement to drive some domain specific applications of the >> common >> > look and feel. The result would be an umbrella-RG (SAGA) with a set of >> > coupled WGs for the different aspects. >> > >> > So there are two ways forward - you have _our_ support which ever way >> > _you_ choose to go forward. >> > >> > If you go forward with then the current charter then you will need >> to be >> > explicit as to which areas you will be doing (to allow space for other >> > WG's to come forward), i.e. you need to define your API scope. >> Elements >> > of the API will change at different rates and putting this all into >> one >> > specification adds to its complexity. Small tightly focussed >> > specifications have had much greater success within GGF. This may be >> > something else to consider. >> > >> > As a conclusion we hope that you will think about this great >> opportunity >> > to take the responsibility for the bigger picture, and that you will >> > adapt your plans accordingly from this feedback. We would certainly be >> > available to support you in this quest. At the same time, it has also >> > been agreed to continue the regular bit-flipping procedure with your >> > charter, should you insist on your currently proposed approach. >> > >> > Steven & Dieter >> -- >> +-----------------------------------------------------------------+ >> | Andre Merzky | phon: +31 - 20 - 598 - 7759 | >> | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | >> | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | >> | De Boelelaan 1083a | www: http://www.merzky.net | >> | 1081 HV Amsterdam, Netherlands | | >> +-----------------------------------------------------------------+ > -- ------------------------------------------------------------------------ Rosa M. Badia Dept. Computer Architecture Universitat Politecnica Catalunya Campus Nord - Modul D6 c/Jordi Girona 1-3 E08034 - Barcelona http://personals.ac.upc.edu/rosab/ Tel. +34 - 934017191 Fax: +34 - 934017055 ------------------------------------------------------------------------

2 1

Notes from meeting with BES and Byte-IO
by Tom Goodale 20 Jan '06

20 Jan '06

Hi, I had just had a meeting with the BES and Byte-IO people to check that we are all vaguely in sync. The overall reception was postive, and there were a comments which came up when we were ging through things: Namespaces: getURL -> getCWD - as we are already talking about namespaces, and have the concept of a directory, URL was seen to be inappropriate here. I've just thought why didn't we just name it getPath ? Links aren't generic - could we move those off the generic namespace interface ? RNS-Lite has a unified namespace. (I guess this is similar to the GAT advert-service type namespace where any object could be stored.) The suggestion was that SAGA should have one, and then there would be a namespace method which got the opaque token (EPR for RNS-Lite) which could be passed into the constructor of a file, etc; with isX methods to tell what sort of object was being pointed to by the opaque token. - this has obvious advantages in having just one namespace about, but does break the complete seperation of underlying implementation that we have currently, plus also requiring at least one call to instantiate an object, as currently have open methods which act as the appropriate factory methods to return files, logical files, etc. Files: writeP and readP are not described in the document. Are they really distinct from the writeV and readV or just a subset of the functionality ? Jobs: Three more possible states were suggested Staging In Staging Out Exception starting job (BES has this) Contexts/sessions: There was a suggestion that perhaps OGSA needs to pass generic context data about, sparked by the observation that SAGA and other systems do have the concept of a 'context'; would it make sense for SAGA for us to have sessions (or maybe all objects) provide the attribute interface and allow users to associate arbitrary key value pairs with it ? Cheers, Tom

2 1

Fwd (hiro.kishimoto@jp.fujitsu.com): [gfsg] definition of "implement," "conformance," and "compliance."
by Andre Merzky 17 Jan '06

17 Jan '06

Hi, Hiros remarks about "implementation" "conformance," and "compliance" are quite interesting I think. We might well be able to use the same criteria for SAGA. However, these definitions are still under discussion... Cheers, Andre. ----- Forwarded message from Hiro Kishimoto <hiro.kishimoto(a)jp.fujitsu.com> ----- > Date: Wed, 18 Jan 2006 01:21:03 +0900 > From: Hiro Kishimoto <hiro.kishimoto(a)jp.fujitsu.com> > To: GFSG <gfsg(a)ggf.org> > Subject: [gfsg] definition of "implement," "conformance," and "compliance." > > Hi all, > > Per request during today's GFSG standard call, I will share > definition of "implement," "conformance," and "compliance." > > It is in the OGSA roadmap document (GFD.53) written by Dave Berry. > > 2.3 OGSA Branding > Given the requirement for consistency across OGSA specifications and > other documents, GGF defines in [OGSA-Related Naming Guidelines] the > criteria to be used in determining whether to brand entities such as > working groups and documents with an OGSA prefix. > The OGSA normative documents are expected to be implemented by multiple > open-source software (OSS) projects and commercial software vendors. > Authors of OGSA Software may claim one of three levels of agreement with > the OGSA normative documents: > > 1. A claim to implement a specification or profile is a statement of > ?best effort? to satisfy the requirements of the specification. There > are no test mechanisms to guarantee the correctness of the implementation. > > 2. A claim of conformance to a specification or profile is a statement > of intent to interoperate with other conformant implementations. Two or > more implementations may test their conformance by testing how well they > interoperate. A conformance claim mechanism as defined in the profile > should be used to communicate conformance. > > 3. A claim of compliance would imply acceptance by a set of OGSA > Compliance Tests. At the time of writing no such tests exist. > We may expect that OGSA compliance tests will be developed that define > the practical criteria that must be satisfied for a software system to > be called ?OGSA-compliant.? Until such a suite is available, claims of > OGSA compliance should not be made. -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

1 0

Next conference call
by Tom Goodale 05 Jan '06

05 Jan '06

Hi, so far there has been minimal feedback on dates and times for the next conference call. As a suggestion, how does 1400GMT on Wednesday the 11th work for everyone ? Cheers, Tom

3 2