October 2005 - saga-rg - lists.ogf.org

SAGA use case document: comments needed! (this week)
by Thilo Kielmann 13 Dec '05

13 Dec '05

Dear all, the SAGA-RG has its use case document up for public comment, ending oct 30. So far, there are 0 comments! :-( I hereby urge everybody to have a look and comment, even if the comment is just trivially in favour... Thanks for your help, Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

10 16

Fwd (cdekate@cct.lsu.edu): SAGA API comments
by Andre Merzky 06 Nov '05

06 Nov '05

Hi, below are Chirags comments to the saga strawman and his use case. I think the suggession to add an update_location method to logical files is a good one. It basically would be an atomic version of a remove_location add_location pair. Thanks Chirag! Cheers, Andre. ----- Forwarded message from Chirag Dekate <cdekate(a)cct.lsu.edu> ----- > Date: Fri, 28 Oct 2005 11:00:14 -0500 > From: Chirag Dekate <cdekate(a)cct.lsu.edu> > To: Hartmut Kaiser <Hartmut.Kaiser(a)gmail.com>, > Andre Merzky <andre(a)merzky.net> > Subject: SAGA API comments > Cc: Gabrielle Allen <gallen(a)cct.lsu.edu> > > In the Logical Files section, > > Consider a scenario where one logical file name (LFN) maps to > different physical file names (PFN). The API description in its > current form allows end user to create such LFN-PFN mappings. > However in cases where the LFN - PFN mappings needs to be updated > based on the current state / availability of the Physical Files, the > existing API will need to be extended. > > General comments regarding the Strawman API Document: > The Context API : To be able to seamlessly switch between different > authentication and authorization mechanisms would enable several > scenarios that would enable use of datasets and resources in practical > situations where environments are based on heterogeneous middleware > technologies. > > As an overall comment, once the strawman is finalized it would be > helpful if samples with ample documentation are made available. There > are several API constructs whose use would become clearer if sample > application strategies are presented. One way would be an idea bank > that acts as a repository for ideas on how the API would be used to > solve different usecases, moderated by SAGA steering committee. > > Chirag Dekate ----- End forwarded message ----- -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

1 1

Task model...
by Andre Merzky 04 Nov '05

04 Nov '05

Hi Group, as you have seen in the notes from last GGF, there was some discussion about the SAGA task model. This mail summarizes this discussion, and proposes some changes to the task model to accommodate the suggestions made in Boston. Sorry that this mail comes late, I pushed that off far too long... As a reminder, the current task model in action looks like this: --------------------------------------------------------------------- Example 1: saga::directory d ("/tmp/"); saga::directory::task_factory dtf = d.get_task_factory (); saga::task t = dtf.mkdir ("test/"); t.run (); t.wait (); --------------------------------------------------------------------- So, the minimal number of code lines needed for running a task is 4. Note that the asynchroneous mkdir method has the same signature as the synchroneous, but has a different return value. Compare that to the mechanism used in MPI or used in GridRPC: --------------------------------------------------------------------- Example 2: saga::directory d ("/tmp/"); saga::task t = d.mkdir_async ("test/"); t.wait (); --------------------------------------------------------------------- The number of lines is 2, and there is no explicit additional object (the task factory). Please note that the created task is automatically run on creation time. The discussion in Boston showed that people favor the similar mechanism of example 2. Also, SAGA has not a single use case which requires (or even motivates) the task_factories as explicit objects in the SAGA API. But, we do have use cases which motivate that tasks are NOT autimatically run. In particular the following scenario of bulk operations shows that: --------------------------------------------------------------------- Example 3: saga::file f ("/tmp/t.dat"); saga::file::task_factory dtf = d.get_task_factory (); saga::task_container tc; for ( int i = 0; i < 1000; i++ ) { saga::task t = ftf.read (i, 1, buffer[i]); tc.add_task (t); } tc.run (); tc.wait (); --------------------------------------------------------------------- As the tc.run starts all tasks at the same time, the implementation has the chance to optimize that, and to perform one remote operation instead of 1000 independend ones. With the model shown in example 2, this optimization would be impossible, and the bulk related use cases in SAGA would be difficult to implement (efficiently). In order to accommodate the explicit run() method on tasks, there are two possibilities (well, there are more, but two seem particularily obvious): --------------------------------------------------------------------- Example 4a: distinguish task from async call saga::directory d ("/tmp/"); saga::task t_1 = d.mkdir_async ("test/"); saga::task t_2 = d.mkdir_task ("test/"); t_2.run () t_1.wait (); t_2.wait (); Example 4b: flag to start task saga::directory d ("/tmp/"); saga::task t_1 = d.mkdir_async ("test/", TRUE); // TRUE optional saga::task t_2 = d.mkdir_async ("test/", FALSE); t_2.run () t_1.wait (); t_2.wait (); --------------------------------------------------------------------- Although the version 4b seems somewhat simplier, most people (including me) seem to oppose a seemingly artificial flag on all async calls to flag the runnnin of the task. Also, this would change the signature if compared to the synchroneous version of the method. Version 4a OTOH increases the number of methods. Then again, that might not be sooooo bad. The current task model doubles the number of methods as well. But as we describe the model in text, and do not actually add all signatures twice in the spec, the spec stays terse (for a very specific definition of terse, to be sure). So, version 4a would add only one paragraph to the spec :-) Also, 4a is strictly spoken just a marker to the sync methods, which makes them asynchroneous. In different language bindings, this could look very different, and convenient: --------------------------------------------------------------------- Example 4a: more versions in C++ 4a1: d.mkdir ("test/"); d.mkdir_sync ("test/"); // same saga::task t_1 = d.mkdir_async ("test/"); saga::task t_2 = d.mkdir_task ("test/"); 4a2: d.mkdir ("test/"); d.sync ::mkdir ("test/"); // same saga::task t_1 = d.async::mkdir ("test/"); saga::task t_2 = d.task ::mkdir ("test/"); 4a3: d.mkdir ("test/"); d.mkdir <sync> ("test/"); // same saga::task t_1 = d.mkdir <async> ("test/"); saga::task t_2 = d.mkdir <task> ("test/"); --------------------------------------------------------------------- All these versions are equivalent. I personally like 4a2 best, Thilo likes 4a1 (close to MPI), Hartmut favors 4a3 (nice to implement in C++). Well, here we are really: the proposal is following: P1) get rid of the task factory (there seems no need for it) P2) allow tasks which do run() on creation P3) allow tasks which do NOT run() on creation All both 4a and 4b deliver that, with only small changes in the API spec (but obviously larger changes in the language bindings). For 4a, the versions 4a1 to 4a3 show different posssible C++ language bindings. Questions: Q1) any comments to the Boston discussion? I hope I reflected that correctly... Q2) are there any drawbacks by introducing 4a or 4b, apart from the additional number of calls? Q3) Are the advantages (simplicity for most usages, powerfull for bulk operations and others) good enough to introduce 4a or 4b? Q4) You like 4a or 4b better? Q5) Any comments to 4a1, 4a2 or 4a3? (not part of the Strawman!) Please answer before November 10th: firstly, we would like to finish the task model soon, in order to get the Strawman stabilizwed before GGF16; secondly, we would like to come to a decision during the F2F meeting at SuperComputing in Seattle (November 12-18). Cheers, Andre. -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

7 20

Last official day to comment on GridCPR-WG drafts
by Derek Simmel 30 Oct '05

30 Oct '05

Folks, Something to do with the extra hour you have today thanks to the return to Standard Time here in the U.S. ;) Today is the last day of the current public comment period for the two GridCPR-WG draft documents, 'Use Cases for Gird Checkpoint and Recovery' and 'An Architecture for Grid Checkpoint Recovery (GridCPR) Services and a GridCPR Application Programming Interface.' You can get to these documents at http://gridcpr.psc.edu/GGF or at the GGF website at http://www.ggf.org/ggf_docs_public.htm Your input is important to both improve the drafts and to move the drafts forward for publication! We appreciate any and all comments or questions posted! You can post your comments via the interface provided at http://www.ggf.org/ggf_docs_public.htm or you can send them here to the list. If you have any difficulty posting your comments, please send them to either GridCPR-WG Chair, Derek Simmel <dsimmel(a)psc.edu> or Paul Stodghill <stodghil(a)cs.cornell.edu>, and one of us will make sure that your comments are posted. Many Thanks! - Derek -- Derek Simmel <dsimmel(a)psc.edu> Pittsburgh Supercomputing Center (412) 268-1035

1 0

RE: [saga-rg] SAGA use case document: comments needed! (this week)
by Rajic, Hrabri 25 Oct '05

25 Oct '05

I have Acrobat v7 and it works fine for me. I have seen the problems Tomasz described with earlier version, but only when documents were opened with a browser Acrobat plug-in. Saving it on the hard disk and later opening with Acrobat solves the problem. Cheers Hrabri >-----Original Message----- >From: owner-saga-rg(a)ggf.org [mailto:owner-saga-rg@ggf.org] On Behalf Of Jon >MacLaren >Sent: Monday, October 24, 2005 6:58 PM >To: Tomasz Haupt >Cc: 'Ed Seidel'; 'Thilo Kielmann'; apps-rg(a)ggf.org; saga-rg(a)ggf.org >Subject: Re: [saga-rg] SAGA use case document: comments needed! (this week) > >Hi Tomasz, > >I've just downloaded this from the link you gave, and it works fine >for me. > >Jon. > > >On Oct 24, 2005, at 4:29 PM, Tomasz Haupt wrote: > >> There is something wrong with the document posted on >> http://www.ggf.org/Public_Comment_Docs/Documents/Sep-2005/draft-ggf-saga - >use-cases.pdf >> >> Starting on page 37, it is blank all the way to page 199. >> Apparently the >> conversion to pdf failed. >> >> Tomasz >> >> -----Original Message----- >> From: owner-saga-rg(a)ggf.org [mailto:owner-saga-rg@ggf.org] On >> Behalf Of Ed >> Seidel >> Sent: Monday, October 24, 2005 8:30 AM >> To: Thilo Kielmann >> Cc: apps-rg(a)ggf.org; saga-rg(a)ggf.org >> Subject: Re: [saga-rg] SAGA use case document: comments needed! >> (this week) >> >> I'd like to encourage people to take this seriously and spend a >> little time on it. In my opinion, it is among the very important >> activities going on in GGF. SAGA specifically aims to support the >> developers of scientific applications, so it is very important to get >> as much feedback as possible from apps groups. We hope to develop >> APIs that are so compelling, that they will be available to and >> adopted by application developers everywhere. As SAGA has now turned >> into a Working Group, it is now developing what should turn out to be >> a standard. >> >> Please take some time to look at this, and give feedback on use cases >> so we can make this work well. >> >> Best, >> Ed >> >> >> On Oct 24, 2005, at 6:26 AM, Thilo Kielmann wrote: >> >> >>> Dear all, >>> >>> the SAGA-RG has its use case document up for public comment, ending >>> oct 30. >>> >>> So far, there are 0 comments! :-( >>> >>> I hereby urge everybody to have a look and comment, even if the >>> comment is just >>> trivially in favour... >>> >>> >>> Thanks for your help, >>> >>> >>> Thilo >>> -- >>> Thilo Kielmann http://www.cs.vu.nl/ >>> ~kielmann/ >>> >>> >>> >> >> >>

1 0

Please comment on GridCPR documents
by Paul Stodghill 21 Oct '05

21 Oct '05

There are two GridCPR documents that are in their public comment period until Oct 30, 2005. In order for them to continue through the finalization process, it is critical that we have comments on them. Please take a moment to look at these documents and submit comments. Even if you do not have any suggested changes, please submit a comment with your general impressions of the documents. Here are the relevant links, "Use Cases for Grid Checkpoint and Recovery", * Draft - http://www.ggf.org/Public_Comment_Docs/Documents/Sep-2005/draft-ggf-gridcpr… * View and make comments - https://forge.gridforum.org/forum/forum.php?forum_id=565 "An Architecture for Grid Checkpoint and Recovery (GridCPR) Services and a GridCPR Application Programming Interface", * Draft - http://www.ggf.org/Public_Comment_Docs/Documents/Sep-2005/draft-ggf-gridcpr… * View and make comments - https://forge.gridforum.org/forum/forum.php?forum_id=566 Thank you.

2 1

Details for call 20/10/05
by Tom Goodale 20 Oct '05

20 Oct '05

Hi, Sorry for the late notice. Today's details are below. NOTE: this week the regular number was completely booked, so we have a new number; it should be back to the regular number next time. Date: Thu 20 October 2005 Time: 1700 Central European Summer Time 1600 British Summer Time 1000 Central Summer Time 0800 Pacific Summer Time Number: +1-620-782-2200 Code: 808044 Agenda: 1. Agenda bashing 2. Questions about the GGF meeting notes? 3. Questions about the bit flipping procedure? 4. New charter 4. API document structure 5. Requirements document structure 6. Issue list Cheers, Tom

1 0

Modified Charter and SAGA Session #3 notes
by Shantenu Jha 13 Oct '05

13 Oct '05

Appended below are: i) NOtes from the third SAGA session at GGF15 (Boston) ii) Modified charter -- signalling the beginning of the process of bit-flipping to SAGA-WG. We're aiming to forward to GFSG by 01 November, so please be sure to send in suggestions/correction well before. There will be more housekeeping issues related to the bit-flippin, e.g new Gridforge(?) location etc. GGF15 was a busy meeting for SAGA and there were many productive developments outside the formal (three) SAGA-RG sessions. Stay tuned for some general notes from GGF15. ===== SAGA-RG Seesion 3 Notes ====== 0. Agenda Bashing 1. Revised Charter & Milestones. 2. Requirement Documents 3. F2F at SC05 4. API Strawman Discussion: - Task Model (revisit) - sessions? - security? 5. AOB Chaired by Andre, Shantenu Thilo (saga-notetaker-in-chief), Dieter Kranzlmueller (AD), Pascal and Ninf-G/NAREGI representative. Shantenu and Andre presented the revised "Charter and Milestones" , required to flip the bit to a WG. The session was almost entirely devoted to discussing the revised "Charter and Milestones". Revised document essentially reflects the discussion. Deadline for comments on "Charter and Milestones" was left to chairs to determine. DEADLINE FOR COMMENTS/CRITICISM/MODIFICATIONS SET TO BE ********* 01 NOVEMBER, 2005. ************** In the last five minutes, it was formally agreed to have a SAGA face-to-face at SC05 and the "Task Model" was revisited -- based upon some informal discussions between Andre, Chris Smith and Shantenu. Email by Andre to be sent to the mailing list soliciting wider response to the modified task model. - Group did not get to discussing sessions and security, as charter revision discussions took up longer than expected. Appended below the Revised Charter [saga-cvs initial version 1.1] [Please note request for comments is 01 November] ======= Prosposed Charter for SAGA-WG ================= Informational Section --------------------- Area: Applications [Standards] Name of group: Simple API for Grid Applications Acronym: SAGA-WG Type of group: Working Group (WG) Chairs: Tom Goodale goodale(a)cct.lsu.edu Shantenu Jha s.jha(a)ucl.ac.uk Andre Merzky andre(a)merzky.net Secretaries: TBD Email list: saga-wg(a)ggf.org Web page: http://forge.ggf.org/projects/saga-wg/ Charter: ------- Focus/Purpose: ------------- Many application developers wish to make use of the exciting possibilities opened up by the advent of the Grid. Application developers, however, have their own agendas to pursue and often cannot spare the time or resources to fully investigate the vast wealth of Grid technologies and APIs which currently exist. They would rather be presented with a simple API close to the programming paradigms and interfaces they are used to. For example, the process of copying of a remote file may involve interaction with a replica location service, a data transport service, a resource management service, and a security service. It may involve communication via LDAP/LDIF, GRAM, HTTP, and GSI, as protocols or protocol extensions. A Fortran application programmer, however, wants to see a call very much like: call fileCopy (source, destination) Although this example is simplified, it illustrates the motivation for our work. The APIs specified by this WG will deliver a similar level of abstraction for a range of basic operations which need to be grid aware. The precise set of operations is to be decided by the WG based upon application requirements and use cases; examples of such operations may be file access, job submission, monitoring or steering. The group will lower the barrier for application developers to make use of the grid by providing a small, consistent API for the operations of interest, the Simple API for Grid Applications (SAGA). Scope: ----- The proposed API specifically targets applications, which aim to take advantage of some of the features that the Grid offers. The scope of the API is open to all application areas -- scientific and non-scientific. The scope is primarily defined by the Use Cases received; but will also reflect the areas of expertise of active members of the group. Simplicity and conciseness will be the governing principles. Currently the use cases received are from scientific application areas and are collected in the "SAGA Use Case document" (http:// i will add later). The API targets developers of applications who wish to grid enable their applications whilst spending as little time as possible learning new paradigms. Such developers typically wish to devote their time to their own goals and minimise the time spent coding infrastructure functionality. The API will insulate application developers from middleware. The specification of services and the protocols to interact with them is out of the scope of the WG. Rather, the API seeks to hide the detail of any service infrastructures that may or may not exist to implement the functionality that the application developer needs. The WG will, however, actively liase with all grid-middleware groups within the GGF to ensure compatability. The WG will continue to identify projects outside GGF with similar API-focus and goals, and will seek their input in the development of the API and its implementation. The WG will provide detailed examples and cook-book. Goals: ------ The SAGA-WG is the logical evolution of the SAGA-RG, which produced the informational document ("SAGA Use-Case Document") and did preliminary work on the "SAGA Requirements Document". The main goal of the WG is the creation of an API specification as a GGF standard recommendation. On the road to that specification, SAGA-WG will build upon the SAGA-RG documents and produce the following: * Informational Document: "Requirements document" - define exact scope of API - define target user groups for API - define programming languages to be supported - define organisation of API documents (e.g. one per subsystem or language, or one complete document) * Informational Document: "SAGA compatability with GGF middleware" This document will survey the compatability of SAGA reference implementations with underlying models of grid-middleware including, but not confined to OGSA. * Recommendation Document: "SAGA-API Specification" The SAGA API specification. Milestones: ----------- GGF16: Presentation of "Requirements document" Submit Informational Document "SAGA Compatability with GGF middleware" SAGA API Specification pre-v1.0 GGF17: Submission of SAGA API Specification v1.0 Implementation details review. GGF18: Work commences on extending the functional scope of SAGA (v2.0) GGF19: Present Draft Language Bindings (C++/Java/) -- Shantenu Jha, Research Fellow Centre for Computational Science http://www.realitygrid.org University College London (UCL) PGP Key ID 0x78F823E6 s dot jha at ucl dot ac dot uk +44 (0) 20-7679-5300

3 3

Conference calls
by Tom Goodale 13 Oct '05

13 Oct '05

Hi, we didn't have our regular call last week because of GGF, and I am assuming people are recovering this week, so should we start the calls again next week, on the 20th of November ? Cheers, Tom

2 2

current API draft, downloadable?
by Thilo Kielmann 11 Oct '05

11 Oct '05

Looks like I am overlooking the obvious again... Where can I find the current API draft as a single, downloadable document? I'd prefer plain ASCII.... Thanks! Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

2 1