- gridrpc-wg - lists.ogf.org

Re: [saga-rg] Task model...
by Andre Merzky 09 Nov '05

09 Nov '05

Ah, right, sorry, I forgot about that model... There are two points to it, really: a) synchroneous calls alse return a (finished) task b) a object can be either synchroneous or asynchroneous (a) is actually a more general point: it would be interesting for the other models we listed earlier as well. I kind of like it, as it makes implementation simplier, but OTOH it makes error handlin somewhat more complex for the synchroneous case, as you need to query the task state and error condition explicitely. (b) was indeed discussed, but if my memory does not trick me, it was discarded as it does not allow to - create a directory object - do a synchroneous mkdir - copy 100 files asynchroneously into that dir you cannot mix sync and async calls in that model. Is that something you would think is no problem? We don't have very detailed use cases on asynchroneous calls, apart from what we learned from the GridRPC folx. It might be worth to write some down explicitely... Cheers, Andre. Quoting [Pascal Kleijer] (Nov 07 2005): > > Hello, > > Here is an example of what was discussed in Boston. > > // Type g1: No factory > // The class has a flag in the constructor to tell if it work > // synchronous or not. > Directory d= new Directory("\", true); > Task t= d.mkdir("test\"); > t.wait(); > > //Type g2: with factory > DirectoryFactory f= new SynchronousDirectoryFactory(); > Directory d= f.newInstance("\"); > Task t= d.mkdir("test"); > t.wait(); > > Whatever the nature of the call, synchronous or not, a task is returned. > A synchronous call will return a finished task. By providing a tight > encapsulation it is possible to sub-class easily to add new features. > For example you can implement a DebugDirectory class easily adding some > extra debugging information without having to change the main code. > > My preference goes to the g1 case since it is the most direct and no > factory is needed. Pattern design (g2) is nice, but I'm not sure if this > is really necessary for this model. > > I know this solution is really OO designed, but that?s just because I > always think in terms of objects. > > > Best regards, > Pascal Kleijer > > > >Hi Pascal, > > > >there have been more models in discussion in Boston, true > >enough. However, I can't really figure out what model you > >refer to. Some example code would be helpful :-) > > > >Cheers, Andre. > > > > > >Quoting [pascal.kleijer(a)knd.biglobe.ne.jp] (Nov 03 2005): > >>Hello all, > >> > >>Lets enter the arena if it is not too late, sorry for that. > >>I do agree with many things that have been discussed here. > >>Being myself a POP (pattern oriented programming) and OOP > >>(Object Oriented programming) guy I tend to favor nicely > >>encapsulated models. > >> > >>As mentioned by Tom, the template is possible in Java as > >>well. But that forces the usage of Java 5.0 (aka 1.5) at > >>least. This isn't recommended since many programs > >>still run with 1.4. Using them would probably create a very > >>different binding in Java, C++ or Python, which would end up > >>in 2 different SAGA implementations/usage, one for OO > >>languages and one for old fashioned procedurals. > >> > >>As to exposing the different implementation to the user I > >>think this could be avoided in OO languages, or at least it > >>should not be as obvious as in 4a2. During the GGF meeting > >>their was a last minute discussion about a `g' solution > >>which is pretty much pure OO design with a simple method > >>call whatever model is used. This is I think a good choice > >>for an OO point of view. You can derivate an abstract class > >>to specialized classes with synchronous or asynchronous > >>implementation. This would even open the room for sub > >>classing it to add custom code for debugging for example or > >>other treatment without affecting the initial code. > >>Depending on how the programming is tackled factories aren't > >>necessary. The problem might come when implementation is > >>made in procedural languages, it might be necessary to add > >>additional parameters to the constructor or the calls. > >> > >>I don't have the time this morning to make a sample > >>code. But I hope you got the idea. > >> > >>Best Regards, > >>Pascal Kleijer > >> > >>>My vote is for 4a1. (Which was option D in our original discussion I > >>>believe.) > >>> > >>>Cheers, > >>> > >>>Tom > >>> > >>>On Thu, 3 Nov 2005, Andre Merzky wrote: > >>> > >>>>Hi John, Thilo, Group, > >>>> > >>>>Hartmut and I would like to argue about the C++ bindings. > >>>>We would like to come to an agreement soon, as our > >>>>implementation is dealing with the async part right now. > >>>> > >>>>As reminder: > >>>> > >>>>--------------------------------------------------------------------- > >>>>Example 4a: more versions in C++ > >>>> > >>>>4a1: d.mkdir ("test/"); > >>>> d.mkdir_sync ("test/"); // same > >>>> saga::task t_1 = d.mkdir_async ("test/"); > >>>> saga::task t_2 = d.mkdir_task ("test/"); > >>>> > >>>>4a2: d.mkdir ("test/"); > >>>> d.sync .mkdir ("test/"); // same > >>>> saga::task t_1 = d.async.mkdir ("test/"); > >>>> saga::task t_2 = d.task .mkdir ("test/"); > >>>> > >>>>4a3: d.mkdir ("test/"); > >>>> d.mkdir <sync> ("test/"); // same > >>>> saga::task t_1 = d.mkdir <async> ("test/"); > >>>> saga::task t_2 = d.mkdir <task> ("test/"); > >>>> --------------------------------------------------------------------- > >>>> > >>>> > >>>>Quoting [John Shalf] (Nov 02 2005): > >>>>>> Q5) Any comments to 4a1, 4a2 or 4a3? (not part of the Strawman!) > >>>>>I prefer 4a1 because it is more readable and the implementation would > >>>>>be quite straightforward. > >>>>You favour 4a1. We think that implementation is very straight > >>>>forward for all three versions. Readability seems not much > >>>>different to me, and might largely be a matter of taste. > >>>> > >>>> > >>>>>It is also a familiar paradigm for any MPI > >>>>>programmers and anyone who has played with various proprietary Async > >>>>>I/O implementations. (its a very familiar and conventional approach) > >>>>Well, these are C-API's. Is there a C++ binding for MPI, > >>>>and does it look the same? > >>>> > >>>> > >>>>>I kind of like 4a2 as well from the standpoint of a C++ programmer > >>>>>(even with Andre's syntax corrections). However, the resulting > >>>>>bindings will not be very consistent with the approach we would take > >>>>>for Fortran or C bindings (eg. those would likely look more like > >>>>>4a1). > >>>>But, well, that is the idea of the language binding! I agree > >>>>that C and Fortran would look more like 4a1, of course. But that > >>>>is no reason that C++ should look like that as well, or Java, > >>>>Perl etc. > >>>> > >>>> > >>>>>It is not really much more readable than 4a1. Therefore, I'm > >>>>>uncertain if it is worth fragmenting our approach to bindings in > >>>>>different languages when there is not a clear benefit in terms of > >>>>>readability or implementation complexity. > >>>>I think that 4a2/4a3 actually allow nicer implementations, as it > >>>>allows to have the different async parts somewhat separate from > >>>>the sync parts. We think its nicer :-) > >>>> > >>>> > >>>>>I do a lot of C++ programming, but I find the 4a3 option a bit > >>>>>obscure both in terms of readability and any advantages it might > >>>>>confer in terms of implementation. > >>>>Hehe - I thought the same :-) Hartmut likes that version very > >>>>much. To me it appealed after pondering over it for a couple of > >>>>days. Now I think it is cute, and quite expressive. > >>>> > >>>> > >>>>>It would certainly be easier to > >>>>>create multi-language bindings for a single code base if we stick > >>>>>with something more conventional like 4a1. > >>>>I think that C and Fortran bindings for this part are straight > >>>>forward anyway, there is no need to reflect that in C++... > >>>> > >>>> > >>>>>Each approach is equally readable to me (less so for 4a3). I'm > >>>>>certainly open to any additional information on how the 4a2 and 4a3 > >>>>>approaches could simplify implementation. > >>>>The main point really is that the object itself has only the sync > >>>>method calls, and the async calls can be put into separate > >>>>header/sources easily, and build (by default) on top of the sync > >>>>calls. Of course, you can do that with all three versions, but > >>>>its not as obvious in 4a1. > >>>> > >>>> > >>>>>If the other approaches > >>>>>can offer some implementation benefits, then maybe I'd give them > >>>>>extra consideration, but otherwise, I would prefer a more > >>>>>conventional approach like 4a1. > >>>>I would vote for 4a2 or 4a3 (in that order), but 4a1 would be ok > >>>>if the majority likes that most, of course. Basically its > >>>>a matter of taste I think. I am happy that the general point > >>>>seems accaptable to all so far: having sync, async, and task > >>>>versions of the calls, w/o explicit task factory. > >>>> > >>>>>The only implementation I'm outright > >>>>>against is the 4b example. > >>>>Good! :-) > >>>> > >>>>Thanks, Andre. > >>>> > >>>> > >>>>>-john > >>>> > >>>> > >>>> > >>> > > > > > > > -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

2 2

Re: GridRPC and SAGA
by Craig Lee 26 Sep '05

26 Sep '05

Absolutely. I'll urge GridRPC folks to attend. --Craig At 03:02 AM 9/26/2005, Andre Merzky wrote: >Hi Craig, > >The GridCPR folx will have a talk in our Monday SAGA session >(14:00), to coordinate their API efforts with the SAGA API. >Would the GridRPC group be interested in joining that >discussion? > >Cheers, Andre. > > > >-- >+-----------------------------------------------------------------+ >| Andre Merzky | phon: +31 - 20 - 598 - 7759 | >| Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | >| Dept. of Computer Science | mail: merzky(a)cs.vu.nl | >| De Boelelaan 1083a | www: http://www.merzky.net | >| 1081 HV Amsterdam, Netherlands | | >+-----------------------------------------------------------------+

1 0

Fw: [ggf-editor | Submit GGF Draft - 1166] A GridRPC Model and API for End-User Applications
by Yoshio Tanaka 07 Sep '05

07 Sep '05

Dear all, "A GridRPC Model and API for End-User Applications" has been confirmed by the GFSG for publication. The draft recommendation will be published as GFD-R-P.052. Thanks to the authors and all members of GridRPC WG. Sincerely, -- Yoshio Tanaka (yoshio.tanaka(a)aist.go.jp) http://ninf.apgrid.org/ http://www.apgrid.org/

1 0

session at GGF15
by Hidemoto Nakada 31 Aug '05

31 Aug '05

All, I have requested one 90 min. session at the coming GGF15. The planned agenda is - discuss on Middleware API, especially on the data-handle issue. Please join us. hidemoto

1 0

Fw: [ggf-editor | Submit GGF Draft - 1166] A GridRPC Model and API for End-User Applications
by Yoshio Tanaka 26 Aug '05

26 Aug '05

Dear all, A good news! Dennis has approved "A GridRPC Model and API for End-User Appplications". Let me check what the next phase is :-) Thanks, -- Yoshio Tanaka (yoshio.tanaka(a)aist.go.jp) http://ninf.apgrid.org/ http://www.apgrid.org/

1 0

Fw: [ggf-editor | Submit GGF Draft - 1166] A GridRPC Model and API for End-User Applications
by Yoshio Tanaka 24 Aug '05

24 Aug '05

Dear all, A GridRPC Model and API for End-User Applications is now waiting comments from Dennis. -- Yoshio Tanaka (yoshio.tanaka(a)aist.go.jp) http://ninf.apgrid.org/ http://www.apgrid.org/

1 0

Revised End-User API doc
by Yoshio Tanaka 26 Jul '05

26 Jul '05

Dear all, Based on public comments and discussions at the GGF14, Hidemoto revised the GridRPC Models and End-User API document. Please find the attached revised document and our comments to the public comments. If you have any comments, questions, suggestions, etc., please send it to the GridRPC ML. I hope the revised document will be submitted to the editor in these few weeks. Regards, -- Yoshio Tanaka <yoshio.tanaka(a)aist.go.jp> http://ninf.apgrid.org/ http://www.apgrid.org/

1 0

Response to public comments
by Yoshio Tanaka 30 Jun '05

30 Jun '05

Dear all, Please find the attached file that describes public comments and our reponse to them which was discussed at the GridRPC WG at GGF14. Based on the discussions, Hidemoto will revise the API document in two weeks. If you have any comments, questions, etc., please send email to this ML. cheers, -- Yoshio Tanaka <yoshio.tanaka(a)aist.go.jp> http://ninf.apgrid.org/ http://www.apgrid.org/

1 0

questions about function handles
by Laurent Philippe 24 Jun '05

24 Jun '05

All, After the last GridRPC meeting in Seoul, we promised to send a new proposal for data management in GridRPC and an answer to Hidemoto's comments. However, in our discussions we still face a problem on the definition of function handles. There two ways to initialize a function handle. - first, we can do it with grpc_function_handle_init. In this case, the server is explicitly given by the client. As data location is known (client, server or data repository), there no problem to bind the data to the server. All data management can be done by the client: placement, transfers or removal. The location will be given in the data management functions. - second, the function handle could be initialized by grpc_function_handle_default. In that case, the GridRPC API document says: "This default could be a pre-determined server or it could be a server that is dynamically chosen by the resource discovery mechanisms of the underlying GridRPC implementation". Does that mean, the function handle will contain a server reference in it after grpc_function_handle_default call ? Or does that mean, the function handle will reference a default discovery (or GridRPC server) while the computational server will be chosen during grpc_call ? If the function handle contains a reference to a server, then the data management can be done in the same way as for grpc_function_handle_init. If the function handle does not reference the computational server, there no way to know where to place data before issuing grpc_call. This is the way function handles are implemented in Diet and Netsolve (2.0, any changes ?) GridRPC interfaces. However, we should provide a way to dynamically choose a server... Any comments ? L. Philippe & B. DelFabbro -- Laurent PHILIPPE http://lifc.univ-fcomte.fr/~philippe philippe(a)lifc.univ-fcomte.fr Laboratoire d'Informatique (LIFC) tel: (33) 03 81 66 66 54 route de Gray fax: (33) 03 81 66 64 50 25030 Besancon Cedex - FRANCE

2 7

Public comments to the End-User API Doc
by Yoshio Tanaka 20 Jun '05

20 Jun '05

Dear all, Attached file includes public comments for the GridRPC Document. We are requested to review and respond to the comments, and if needed upload a new version of the document. I'll discuss with Hide and submit proposed actions by GGF14. Let's discuss in detail in Chicago. Of course, your comments and suggestions are very appreciated. Sincerely, -- Yoshio Tanaka <yoshio.tanaka(a)aist.go.jp> http://ninf.apgrid.org/ http://www.apgrid.org/

1 0