November 2005 - saga-rg - lists.ogf.org

task states
by Andre Merzky 05 Nov '05

05 Nov '05

Hi all, we had some open notes about task states. Attached is a proposal for a task state model, which incorporates the task states we have in the strawman with the proposals. Does anybody see any difficulties with that? Best regards, Andre. -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

2 1

task error handling...
by Andre Merzky 05 Nov '05

05 Nov '05

Hi again, the task error handling is somewhat unclear in the spec. We had soe discussions with the GridRPC folx at GGF about that, and I had some earlier notes. So, here is what came out of the discussions: - a running task does NEVER throw an error during its run - if a task meets an error condition, the error (or exception) is to be stored (if possible) - if an error is encountered by a task, the task state has to be changed to Failed - the throw () method causes the task to throw any error which was previously encountered. In all states _but_ in the Failed state the call does nothing. Example code looks like: { task.run (); task.wait (); if ( task.get_status = saga::task::Failed ) { try { task.throw (); } catch ( saga::exception e ) { std::cerr << "Oops!\n"; } } } Note that the 'if' check for the task state is not required - for all other states the try would yield nothing. Does anybody see problems with that approach? Is it simple enought? The get_error method still exists, and returns the error string (basically the string from exception.what() ). Cheers, Andre. -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

1 0

Task model...
by Andre Merzky 04 Nov '05

04 Nov '05

Hi Group, as you have seen in the notes from last GGF, there was some discussion about the SAGA task model. This mail summarizes this discussion, and proposes some changes to the task model to accommodate the suggestions made in Boston. Sorry that this mail comes late, I pushed that off far too long... As a reminder, the current task model in action looks like this: --------------------------------------------------------------------- Example 1: saga::directory d ("/tmp/"); saga::directory::task_factory dtf = d.get_task_factory (); saga::task t = dtf.mkdir ("test/"); t.run (); t.wait (); --------------------------------------------------------------------- So, the minimal number of code lines needed for running a task is 4. Note that the asynchroneous mkdir method has the same signature as the synchroneous, but has a different return value. Compare that to the mechanism used in MPI or used in GridRPC: --------------------------------------------------------------------- Example 2: saga::directory d ("/tmp/"); saga::task t = d.mkdir_async ("test/"); t.wait (); --------------------------------------------------------------------- The number of lines is 2, and there is no explicit additional object (the task factory). Please note that the created task is automatically run on creation time. The discussion in Boston showed that people favor the similar mechanism of example 2. Also, SAGA has not a single use case which requires (or even motivates) the task_factories as explicit objects in the SAGA API. But, we do have use cases which motivate that tasks are NOT autimatically run. In particular the following scenario of bulk operations shows that: --------------------------------------------------------------------- Example 3: saga::file f ("/tmp/t.dat"); saga::file::task_factory dtf = d.get_task_factory (); saga::task_container tc; for ( int i = 0; i < 1000; i++ ) { saga::task t = ftf.read (i, 1, buffer[i]); tc.add_task (t); } tc.run (); tc.wait (); --------------------------------------------------------------------- As the tc.run starts all tasks at the same time, the implementation has the chance to optimize that, and to perform one remote operation instead of 1000 independend ones. With the model shown in example 2, this optimization would be impossible, and the bulk related use cases in SAGA would be difficult to implement (efficiently). In order to accommodate the explicit run() method on tasks, there are two possibilities (well, there are more, but two seem particularily obvious): --------------------------------------------------------------------- Example 4a: distinguish task from async call saga::directory d ("/tmp/"); saga::task t_1 = d.mkdir_async ("test/"); saga::task t_2 = d.mkdir_task ("test/"); t_2.run () t_1.wait (); t_2.wait (); Example 4b: flag to start task saga::directory d ("/tmp/"); saga::task t_1 = d.mkdir_async ("test/", TRUE); // TRUE optional saga::task t_2 = d.mkdir_async ("test/", FALSE); t_2.run () t_1.wait (); t_2.wait (); --------------------------------------------------------------------- Although the version 4b seems somewhat simplier, most people (including me) seem to oppose a seemingly artificial flag on all async calls to flag the runnnin of the task. Also, this would change the signature if compared to the synchroneous version of the method. Version 4a OTOH increases the number of methods. Then again, that might not be sooooo bad. The current task model doubles the number of methods as well. But as we describe the model in text, and do not actually add all signatures twice in the spec, the spec stays terse (for a very specific definition of terse, to be sure). So, version 4a would add only one paragraph to the spec :-) Also, 4a is strictly spoken just a marker to the sync methods, which makes them asynchroneous. In different language bindings, this could look very different, and convenient: --------------------------------------------------------------------- Example 4a: more versions in C++ 4a1: d.mkdir ("test/"); d.mkdir_sync ("test/"); // same saga::task t_1 = d.mkdir_async ("test/"); saga::task t_2 = d.mkdir_task ("test/"); 4a2: d.mkdir ("test/"); d.sync ::mkdir ("test/"); // same saga::task t_1 = d.async::mkdir ("test/"); saga::task t_2 = d.task ::mkdir ("test/"); 4a3: d.mkdir ("test/"); d.mkdir <sync> ("test/"); // same saga::task t_1 = d.mkdir <async> ("test/"); saga::task t_2 = d.mkdir <task> ("test/"); --------------------------------------------------------------------- All these versions are equivalent. I personally like 4a2 best, Thilo likes 4a1 (close to MPI), Hartmut favors 4a3 (nice to implement in C++). Well, here we are really: the proposal is following: P1) get rid of the task factory (there seems no need for it) P2) allow tasks which do run() on creation P3) allow tasks which do NOT run() on creation All both 4a and 4b deliver that, with only small changes in the API spec (but obviously larger changes in the language bindings). For 4a, the versions 4a1 to 4a3 show different posssible C++ language bindings. Questions: Q1) any comments to the Boston discussion? I hope I reflected that correctly... Q2) are there any drawbacks by introducing 4a or 4b, apart from the additional number of calls? Q3) Are the advantages (simplicity for most usages, powerfull for bulk operations and others) good enough to introduce 4a or 4b? Q4) You like 4a or 4b better? Q5) Any comments to 4a1, 4a2 or 4a3? (not part of the Strawman!) Please answer before November 10th: firstly, we would like to finish the task model soon, in order to get the Strawman stabilizwed before GGF16; secondly, we would like to come to a decision during the F2F meeting at SuperComputing in Seattle (November 12-18). Cheers, Andre. -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

7 20

SC05 F2F Meeting...
by Andre Merzky 03 Nov '05

03 Nov '05

Hi all, as was proposed in Boston, we are organizing a SAGA F2F meeting in SuperComputing'05. Currently, the Afternoon of Thursday, Nov. 17th, is an open slot for those people I ask (that means exactly Shantenu to be honest ;). Would that create a conflict for anyone? Say from 13:00 - 18:00? Also, please give us a ping if you intent to participate, as we need a rough headcount for room planning. There is no fixed agenda as of yet, but some point which might make it onto the agenda: - WG charter (finalize) - SAGA task model (finalize) - use case evaluation (update, feed into req. doc) Suggestions for additional agenda items would be very welcome. Best regards, Andre. -- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky(a)cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+

3 3

Details for call 03/11/05 (fwd)
by Tom Goodale 03 Nov '05

03 Nov '05

Hi, there doesn't seem to be anyone else joining; I will give it another couple of minutes... Cheers, Tom ---------- Forwarded message ---------- Date: Wed, 2 Nov 2005 10:54:12 +0000 (GMT) From: Tom Goodale <goodale(a)cct.lsu.edu> To: saga-rg(a)ggf.org Subject: [saga-rg] Details for call 03/11/05 Hi, Tomorrow's details are below. NOTE: this week the regular number was again completely booked, so we have a new number; I will try to book earlier next time so hopefully we should be back to the regular number soon... Date: Thu 3 November 2005 Time: 1700 Central European Summer Time 1600 Greenwich Mean Time 1000 Central Standard Time 0800 Pacific Standard Time Number: +1-620-782-2200 Code: 808044 Tentative Agenda: 1. Agenda bashing 2. Note Taker 3. New charter 4. Requirements document 5. Issue list I will send a final agenda tomorrow, so if there are more things to discuss, or you want to remove any of the above, let me know. Cheers, Tom

2 1

Details for call 03/11/05
by Tom Goodale 02 Nov '05

02 Nov '05

Hi, Tomorrow's details are below. NOTE: this week the regular number was again completely booked, so we have a new number; I will try to book earlier next time so hopefully we should be back to the regular number soon... Date: Thu 3 November 2005 Time: 1700 Central European Summer Time 1600 Greenwich Mean Time 1000 Central Standard Time 0800 Pacific Standard Time Number: +1-620-782-2200 Code: 808044 Tentative Agenda: 1. Agenda bashing 2. Note Taker 3. New charter 4. Requirements document 5. Issue list I will send a final agenda tomorrow, so if there are more things to discuss, or you want to remove any of the above, let me know. Cheers, Tom

1 0