July 2006 - saga-rg - lists.ogf.org

Re: Fwd (replogle@ggf.org): [gf-chairs] GGF18 Session Scheduling: Deadline 7 July
by Andre Merzky 06 Aug '06

06 Aug '06

Hi groups, I just realized that I can wait forever on answers on mails which I send only to myself, and not to the lists... Sorry for that! Anyway, we have now 2+2 sessions scheduled for GGF18, with somewhat fuzzy agendas. Please let us know if you are interested to see specific topics on the agendas. Cheers, Andre. Quoting [Andre Merzky] (Jun 26 2006): > Date: Mon, 26 Jun 2006 09:49:43 +0200 > From: Andre Merzky <andre(a)merzky.net> > To: saga-rg(a)merzky.net, saga-core-wg(a)merzky.net > Subject: Fwd (replogle(a)ggf.org) [gf-chairs] GGF18 Session Scheduling: Deadline 7 July > > Hi group(s), > > its time again to plan for the sessions at GGF18. Do you > guys think that the 2+2 scheme from ggf17 worked well? Do > we want to repeat that pattern? > > For the Core API spec, we don't have major open issues > anymore (some minor ones though), so that won't eat up much > time. However, we should devote some time to the CookBook > and user manual, and probably again to the implementations. > > In the RG, the sessions topics depend, I guess, from the > status of the various API suggesstions (RPC, Info, > Messaging) - I am somewhat unsure about their status by > then. > > Any thoughts? > > Cheers, Andre. > > > > ----- Forwarded message from Joel Replogle <replogle(a)ggf.org> ----- > > > To: gf-chairs(a)ggf.org > > From: Joel Replogle <replogle(a)ggf.org> > > Subject: [gf-chairs] GGF18 Session Scheduling: Deadline 7 July > > Date: Sun, 25 Jun 2006 22:53:05 -0400 > > > > GGF Chairs - > > > > The next GGF event, GGF18, is coming up and session scheduling is now > > open! GGF18 will be held in conjunction with GridWorld September > > 11-14, 2006 in Washington, DC USA. > > > > Please go to the following link for Group Session Requesting > > instructions and the link to the request form: > > http://www.ggf.org/ggf_events_next_chairinfo.htm > > > > The scheduling deadlines are: > > > > Draft Documents: > > - Deadline for submitting working drafts is 11 August, 2006. > > Session Requests: > > - Deadline for session requests for a CG, WG or RG is 7 July, 2006 > > - Deadline for AD Hoc/Charter Discussion BOFs is 7 July, 2006. > > > > We anticipate that the schedule will be finalized by the last week in > > July. > > > > Thanks, > > > > Joel > > > > ------------------------------------------------------------------------ > > ----------- > > Joel Replogle - Manager of Standards, Global Grid Forum > > replogle(a)ggf.org http://www.ggf.org > > ----- End forwarded message ----- -- "So much time, so little to do..." -- Garfield

2 2

Fwd (andre@merzky.net): Re: Fwd (andre@merzky.net): Re: Fwd (andre@merzky.net): Re: [saga-rg] context problem
by Andre Merzky 02 Aug '06

02 Aug '06

[damned, majordomo seems really broken - forward to the list again] ----- Forwarded message from Andre Merzky <andre(a)merzky.net> ----- > Date: Sun, 16 Jul 2006 19:38:54 +0200 > From: Andre Merzky <andre(a)merzky.net> > To: Thilo Kielmann <kielmann(a)cs.vu.nl> > Cc: Andre Merzky <andre(a)merzky.net> > Subject: Re: Fwd (andre(a)merzky.net) Re: Fwd (andre(a)merzky.net) Re: [saga-rg] context problem > > Quoting [Thilo Kielmann] (Jul 16 2006): > > > > Merging 2 mails from Andre: > > > > > very good points, and indeed (1) seems cleanest. However, > > > it has its own semantic pitfalls: > > > > > > saga::file f (url); > > > saga::task t = f.write <saga::task::Task> ("hello world", ...); > > > > > > f.seek (100, saga::file::SeekSet); > > > > > > t.run (); > > > t.wait (); > > > > > > > > > If on task creation the file object gets copied over, the > > > subsequent seek (sync) and write (async) work on different > > > object copies. In particular, these copies will have > > > different state - seek on one copy will have no effect on > > > where the write will occur. > > > > I cannot see a problem here: With object copying, you will simply have the > > same file open twice. And given the operations you do, this might even be > > the right thing... > > This example is very academic: can you show an example where the sharing of > > state between tasks is useful, actually? > > The problem here is, that I at a user would expect the write > to happen at byte 100, but it will happen at byte 0: the > seek happens on a different object than the write. > > What might be a more obvious example, which goes wrong along > the same lines: > > f.write ("line 1\n"); > f.write ("line 2\n"); > f.write ("line 3\n"); > > That will result in a file > > line 1 > line 2 > line 3 > > whereas the coed > > saga::task t1 = f.write ("line 1\n"); t1.run (); t1.wait (); > saga::task t2 = f.write ("line 2\n"); t2.run (); t2.wait (); > saga::task t3 = f.write ("line 3\n"); t3.run (); t3.wait (); > > will result in a file > > line_3 > > the last write will start on 0, as the previous write > operated on a different file pointer. In general, you > cannot execute any two tasks on a single object, at least > not if any state is of concern, such as file pointer, pwd, > replica name, stream server port, job id, ... > > That is a no-go in my opinion, as it is counter-intuitive, > and breaks a large number of use cases. And is incosistent > with the syncroneou method calls. > > Yes, you can wreak havoc with state as well: > > saga::task t1 = f.write ("line 1\n"); > saga::task t2 = f.write ("line 2\n"); > > t1.run (); > t2.run (); > > t1.wait (); > t2.wait (); > > will likely result in > > linline 2 > e 1 > > or such - the user does need to think when doing multiple > async ops at once. I don't see a way around that (and don't > see a need for it either: we want to make the Grid stuff > easy, but not revolutionize programming styles). > > > > > I should have added that I'd prefer 3: > > > > > > > > 3. when creating a task, all parameter objects are passed "by reference" > > > > > + no enforced copying overhead > > > > > - all objects are shared, lots of potential error conditions > > > > > > The error conditions I could think of are: > > > > > > - change state of object while a task is running, hence > > > having the task doing something differently than > > > intended > > > Change of state, > > That is intentional - see above. > > > > like destruction of objects > > Well, that is what we discuss :-) 3 would delay destruction > until its save (state is not needed anymore). > > > > or change of objects. > > What doe you mean here? > > > > Not to speak of synchronization conditions: supposed you > > have non-atomic write operations (which is everything that > > writes more than a single word to memory): do you thus > > also enforce object locking by doing this? > > If not, you can have inconsistent object state that can be > > seen by one task, just because another task is halfway > > through writing the object... (all classical problems of > > shared-memory communication apply) > > See above. You are right, but I don't see a way around > that, without causing more harm than good (child and bathtub > come to my mind for some reason...). > > BTW: the bulk optimization we have now assumes that tasks > which run at the same time are, by their very definition, > independent from each other, do not depend on any specific > order of execution, and do not depend from each other in > respect to object state. That are the very points we talk > here about - I think its a very sensible assumption. I have > the same behaviour on a unix shell BTW: > > touch file > date >> file & > date >> file & > > I would not be able to make assumptions about the file > contents... (well, here I could make a save bet, but you > know what I mean). > > > > > - limited control over resource deallocation > > > > this is the same thing as above > > > > The problem really is that there is no "object lifecycle" > > defined. There is no way to define which task or thread > > might be responsible or even allowed to destroy objects or > > change objects. Is it??? > > Yes, that is what I mean with limited control. > > We had a discussion on this list and in Tokyo about the > semantics of cancel(), which touches the same problem: > should task.cancel() block until resources are freed? As we > might talk about remote resources, and Grids are unreliable, > we might block forever. That does not make sense, at least > not always. > > The resolution we came up with is that cancel() is advisory, > so non-blocking, but can also use a timeout parameter (with > -1 meaning forever) to block until resources are freed. > > Timeouts do not make sense on destructors I believe, but > 'advisory destruction' does, IMHO. > > > > > The advantages I see: > > > > > > - no copy overhead (but, as you say, that is of no > > > concern really) > > > > ok, but minor point. > > right. Lets forget that from now on. > > > > > - simple, clear defined semantics > > no, it is the the most dangerous of the three versions > > Well, see above - I think its the most sensible semantics > :-) > > > > > - tasks keep objects they operate on alive - > > > objects keep sessions they live in alive - > > > sessions keep contexts they use alive > > > > what is the maening of "alive" here??? Now that you have > > outruled memory management... > > see above: resources get freed if not needed anymore. > > > > > > - sync and asyn operations operate on the same > > > object instance. > > > > Let's forget about "sync" here: it is the task that is > > running in the current thread, so multiple tasks share > > object instances. > > Well, it would be nice to have same semantics for sync and > async, don't you think? :-) > > > > > Either way (1, 2 or 3), we have to have the user of the > > > API thinking while using it - neither is not idiot > > > proof. > > > > Well, we should strive to limit the mental load on the > > programmer as much as possible... > > > > > I think (2) is most problematic, if I understant your > > > 'hand-over' correctly: that would mean you can't use the > > > object again until the task was finished? > > > > No, it means you will never ever again be allowed to use > > these objects. (hand over includes the hand over of the > > responsibility to clean up...) > > Right. So you can never do a async read, and then a sync > seek, and then a async read again. At least not with > sensible results. > > Also, I need to create 100 file instances to do 100 reads? > Remember that opening a file is a remote op in itself, > potentially. Then we don't need the task model anymore. > > That is broken IMHO. > > Cheers, Andre. > > > > Thilo > > -- "So much time, so little to do..." -- Garfield ----- End forwarded message ----- -- "So much time, so little to do..." -- Garfield

7 22

Fwd (maclaren@cct.lsu.edu): [Grid] Fwd: [gce-rg] Call For Papers: GCE06
by Andre Merzky 27 Jul '06

27 Jul '06

Hi Folx, thish sounds like an interesting CFP... Cheers, Andre. ----- Forwarded message from Jon MacLaren <maclaren(a)cct.lsu.edu> ----- > From: Jon MacLaren <maclaren(a)cct.lsu.edu> > Date: Wed, 26 Jul 2006 21:18:37 -0400 > To: Grid Computing Research Group <grid(a)cct.lsu.edu> > Subject: [Grid] Fwd: [gce-rg] Call For Papers: GCE06 > > Begin forwarded message: > > > >CALL FOR PAPERS > > > >GCE06 - Second International Grid Computing Environments and Portals > > Workshop 06 > > > >* Conference Web Page: http://www.cogkit.org/GCE06 > >* held in conjunction with with [http://sc06.supercomp.org/ > >SC06], Tampa, FL > >* Meeting dates: Sunday, Nov 12, 8:00am-3:00pm and > > Monday, Nov 13, 8-5. 8:00am - 3:00pm > >* Meeting Location, Salon G > > > >Abstract > > > >The purpose of this Workshop is to bring together the community to > >discuss and foster the exchange of ideas for the development of tools, > >methodologies and frameworks to build Grid computing > >environments. Last years conference at SC05 focussed on tools for Grid > >Portals. This years conference will broaden the discussion to also > >include efforts of other tools that make the development and use of > >Grids more easily accesible from the desktop. The results of the > >Workshop will be published in a special edition of > >[http://www.cc-pe.net/journalinfo/ Concurrency Practice and > >Experience]. > > > >Scope > > > >Scientific portals and gateways have emerged to be important > >components of many large-scale scientific computing and Grid > >projects. They are characterized by delivering well-established > >mechanisms for providing familiar interfaces to secure Grid resources, > >services, applications, tools, and collaboration services for > >communities of scientists. In most cases access is enabled through a > >web browser without the need to download or install any specialized > >software or worry about networks and ports. As a result, the science > >application user is isolated from the complex details and > >infrastructure needed to operate an application on the Grid. > > > >In addition to Web-based tools we see the emergance of Desktop based > >Grid tools that make the development and also the use of Grids easier. > > > >This workshop will focus on projects and technologies that are > >adopting these new technologies and services as well as develop new > >once. We seek papers from all aspects of portal, gateway, and tool > >development including architecture design and standards studies, > >toolkits developed in support of Grid portals and gateways, as well as > >Grid middleware. > > > >GCE 2006 topics of interest include, but are not limited to: > > > >* Applications of Grid Portals and Gateways (including User > > Portals, Application Portals, Science Gateways, Education Portals, > > User interface/usability studies). > >* Design and architecture of Grid portals and gateways including > >* Tools and frameworks that make developing Grid Portals and > > gatweaways easier > >* Portal security models and solutions. > >* Middleware solutions in support of portals and Gatwways > > including: Web Services; WSRF Services; Grid technologies; Databases; > > Workflows; WSRP and other standards; Web services in support of > > Portals and Gateways. > >* Interaction with Commodity and Commercial tools such as Perl, > > Python, Matlab, and Mathematica. > >* Interaction with Desktop aplications. > > > >Paper Submission and Publication > > > >Authors are invited to submit a full paper on original and unpublished > >work (which is also not submitted elsewhere for review) reporting > >omsignificant projects and results in any aspect of Grid portals, > >gateways and desktops, as well as their related technologies. Papers > >should not exceed 10 single-spaced pages of text including images and > >references using 11 point size type on 8.5 x 11 inch paper with 1in > >margins. The document format is PDF. Electronic submission will be > >done through the workshop website. Submission implies the willingness > >of at least one of the authors to register and present the paper. > > > >Proceedings > > > >All papers selected for this workshop will be peer-reviewed. The > >accepted authors will have the posibility to publish an enhanced > >version in a special issue of Concurrency and Computation: Practice > >and Experience (http://www.cc-pe.net/journalinfo/) under the journal > >guidelines. > > > >Important Dates: > > > >* 11-Sep-06: Papers Due > >* 03-Oct-06: Acceptance Notification > > > >Organization: > > > >Workshop Chair: > > > >Gregor von Laszewski (ANL/UC) gregor(a)mcs.anl.gov > > > >Steering/Advisory Committee: > > > >* Dennis Gannon, Indiana Univ. (gannon(a)cs.indiana.edu) > >* Geoffrey Fox, Indiana Univ. (gcf(a)indiana.edu) > >* Rob Allan, Daresbury (r.j.allan(a)dl.ac.uk) > >* Mary Thomas, (mthomas(a)rohan.sdsu.edu) > >* Marlon Pierce, (mpierce(a)cs.indiana.edu) Last years chair. > > > >Proposed Technical Program Chairs: > > > >* Marlon Pierce (IU) (mpierce(a)cs.indiana.edu) > > > >Confirmed Techncal Program Committee > > > >* Tom Hacker (hacker(a)iu.edu) > >* Tomasz Haupt (haupt(a)cavs.msstate.edu) > >* Mary Thomas, San Diego Sate University, (mthomas(a)roham.sdsu.edu) > >* Ian Kelley, Center for Computation and Technology, > >(ikelley(a)cct.lsu.edu) > >* Suresh Marru, Indiana University > >* Masimo Cafaro, (massimo.cafaro(a)unile.it) > >* Oliver Wehrens, LSU (oliver(a)wehrens.de) > >* Marlon Pierce, Indiana University, (mpierce(a)cs.indiana.edu) > >* Ruth Pordes, Fermi National Laboratory, (ruth(a)fnal.gov) > >* Gregor von Laszewski, Argonne National Laboratory, > >(gregor(a)mcs.anl.gov) > >* Nancy Wilkins-Diehr (wilkinsn(a)sdsc.edu) > > > >other names will be added once we received official comittments > > > > > > > > > >-- > > > >=============================================================== > >Mary Thomas Department of Computer Science > >mthomas(a)rohan.sdsu.edu San Diego State University > >(619) 594-1694 (office) San Diego, CA 92182-7720 > >(619) 594-6746 (fax) http://rohan.sdsu.edu/faculty/mthomas > >Adv. Comp. Env. Lab http://acel.sdsu.edu > > > > > > _______________________________________________ > Grid mailing list > Grid(a)cct.lsu.edu > https://mail.cct.lsu.edu/mailman/listinfo/grid ----- End forwarded message ----- -- "So much time, so little to do..." -- Garfield

1 0

SAGA thread model
by Thilo Kielmann 17 Jul '06

17 Jul '06

I am sorry to say, but SAGA's task model seems to me severely flawed. This is for two reasons: 1. the "main" thread (executing sync operations) needs to be considered as yet another task 2. there must be a concise definition of the shared state. current solutions are ad-hoc and mostly undefined. shared state is: - local objects, shared between multiple tasks of the same process here: definition of synchronization between tasks - remote objects, in the service(s) here: definition of legal execution orders so far, I can see only a few incidental definitions, but they are far from being concise. "tasks in a bulk operation have to be independent" "a task cancel is doing 'best effort' but can not guarantee cancelation" The latter, BTW, is a special case, because this is about connection termination for which you can formally prove that there is no protocol that can guarantee this AND notify both parties of successful termination. To be constructive: what the task model must do first thing is - define tasks - define which data is shared between tasks and which concurrency control happens on this shared data That is the only way to define clearly what tasks will do in the event of sharing, really. You may want to look at: http://www.amazon.com/gp/product/0201695812/qid=1153102128/sr=2-3/ref=pd_bb… This is: Doug Lea, "Concurrent Programming in Java: Design Principles and Patterns" This book uses 280 pages on objects, shared state and concurrency control before using 95 pages for the thread operations... -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

3 4

IRC
by Andre Merzky 16 Jul '06

16 Jul '06

Hi folx, I am happy to announce that SAGA RG and SAGA CORE WG once more are to serve as guinea pigs for the OGF IT gimmicks! More precisely, you should be able to connect to the OGF IRC server. There is a channel #saga-rg always open, and its contents is nicely logged, so that is a viable alternative to email discussions, and also should enable us to create meeting minutes etc. on the fly For more info, have a peek at http://forge.ggf.org/sf/wiki/do/viewPage/projects.ggf-info/wiki/GGFIRC Please let me know if you have any trouble. Cheers, Andre. PS.: I tested access with gaim and xchat - that works nicely. One glitch: both clients are somewhat silent if login fails. The usual problem is that you did not use your gridforge login... I would be happy to learn about other clients behaviour - good or bad. -- "So much time, so little to do..." -- Garfield

1 0

Fwd (kielmann@cs.vu.nl): Re: Fwd (andre@merzky.net): Re: Fwd (andre@merzky.net): Re: [saga-rg] context problem
by Thilo Kielmann 16 Jul '06

16 Jul '06

and now my mail didn't make it to the list :-( ----- Forwarded message from Thilo Kielmann <kielmann(a)cs.vu.nl> ----- > Date: Sun, 16 Jul 2006 19:05:53 +0200 > From: Thilo Kielmann <kielmann(a)cs.vu.nl> > To: Andre Merzky <andre(a)merzky.net> > Subject: Re: Fwd (andre(a)merzky.net) Re: Fwd (andre(a)merzky.net) Re: [saga-rg] context problem > > Merging 2 mails from Andre: > > > very good points, and indeed (1) seems cleanest. However, > > it has its own semantic pitfalls: > > > > saga::file f (url); > > saga::task t = f.write <saga::task::Task> ("hello world", ...); > > > > f.seek (100, saga::file::SeekSet); > > > > t.run (); > > t.wait (); > > > > > > If on task creation the file object gets copied over, the > > subsequent seek (sync) and write (async) work on different > > object copies. In particular, these copies will have > > different state - seek on one copy will have no effect on > > where the write will occur. > > I cannot see a problem here: With object copying, you will simply have the > same file open twice. And given the operations you do, this might even be > the right thing... > This example is very academic: can you show an example where the sharing of > state between tasks is useful, actually? > > > I should have added that I'd prefer 3: > > > > > > 3. when creating a task, all parameter objects are passed "by reference" > > > > + no enforced copying overhead > > > > - all objects are shared, lots of potential error conditions > > > > The error conditions I could think of are: > > > > - change state of object while a task is running, hence > > having the task doing something differently than > > intended > Change of state, like destruction of objects or change of objects. > Not to speak of synchronization conditions: supposed you have non-atomic > write operations (which is everything that writes more than a single > word to memory): do you thus also enforce object locking by doing this? > If not, you can have inconsistent object state that can be seen by one > task, just because another task is halfway through writing the object... > (all classical problems of shared-memory communication apply) > > > > > - limited control over resource deallocation > > this is the same thing as above > > The problem really is that there is no "object lifecycle" defined. > There is no way to define which task or thread might be responsible or > even allowed to destroy objects or change objects. Is it??? > > > The advantages I see: > > > > - no copy overhead (but, as you say, that is of no concern > > really) > > ok, but minor point. > > > - simple, clear defined semantics > no, it is the the most dangerous of the three versions > > > - tasks keep objects they operate on alive > > - objects keep sessions they live in alive > > - sessions keep contexts they use alive > > what is the maening of "alive" here??? > Now that you have outruled memory management... > > > - sync and asyn operations operate on the same object > > instance. > Let's forget about "sync" here: it is the task that is running in the > current thread, so multiple tasks share object instances. > > > Either way (1, 2 or 3), we have to have the user of the API > > thinking while using it - neither is not idiot proof. > > Well, we should strive to limit the mental load on the programmer as much > as possible... > > > I think (2) is most problematic, if I understant your > > 'hand-over' correctly: that would mean you can't use the > > object again until the task was finished? > > No, it means you will never ever again be allowed to use these objects. > (hand over includes the hand over of the responsibility to clean up...) > > > Thilo > -- > Thilo Kielmann http://www.cs.vu.nl/~kielmann/ ----- End forwarded message ----- -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

1 0

Re: Fwd (andre@merzky.net): Re: Fwd (andre@merzky.net): Re: [saga-rg] context problem
by Andre Merzky 16 Jul '06

16 Jul '06

I should have added that I'd prefer 3: > > 3. when creating a task, all parameter objects are passed "by reference" > > + no enforced copying overhead > > - all objects are shared, lots of potential error conditions The error conditions I could think of are: - change state of object while a task is running, hence having the task doing something differently than intended - limited control over resource deallocation The advantages I see: - no copy overhead (but, as you say, that is of no concern really) - simple, clear defined semantics - tasks keep objects they operate on alive - objects keep sessions they live in alive - sessions keep contexts they use alive - sync and asyn operations operate on the same object instance. Either way (1, 2 or 3), we have to have the user of the API thinking while using it - neither is not idiot proof. I think (2) is most problematic, if I understant your 'hand-over' correctly: that would mean you can't use the object again until the task was finished? Cheers, Andre. Quoting [Andre Merzky] (Jul 16 2006): > > Hi Thilo, > > very good points, and indeed (1) seems cleanest. However, > it has its own semantic pitfalls: > > saga::file f (url); > saga::task t = f.write <saga::task::Task> ("hello world", ...); > > f.seek (100, saga::file::SeekSet); > > t.run (); > t.wait (); > > > If on task creation the file object gets copied over, the > subsequent seek (sync) and write (async) work on different > object copies. In particular, these copies will have > different state - seek on one copy will have no effect on > where the write will occur. That is the reason why we have > the delayed object destruction in the first place: that > tasks can share state. > > Cheers, Andre. > > > > Quoting [Thilo Kielmann] (Jul 16 2006): > > Date: Sun, 16 Jul 2006 18:20:50 +0200 > > From: Thilo Kielmann <kielmann(a)cs.vu.nl> > > To: saga-rg(a)ggf.org > > Subject: Re: Fwd (andre(a)merzky.net) Re: Fwd (andre(a)merzky.net) Re: [saga-rg] context problem > > > > OK, let's assume that creation/deletion of SAGA objects is independent of > > memory management. > > > > > > saga_task* t; > > > > > > > > saga_session s = saga_session_create (); > > > > saga_file f = saga_file_create (s, url); > > > > > > > > saga_task t = saga_file_copy_task (target); > > > > > > > > saga_session_destroy (s); > > > > saga_file_destroy (f); > > > > > > > > saga_task_run (t); > > > > What should be the meaning of destroying s and f, while they are still in use? > > (if it is NOT memory management) > > > > Assuming that we still strive for simplicity, I can not really see why it > > should be easier to destroy objects, esp. if this does not free memory. > > > > Do I understand you right that you propose an "advice", with the semantics: > > saga_*_destroy may or may not destroy the object actually, depending on > > whether or not it (who???) still uses the object internally? > > That sounds like semantics that nobody will understand... > > (And that means what?? A destroy that is not about memory management and that > > destroys MAYBE???) > > > > Isn't it that the special thing is that a task (or async op) is running > > asynchronously from the piece of code that has created the task? > > > > I see three different (and more-or-lee clean) ways of describing (in the > > language-independent spec) what shall happen to parameters (like s and f) > > that are shared between these two threads/activities/tasks: > > > > 1. when creating a task, all parameter objects are copied to the new task. > > + the originating task is free to do with its own objects whatever it likes > > + the newly created task is a self-contained entity w.r.t. its objects > > - enforced copying means enforced runtime overhead > > 2. when creating a task, all parameter objects are "handed over" to the new > > task > > + no enforced copying overhead > > + the newly created task is a self-contained entity w.r.t. its objects, > > as long as the programmer obeys the hand-over and does not modify or > > destroy the objects in the creating task/thread/activity > > - the "if" bears the potential for a lot of tricky error conditions, > > because objects are shared de-facto > > 3. when creating a task, all parameter objects are passed "by reference" > > + no enforced copying overhead > > - all objects are shared, lots of potential error conditions > > > > > > I think the cleanest version would be 1. > > (As SAGA always assumed that its own overhead does not matter compared to > > service invocations.) > > > > > > Thilo > > > > PS: Is there a third person on the list who dares sharing his/her opinion??? -- "So much time, so little to do..." -- Garfield

1 0

Fwd (andre@merzky.net): Re: Fwd (andre@merzky.net): Re: [saga-rg] context problem
by Thilo Kielmann 16 Jul '06

16 Jul '06

and again: back to the list ----- Forwarded message from Andre Merzky <andre(a)merzky.net> ----- > Date: Sat, 15 Jul 2006 16:22:47 +0200 > From: Andre Merzky <andre(a)merzky.net> > To: Thilo Kielmann <kielmann(a)cs.vu.nl> > Cc: Andre Merzky <andre(a)merzky.net> > Subject: Re: Fwd (andre(a)merzky.net) Re: [saga-rg] context problem > > Let me re-formulate the example: > > saga::session *s = new saga::session (); > saga::file *f = new saga::file (s, url); > > saga::task t = f->copy <saga::task::Task> (target); > > delete (s); > delete (f); > > t.run (); > > That is the C++ version with explicit memory management, and > boils down to the same problem, right? Now, that looks > similar in Java I assume, but I know for sure its the same > for Python, Perl and C: > > saga_task* t; > > saga_session s = saga_session_create (); > saga_file f = saga_file_create (s, url); > > saga_task t = saga_file_copy_task (target); > > saga_session_destroy (s); > saga_file_destroy (f); > > saga_task_run (t); > > > It can be implemented, easily, in both ways: to be valid, > and to be invalid. In the C example, assume that s, f and t > are actually integers, which act as handles to the internal > memory. If that internal memory is freed or not on > ..._destroy is a SAGA policy, not a C memory management > issue. If that code is valid or not is not a C/C++ > question, it is a question the language independent SAGA > spec must address. > > > "keeping objects alive" is a C++ thing. > > No, I disagree. I think you mean the memory management: that > is an C++ thing here because it is an C++ example. > > If the user can expect the above code to work or not is not > a C++ question, it needs to be addressed in the language > independent spec. > > Well, and getting back to the original point: I would > actually argue for that code to be valid, as it allows to > pass task handles around, w/o burdening the application > programmer to pass the objects, session and contexts around > as well. > > Cheers, Andre. > > > > Quoting [Thilo Kielmann] (Jul 15 2006): > > > > > > Right, we agree. If there are more/other contexts that might still be OK. > > > > IMHO, if the programmer is removing a context that is still needed, that is > > > > a mistake that might lead to an error. > > > > > > Yes, we could do that, agree. However, we are already going > > > a different route with sessions and tasks: > > > > > > -------------------------------------------------------- > > > saga::task t; > > > > > > { > > > saga::session s; > > > saga::file f (s, url); > > > > > > t = f.copy <saga::task::Task> (target); > > > } > > > > > > t.run (); > > > // session and file object are gone here - however, the > > > // task still lives, and keeps the session and object > > > // resources alive. At least some state must be > > > // maintained internally > > > -------------------------------------------------------- > > > > > > As usual, that example is in C++, but the problem is > > > independent of the language. > > > > :-) > > > > > > > > I think that this is the same problem really as with the > > > context. I think we should solve the the same way. > > > > Absolutely. > > > > > The problem really is, I think, that the above example is > > > the dominating one. Tasks, as they represent async ops, > > > are bound to get passed around through different scopes, > > > and it would be burdensome to expect the programmer to > > > keep all associated objects, sessions and contexts etc. > > > alive. > > > > "keeping objects alive" is a C++ thing. > > This means, both for contexts and sessions (and all other similar objects), > > that the C++ language binding has to define a "object lifecycle policy" that > > clearly states who (which class, e.g. user) is in charge of creating and > > destroying objects. Part of this policy is to say whether or not objects > > are copied when passed to another object, or whether they are used "by > > reference". > > > > So far, C++. And now back to SAGA semantics. > > > > What I can see in the code example above is that there is no problem for the > > SAGA operations. It is 100% a C++ problem: > > a) if session and file are passed to the task by reference, then the above > > code is simply incorrect > > b) if session and file are copied to the task, this code is fine > > > > > > Cheers, > > > > > > Thilo > -- > "So much time, so little to do..." -- Garfield ----- End forwarded message ----- -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

1 1

Re: Fwd (andre@merzky.net): Re: [saga-rg] context problem
by Andre Merzky 15 Jul '06

15 Jul '06

Let me re-formulate the example: saga::session *s = new saga::session (); saga::file *f = new saga::file (s, url); saga::task t = f->copy <saga::task::Task> (target); delete (s); delete (f); t.run (); That is the C++ version with explicit memory management, and boils down to the same problem, right? Now, that looks similar in Java I assume, but I know for sure its the same for Python, Perl and C: saga_task* t; saga_session s = saga_session_create (); saga_file f = saga_file_create (s, url); saga_task t = saga_file_copy_task (target); saga_session_destroy (s); saga_file_destroy (f); saga_task_run (t); It can be implemented, easily, in both ways: to be valid, and to be invalid. In the C example, assume that s, f and t are actually integers, which act as handles to the internal memory. If that internal memory is freed or not on ..._destroy is a SAGA policy, not a C memory management issue. If that code is valid or not is not a C/C++ question, it is a question the language independent SAGA spec must address. > "keeping objects alive" is a C++ thing. No, I disagree. I think you mean the memory management: that is an C++ thing here because it is an C++ example. If the user can expect the above code to work or not is not a C++ question, it needs to be addressed in the language independent spec. Well, and getting back to the original point: I would actually argue for that code to be valid, as it allows to pass task handles around, w/o burdening the application programmer to pass the objects, session and contexts around as well. Cheers, Andre. Quoting [Thilo Kielmann] (Jul 15 2006): > > > > Right, we agree. If there are more/other contexts that might still be OK. > > > IMHO, if the programmer is removing a context that is still needed, that is > > > a mistake that might lead to an error. > > > > Yes, we could do that, agree. However, we are already going > > a different route with sessions and tasks: > > > > -------------------------------------------------------- > > saga::task t; > > > > { > > saga::session s; > > saga::file f (s, url); > > > > t = f.copy <saga::task::Task> (target); > > } > > > > t.run (); > > // session and file object are gone here - however, the > > // task still lives, and keeps the session and object > > // resources alive. At least some state must be > > // maintained internally > > -------------------------------------------------------- > > > > As usual, that example is in C++, but the problem is > > independent of the language. > > :-) > > > > > I think that this is the same problem really as with the > > context. I think we should solve the the same way. > > Absolutely. > > > The problem really is, I think, that the above example is > > the dominating one. Tasks, as they represent async ops, > > are bound to get passed around through different scopes, > > and it would be burdensome to expect the programmer to > > keep all associated objects, sessions and contexts etc. > > alive. > > "keeping objects alive" is a C++ thing. > This means, both for contexts and sessions (and all other similar objects), > that the C++ language binding has to define a "object lifecycle policy" that > clearly states who (which class, e.g. user) is in charge of creating and > destroying objects. Part of this policy is to say whether or not objects > are copied when passed to another object, or whether they are used "by > reference". > > So far, C++. And now back to SAGA semantics. > > What I can see in the code example above is that there is no problem for the > SAGA operations. It is 100% a C++ problem: > a) if session and file are passed to the task by reference, then the above > code is simply incorrect > b) if session and file are copied to the task, this code is fine > > > Cheers, > > > Thilo -- "So much time, so little to do..." -- Garfield

1 0

Fwd (andre@merzky.net): Re: [saga-rg] context problem
by Thilo Kielmann 15 Jul '06

15 Jul '06

[[ just putting the thread back onto the list before answering ]] Thilo ----- Forwarded message from Andre Merzky <andre(a)merzky.net> ----- > Date: Fri, 14 Jul 2006 19:53:07 +0200 > From: Andre Merzky <andre(a)merzky.net> > To: Thilo Kielmann <kielmann(a)cs.vu.nl> > Cc: Andre Merzky <andre(a)merzky.net> > Subject: Re: [saga-rg] context problem > > The problem is not really C++ here I think. The code > snippet below is semantically challenging in any > language, and we need to define that semantic in the spec. > > The implementation in C++ is another issue, and simple :-) > > Making a copy of the context does not help. The problem, as > said, is that the 'remove_context' can get called before or > while that context is used, and that needs to have a well > defined semantic in the (lang independent) spec. > > Having it a user level error is an option. Well, not really > an error, but what you say (I think) is that the later > operations on that object cannot use the Globus context - > right? > > Cheers, Andre. > > > Quoting [Thilo Kielmann] (Jul 14 2006): > > > > Hi, > > > > I think this issue "nicely" confuses SAGA semantics and C++ memory management. > > > > From a SAGA viewpoint I would say: it is an error (mistake by the user) > > to remove a context from a session if this context is still to be used, > > e.g. with the task object in the example. > > > > Issues with C++ objects going out of scope when they are still needed > > have to be resolved within a C++ language binding. > > for example: s.add_context(c) could make a copy of the c object. > > Or: the add_context operation prescribes that the c object it got will > > not be destroyed elsewhere (and then the desctructor of s is in charge of > > freeing this memory) > > > > > > Having "remove context advisory" solves neither problem. > > As appliation programmer, I would be confused... > > > > > > Thilo > > > > On Fri, Jul 14, 2006 at 04:46:33PM +0200, Andre Merzky wrote: > > > Date: Fri, 14 Jul 2006 16:46:33 +0200 > > > From: Andre Merzky <andre(a)merzky.net> > > > To: SAGA RG <saga-rg(a)ggf.org> > > > Subject: [saga-rg] context problem > > > > > > Hi folx, > > > > > > I realised that we are running into some problems with our > > > current session and context handling. For example: > > > > > > > > > ------------------------------------------------------------ > > > saga::task t; > > > saga::session s; > > > > > > { > > > saga::context c (saga::context::GSI); > > > > > > s.add_context (c); > > > > > > saga::file f (s, url); > > > > > > t = f.copy <saga::task::Task> (target); > > > > > > s.remove_context (c); > > > } > > > // as it leaves the scope, the gsi context gets destroyed. > > > > > > t.run (); > > > // the task can not use the GSI context, although the file > > > // object and the copy task have been created in a session > > > // with a valid GSI context. > > > ------------------------------------------------------------ > > > > > > > > > The problem really is that, as a context gets removed, no > > > assumption is made about the fact that the session might > > > have active objects which are currently using that context. > > > > > > The example above is harmless - image a context getting > > > removed between two subsequent read calls, or while a task > > > is _running_. > > > > > > > > > I would propose to change the semantics of remove_context to > > > be advisory, i.e. to allow existing objects and tasks to > > > continue to use the context if (and only if) it was already > > > used by them before. The session SHOULD free context > > > related resources as soon as no object uses it anymore. > > > list_context() MUST NOT list the removed context anymore. > > > > > > We have similar mechanisms in place to keep sessions alive > > > if they are needed by objects, and to keep objects alive if > > > they are needed by tasks. So that would be a somewhat > > > coherent approach. > > > > > > The downside of course is that, at one more place, resource > > > deallocation is out of sync with the appearent object > > > destruction. > > > > > > Any opinion? > > > > > > Cheers, Andre. > > > > > > > > > > > > -- > > > "So much time, so little to do..." -- Garfield > -- > "So much time, so little to do..." -- Garfield ----- End forwarded message ----- -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

2 2