August 2006 - saga-rg - lists.ogf.org

"POSIX consistency"
by Thilo Kielmann 10 Aug '06

10 Aug '06

I have just completed the reorganization of the "general design considerations" section. This is a re-arrangement for the purpose of better presentation and understanding. I hope I did not change any content. I have guessed some motivation why error conditions and callbacks should be excluded from deep object copies. Maybe someone can check? Then, I have flagged the text about "POSIX consistency" with a FIXME. As it stands, I simply do not understand what this is about. What kind of operations (and their consistency) is meant with this??? This is all I can do today. Regards, Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

2 1

status of requirements document?
by Thilo Kielmann 10 Aug '06

10 Aug '06

All, when looking on gridforge in the documents in the editor pipeline, I find our requirements document in status "Author Reviewing Edits". Does this mean it is not returned to Greg Newby yet??? Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

2 1

the CVS again :-(
by Thilo Kielmann 09 Aug '06

09 Aug '06

cvs [update aborted]: end of file from server (consult above messages if any) cvs [update aborted]: reading from server: Connection reset by peer Should I have bribed the the admins? ;-) Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

2 1

FW: [cct.lsu.edu #2357] FW: [saga-rg] cvs.cct.lsu.edu not responding :-(
by Hartmut Kaiser 09 Aug '06

09 Aug '06

> -----Original Message----- > From: Ravi Paruchuri via RT [mailto:internal@ticket.cct.lsu.edu] > Sent: Tuesday, August 08, 2006 12:19 PM > To: hkaiser(a)cct.lsu.edu > Cc: afrench(a)cct.lsu.edu; lohitha(a)cct.lsu.edu; > mnoel(a)cct.lsu.edu; rousseau(a)cct.lsu.edu; twright(a)cct.lsu.edu > Subject: Re: [cct.lsu.edu #2357] FW: [saga-rg] > cvs.cct.lsu.edu not responding :-( > > Hi Hartmut, > > We took down CVS server as we got reports about corruption on > there. We are looking in to the issue and we will have cvs > server back up shorlty. > > Thanks, > Ravi > > Hartmut Kaiser via RT wrote: > > >Tue Aug 08 12:06:54 2006: Request 2357 was acted upon. > >Transaction: Ticket created by hkaiser > > Queue: Internal > > Subject: FW: [saga-rg] cvs.cct.lsu.edu not responding :-( > > Owner: Nobody > > Requestors: hkaiser(a)cct.lsu.edu > > Status: new > > Ticket <URL: > https://ticket.cct.lsu.edu/Ticket/Display.html?id=2357 > > > > > > > > >Hi all, > > > >Could please somebody look into this issue? > > > >BTW: I haven't got any CVS commit emails from the VU-SAGA repository > >lately, is that a related issue? > > > >Regards Hartmut > > > > > > > >>-----Original Message----- > >>From: owner-saga-rg(a)ggf.org [mailto:owner-saga-rg@ggf.org] > On Behalf > >>Of Thilo Kielmann > >>Sent: Tuesday, August 08, 2006 12:02 PM > >>To: saga-rg(a)ggf.org > >>Subject: [saga-rg] cvs.cct.lsu.edu not responding :-( > >> > >>I'd like to do my homework according to totday's phone > call, but the > >>cvs server won't talk to me :-( > >> > >>Could somebody (@LSU?) please have a look? > >> > >> > >>Thanks, > >> > >> > >>Thilo > >>-- > >>Thilo Kielmann > >>http://www.cs.vu.nl/~kielmann/ > >> > >> > >> > > > >_______________________________________________ > >Internal-consult mailing list > >Internal-consult(a)cct.lsu.edu > >https://mail.cct.lsu.edu/mailman/listinfo/internal-consult > > > > > > > > -- > Ravi Paruchuri > IT Manager > Center for Computation & Technology > 232 Johnston Hall > Louisiana State University > Phone: 225-578-5890(Work) > 225-610-9557(Cell) > Fax: 225-578-5362 > AIM: raviparuchuri > > >

2 1

cvs.cct.lsu.edu not responding :-(
by Thilo Kielmann 08 Aug '06

08 Aug '06

I'd like to do my homework according to totday's phone call, but the cvs server won't talk to me :-( Could somebody (@LSU?) please have a look? Thanks, Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

2 1

SAGA conf call - Tuesday 08 August
by Shantenu Jha 08 Aug '06

08 Aug '06

Here are the call details: Date: Tuesday 08 August, 2006 2:00 pm London Time: 3:00 pm Amsterdam 8:00 am Baton Rouge 10:00 pm Tokyo Number: 0844.888.8888 (UK) 01805.004.102 (Germany) +44.870.088.5706 (Others) Access Code 97288 Agenda to be posted soon.

2 3

Re: Fwd (replogle@ggf.org): [gf-chairs] GGF18 Session Scheduling: Deadline 7 July
by Andre Merzky 06 Aug '06

06 Aug '06

Hi groups, I just realized that I can wait forever on answers on mails which I send only to myself, and not to the lists... Sorry for that! Anyway, we have now 2+2 sessions scheduled for GGF18, with somewhat fuzzy agendas. Please let us know if you are interested to see specific topics on the agendas. Cheers, Andre. Quoting [Andre Merzky] (Jun 26 2006): > Date: Mon, 26 Jun 2006 09:49:43 +0200 > From: Andre Merzky <andre(a)merzky.net> > To: saga-rg(a)merzky.net, saga-core-wg(a)merzky.net > Subject: Fwd (replogle(a)ggf.org) [gf-chairs] GGF18 Session Scheduling: Deadline 7 July > > Hi group(s), > > its time again to plan for the sessions at GGF18. Do you > guys think that the 2+2 scheme from ggf17 worked well? Do > we want to repeat that pattern? > > For the Core API spec, we don't have major open issues > anymore (some minor ones though), so that won't eat up much > time. However, we should devote some time to the CookBook > and user manual, and probably again to the implementations. > > In the RG, the sessions topics depend, I guess, from the > status of the various API suggesstions (RPC, Info, > Messaging) - I am somewhat unsure about their status by > then. > > Any thoughts? > > Cheers, Andre. > > > > ----- Forwarded message from Joel Replogle <replogle(a)ggf.org> ----- > > > To: gf-chairs(a)ggf.org > > From: Joel Replogle <replogle(a)ggf.org> > > Subject: [gf-chairs] GGF18 Session Scheduling: Deadline 7 July > > Date: Sun, 25 Jun 2006 22:53:05 -0400 > > > > GGF Chairs - > > > > The next GGF event, GGF18, is coming up and session scheduling is now > > open! GGF18 will be held in conjunction with GridWorld September > > 11-14, 2006 in Washington, DC USA. > > > > Please go to the following link for Group Session Requesting > > instructions and the link to the request form: > > http://www.ggf.org/ggf_events_next_chairinfo.htm > > > > The scheduling deadlines are: > > > > Draft Documents: > > - Deadline for submitting working drafts is 11 August, 2006. > > Session Requests: > > - Deadline for session requests for a CG, WG or RG is 7 July, 2006 > > - Deadline for AD Hoc/Charter Discussion BOFs is 7 July, 2006. > > > > We anticipate that the schedule will be finalized by the last week in > > July. > > > > Thanks, > > > > Joel > > > > ------------------------------------------------------------------------ > > ----------- > > Joel Replogle - Manager of Standards, Global Grid Forum > > replogle(a)ggf.org http://www.ggf.org > > ----- End forwarded message ----- -- "So much time, so little to do..." -- Garfield

2 2

Fwd (kielmann@cs.vu.nl): Re: [saga-rg] SAGA, object states, and concurrency
by 'Thilo Kielmann' 04 Aug '06

04 Aug '06

All, the (my) aim of the current discussion is to separate SAGA-related issues from language-specific issues. SAGA-related issues belong to the SAGA spec, the others to specific language-bindings docs. In this context, I am concerned about sections 1.3.6 Object State 1.3.7 Life Time Management 1.3.8 Freeing of Resources and Garbage Collection about 1.3.6: it is not clear (to me ;-) why prev. error condition and callbacks MUST NOT get copied can anybody give a motivation for this, please? (and include it in the document) about 1.3.7.: as I wrote earlier, this should simply become: > The SAGA specification does NOT address issues of object life cycle > and memory management at all. It is subject to the language bindings > of SAGA to define this in a way that suits the respective programming > languages. about 1.3.8: I am worried bout the last sentence. Please make a link to section 1.5 Otherwise, I am afraid, this might open the door to "weakly compliant" implementations. Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/

2 7

Fwd (andre@merzky.net): Re: Fwd (andre@merzky.net): Re: Fwd (andre@merzky.net): Re: [saga-rg] context problem
by Andre Merzky 02 Aug '06

02 Aug '06

[damned, majordomo seems really broken - forward to the list again] ----- Forwarded message from Andre Merzky <andre(a)merzky.net> ----- > Date: Sun, 16 Jul 2006 19:38:54 +0200 > From: Andre Merzky <andre(a)merzky.net> > To: Thilo Kielmann <kielmann(a)cs.vu.nl> > Cc: Andre Merzky <andre(a)merzky.net> > Subject: Re: Fwd (andre(a)merzky.net) Re: Fwd (andre(a)merzky.net) Re: [saga-rg] context problem > > Quoting [Thilo Kielmann] (Jul 16 2006): > > > > Merging 2 mails from Andre: > > > > > very good points, and indeed (1) seems cleanest. However, > > > it has its own semantic pitfalls: > > > > > > saga::file f (url); > > > saga::task t = f.write <saga::task::Task> ("hello world", ...); > > > > > > f.seek (100, saga::file::SeekSet); > > > > > > t.run (); > > > t.wait (); > > > > > > > > > If on task creation the file object gets copied over, the > > > subsequent seek (sync) and write (async) work on different > > > object copies. In particular, these copies will have > > > different state - seek on one copy will have no effect on > > > where the write will occur. > > > > I cannot see a problem here: With object copying, you will simply have the > > same file open twice. And given the operations you do, this might even be > > the right thing... > > This example is very academic: can you show an example where the sharing of > > state between tasks is useful, actually? > > The problem here is, that I at a user would expect the write > to happen at byte 100, but it will happen at byte 0: the > seek happens on a different object than the write. > > What might be a more obvious example, which goes wrong along > the same lines: > > f.write ("line 1\n"); > f.write ("line 2\n"); > f.write ("line 3\n"); > > That will result in a file > > line 1 > line 2 > line 3 > > whereas the coed > > saga::task t1 = f.write ("line 1\n"); t1.run (); t1.wait (); > saga::task t2 = f.write ("line 2\n"); t2.run (); t2.wait (); > saga::task t3 = f.write ("line 3\n"); t3.run (); t3.wait (); > > will result in a file > > line_3 > > the last write will start on 0, as the previous write > operated on a different file pointer. In general, you > cannot execute any two tasks on a single object, at least > not if any state is of concern, such as file pointer, pwd, > replica name, stream server port, job id, ... > > That is a no-go in my opinion, as it is counter-intuitive, > and breaks a large number of use cases. And is incosistent > with the syncroneou method calls. > > Yes, you can wreak havoc with state as well: > > saga::task t1 = f.write ("line 1\n"); > saga::task t2 = f.write ("line 2\n"); > > t1.run (); > t2.run (); > > t1.wait (); > t2.wait (); > > will likely result in > > linline 2 > e 1 > > or such - the user does need to think when doing multiple > async ops at once. I don't see a way around that (and don't > see a need for it either: we want to make the Grid stuff > easy, but not revolutionize programming styles). > > > > > I should have added that I'd prefer 3: > > > > > > > > 3. when creating a task, all parameter objects are passed "by reference" > > > > > + no enforced copying overhead > > > > > - all objects are shared, lots of potential error conditions > > > > > > The error conditions I could think of are: > > > > > > - change state of object while a task is running, hence > > > having the task doing something differently than > > > intended > > > Change of state, > > That is intentional - see above. > > > > like destruction of objects > > Well, that is what we discuss :-) 3 would delay destruction > until its save (state is not needed anymore). > > > > or change of objects. > > What doe you mean here? > > > > Not to speak of synchronization conditions: supposed you > > have non-atomic write operations (which is everything that > > writes more than a single word to memory): do you thus > > also enforce object locking by doing this? > > If not, you can have inconsistent object state that can be > > seen by one task, just because another task is halfway > > through writing the object... (all classical problems of > > shared-memory communication apply) > > See above. You are right, but I don't see a way around > that, without causing more harm than good (child and bathtub > come to my mind for some reason...). > > BTW: the bulk optimization we have now assumes that tasks > which run at the same time are, by their very definition, > independent from each other, do not depend on any specific > order of execution, and do not depend from each other in > respect to object state. That are the very points we talk > here about - I think its a very sensible assumption. I have > the same behaviour on a unix shell BTW: > > touch file > date >> file & > date >> file & > > I would not be able to make assumptions about the file > contents... (well, here I could make a save bet, but you > know what I mean). > > > > > - limited control over resource deallocation > > > > this is the same thing as above > > > > The problem really is that there is no "object lifecycle" > > defined. There is no way to define which task or thread > > might be responsible or even allowed to destroy objects or > > change objects. Is it??? > > Yes, that is what I mean with limited control. > > We had a discussion on this list and in Tokyo about the > semantics of cancel(), which touches the same problem: > should task.cancel() block until resources are freed? As we > might talk about remote resources, and Grids are unreliable, > we might block forever. That does not make sense, at least > not always. > > The resolution we came up with is that cancel() is advisory, > so non-blocking, but can also use a timeout parameter (with > -1 meaning forever) to block until resources are freed. > > Timeouts do not make sense on destructors I believe, but > 'advisory destruction' does, IMHO. > > > > > The advantages I see: > > > > > > - no copy overhead (but, as you say, that is of no > > > concern really) > > > > ok, but minor point. > > right. Lets forget that from now on. > > > > > - simple, clear defined semantics > > no, it is the the most dangerous of the three versions > > Well, see above - I think its the most sensible semantics > :-) > > > > > - tasks keep objects they operate on alive - > > > objects keep sessions they live in alive - > > > sessions keep contexts they use alive > > > > what is the maening of "alive" here??? Now that you have > > outruled memory management... > > see above: resources get freed if not needed anymore. > > > > > > - sync and asyn operations operate on the same > > > object instance. > > > > Let's forget about "sync" here: it is the task that is > > running in the current thread, so multiple tasks share > > object instances. > > Well, it would be nice to have same semantics for sync and > async, don't you think? :-) > > > > > Either way (1, 2 or 3), we have to have the user of the > > > API thinking while using it - neither is not idiot > > > proof. > > > > Well, we should strive to limit the mental load on the > > programmer as much as possible... > > > > > I think (2) is most problematic, if I understant your > > > 'hand-over' correctly: that would mean you can't use the > > > object again until the task was finished? > > > > No, it means you will never ever again be allowed to use > > these objects. (hand over includes the hand over of the > > responsibility to clean up...) > > Right. So you can never do a async read, and then a sync > seek, and then a async read again. At least not with > sensible results. > > Also, I need to create 100 file instances to do 100 reads? > Remember that opening a file is a remote op in itself, > potentially. Then we don't need the task model anymore. > > That is broken IMHO. > > Cheers, Andre. > > > > Thilo > > -- "So much time, so little to do..." -- Garfield ----- End forwarded message ----- -- "So much time, so little to do..." -- Garfield

7 22