February 2006 - saga-rg - lists.ogf.org

Directory.close()?
by Graeme Pound 23 Feb '06

23 Feb '06

Looking at the strawman version 0.3 raised the following question. Graeme -3.32 Does SAGA.File.Directory require a close method? The close() method is required by SAGA.File.File, but I cannot see the purpose for this method in the Directory object.

2 1

Phone Call
by Andre Merzky 23 Feb '06

23 Feb '06

Hi, there was no more discussion about the phone call - so I the liberty to schedule it for today - lets see if anybody can make it (sorry Pascal). Here are the details. Cheers, Andre. Date: 23 February 2006 Time: 5:00 p.m. Central Europe Time Number: 0844.888.8888 (UK) 01805.004.102 (Germany) +44.870.088.5706 (others) Code: 808044 -- "So much time, so little to do..." -- Garfield

2 1

SAGA: streams and monitoring
by Andre Merzky 23 Feb '06

23 Feb '06

Hi guys, I treid to finalize monitoring in SAGA lately, and in particular the combination with streams is challengine, as there is a monitoring mechanism already present in streams (watch). Anyway, here is a code example for the current callback mechanism in the strawman API. Could you please give some feedback if that seem sensible to you, or if that would exclude important use cases etc? Thanks, Andre. Example for async stream server (C++): // my callback container class class my_cb : public saga::callback { privat: // we keep a stream server and a single client stream saga::stream_server ss; saga::stream s; public: // constructor initialises these (note that the // client stream should be not connected at this // point my_cb (saga::stream_server ss_, saga::stream s_) { ss = ss_; s = s_; } // the callback gets called on any bool callback (saga::metric m) { // the stream server got an event triggered, and // should be able to create a client socket now. s = ss.wait (); if ( s.status == saga::stream::open ) { // have a client stream, we are done // don't call this cb again! return (true); } // no valid client stream obtained: keep this // callback alive, and get called again on the // next metric event. return (false); } } int main () { // create a stream server, and an not connected // stream saga::stream_server ss; saga::stream s; // give both to our callback class, and register that // callback with the 'client_connect' metric of the // server. That causes the callback to be invoked on // every change of that metric, i.e. on every event // that changes that metric, i.e. on every client // connect attempt. my_cb cb (ss, s); ss.add_callback ("client_connect", cb); // now we serve incoming clients forever while ( true ) { // check if a new client is connected // the stream status would then be Open if ( s.status == saga::stream::Open ) { // a client got conncted! // handle open socket s.write ("You say hello, I say good bye!\r\n", 32); // and close stream s.close (); // the stream is not Open anymore. We re-add the // callback, and hence wait for the next client // to connect. ss.add_callback ("client_connect", cb); } else { // no client yet, idle, or do something useful sleep (1); } } // we should never get here return (-1); } Note that the example above looks long, but is only ~25 lines (nonempty/no-comment). Also note, that the callback get called immediately after getting added to the metric if a client connected to the ss before the cb got added - no client connections pass unhandled. That is similar to a stream server returning from waitForClient immediately if a client connected earlier. Also note that locking should be applied - there are race conditions if 2 clients connect at the same time. Well, its just an example... Cheers, Andre. class my_cb : public saga::callback { saga::stream_server ss; saga::stream s; public: my_cb (saga::stream_server ss_, saga::stream s_) { ss = ss_; s = s_; } bool callback (saga::metric m) { s = ss.wait (); if ( s.status == saga::stream::open ) { return (true); } return (false); } } int main () { saga::stream_server ss; saga::stream s; my_cb cb (ss, s); ss.add_callback ("client_connect", cb); while ( true ) { if ( s.status == saga::stream::Open ) { s.write ("You say hello, I say good bye!\r\n", 32); s.close (); ss.add_callback ("client_connect", cb); } else { sleep (1); } } return (-1); } -- "So much time, so little to do..." -- Garfield

1 0

Re: SAGA and Security
by Andre Merzky 21 Feb '06

21 Feb '06

Hi group, we managed to corner the Security Area ADs at GGF in Athens, and to get some statements from them in respect to: "What security paradigms are generically available in Grids, and what should be exposed to the end user?" Well, their answer was basically, that there is no agreed upon approach in the scope of GGF, so, the best we can do is to look at Grid implementations, and abstract/generalize their security paradigms. A viable approach in their opinion would be to base security settings on strings, and allow the implementation to interpret them accordingly. That approach is very close to what we have right now for sessions, and what we want to have for streams. Shantenu and I discussed that shortly, and would like to propose as follows: - for the time being, keep security out of the API where not absolutely necessary - where absolutely necessary (case by case), keep exposure of security paradigms simple and generic I think the notion of context that we have in the SAGA API fits that approach well: by default they are invisible. We would be happy to get comments, also from the Cc'ed Security ADs (hope we interpreted your answer correctly). Cheers, Andre & Shantenu. -- "So much time, so little to do..." -- Garfield

3 4

Conference calls
by Tom Goodale 20 Feb '06

20 Feb '06

Hi, was there any discussion of the conference calls at GGF ? When shall we have the next one, and at what time ? Cheers, Tom

3 2

More comments on Strawman API
by Graeme Pound 20 Feb '06

20 Feb '06

Andre, Here are some additional comments on the Strawman API (v0.2) that I have recovered from a HD failure. Would it be possible for you to clarify point 3.25, whereby bytes and strings are possibly confused in the API? Thanks, Graeme ######### Specific comments on the API -3.23 SAGA.File.openFlags should specify read/write modes -3.24 At what point should files be truncated, and to what length? -3.25 Do the operations of SAGA.File.File deal with byte or character arrays? The semantics of this in the API are inconsistent, arguments are described of type 'string', but the documentation discusses bytes. See comment 3.26. -3.26 The SAGA.File read() and write() methods specify arguments of type 'string' as the output/input parameter. However there is no way for the user to specify the character encoding of the file. Because of the unknown provenance of files located on the Grid the default character encoding for the local machine may be incorrect. Is it the intention for these methods to deal with strings/character arrays, or at the level of byte encodings? If SAGA.File read() methods return strings the user must be able to specify the character encoding for the file, if they return byte arrays this is not a problem. The character encoding could be specified as a property of the File class. -3.27 Should SAGA.File.File inherit the interface SAGA.NameSpace.NSEntry? -3.28 SAGA.NameSpace.NSDir.removeFlags.Recursive: This must be set to delete a directory, but how can a user delete a directory without deleting non-empty directories? -3.29 Many of the exceptions described in the specification document are fairly generic, for example the ubiquitous "NoSuccess". Will attempts be made to further characterise error conditions that may be expected to be common to many Grid platforms, such as "AuthenticationFailure" and "AuthorisationFailure". I understand the need to avoid an overload of throwable exceptions within the API, however these exceptions would be expected in a fairly well defined subset of the methods. -3.30 Details of NSDir.find() "the find operates recursively below the current working directory, unless the NoRecursive flag is specified" NoRecursive is not defined in the findFlags enumeration (instead Recursive is defined & default)

2 3

Job states proposal
by Christopher Smith 20 Feb '06

20 Feb '06

Ok ... here's my proposal for modelling the SAGA job states. There is a typical path through the states: New --> Pending --> Running --> Finished You could also submit with a hold, or put a Pending job on hold: New --> Hold <--> Pending You can suspend a job (and resume it): Running <--> Suspended The job can fail in execution: Running --> Failed And the job can be cancelled: Running --> Cancelled New --> Cancelled Pending --> Cancelled Suspended --> Cancelled All states can go to "Unknown". Let's not bother modelling User and System suspended and hold states, pending some more experience and let's not bother with all the pre/post states and file staging stuff pending experience and use cases, but see below on extended states. This model pretty much preserves what we had before (adding unknown, I believe), and it represents the "basic" SAGA state model. Now ... in order to have the ability to add states (say to support the BES state model), we can define a class for JobState that contains the BaseState (one of the enumerated states from above) and that has an ExtendedState attribute defined as a vector of strings. The job can then represent some of the extended states such as "Running PreExecution" or "Running StagingIn", etc, etc. By using the attribute mechanism, coders can get started with SAGA implementations now, and as we extend the state model according to experience and use cases, we merely need to specify acceptable strings, rather than change the underlying data structures. My $0.02 -- Chris

3 5

exceptions...
by Andre Merzky 20 Feb '06

20 Feb '06

Hi Folx, I stumbled over the exceptions in the spec again. Tried to find what we decided on in respect to exception classes, but could not find a definite statement in the mailing list archive nor in the issue list or meeting note. So I discussed with hartmut again - well, turned out his memory is a bad as mine ;-) What we did remember (or though to) is: - if an exception is fatal or not depends on the use case ("Can't suspend job" - fatal or not?) - a non-fatal backend exception could be recovered from by the saga implementation, and would hence not reach the application - so we don't need it. If we can't recover, it might well be fatal. - obviously bad things like 'not enough memory' don't need to be put into saga exceptions - have nothing to do with saga semantics. So, I/we _think_ the agreement was to have only one simple saga::exception class which gets thrown whenever something happens which violates the semantic contract of a saga call. If that can be recovered from or not is up to the application. If there are no comments to the contrary within a week, and if nobody has better search or meeting notes, I'll update the spec accordingly. Cheers, Andre. -- "So much time, so little to do..." -- Garfield

3 8

is_supported on attributes
by 'Andre Merzky' 20 Feb '06

20 Feb '06

Hi all, in Athens we discussed the inspectionb on attributes, and suggested to include 'is_supported' for inspection. That would allow to check if an implementation can handle a specific attribute, e.g. can set tcp buffer size. On a closer look at our implementatin we realized that we can't sensibly implement is_supported, as we do not know to what middleware we will eventually be bound at the time of the method invokation. Consider: { saga::stream s; if ( s.is_supported ("buffer_size") ) { s.set_attribute ("buffer_size", "1024"); } // 'til here adaptor independend - cannot say anything // about supported attribs. The above test is useless. s.read (100, buff); // only on the method call an error could be thrown - we // now bind to a middleware, and know if that supports // buffer_size. But that error is very much delayed, and // seems unrelated right here... } So we would like to propose to leave is_supported out of the interface, and instead define supported attributes in the spec explicitely (as we do anyway). An implementation SHOULD then document any deviation, e.g. what attributes are ignored desprite beeing defined in the spec. An application SHOULD be allowed to use any attrib defined in the spec, in order to allow portability between SAGA implementations. A runtime warning CAN be used to inform the application about ignored attributes. Comments welcome! :-) Cheers, Andre. -- "So much time, so little to do..." -- Garfield

1 0

Secretary for SAGA-WG...
by Andre Merzky 19 Feb '06

19 Feb '06

Hi all, Hartmut volonteered to act as secretary for the SAGA WG! I think that this is a very welcome commitment! :-) I there are no objections raised on this list within a week, I will update the official records and inform ggf office about the change. Thanks, Andre. -- "So much time, so little to do..." -- Garfield

2 1