February 2006 - saga-rg - lists.ogf.org

Re: SAGA and Security
by Andre Merzky 19 Feb '06

19 Feb '06

Hi Dane, I hope you recovered from Athens :-) How should we proceed on the SAGA security front? Should we try to thrwo some more use cases at you right now, or should we first try to solve tghose which are on the table? We had the chance to talk to some GFS folx in Athens before we left - they encouraged us to go for ACLs for file permissions. Would you agree that ACLs are likely to be _the_ way to go for file permissions? Would that also be ok for replica systems and information systems? (SAGA has a notion of name_spaces, which are used for information systems, logical files and physical files). Cheers, Andre. Quoting [Dane Skow] (Feb 14 2006): > Cc: SAGA RG <saga-rg(a)ggf.org>, mulmo(a)pdc.kth.se > From: Dane Skow <skow(a)mcs.anl.gov> > Subject: Re: SAGA and Security > Date: Tue, 14 Feb 2006 11:29:15 +0200 > To: Andre Merzky <andre(a)merzky.net> > > > Groan, I just realized that my presentation at the GT4 workshop is at > 10:30 session on Wednesday AM so a 3-way conflict. I don't think > either Olle or I would be able to make that session. I think Anne's > case below is an interesting example to walk through. Let's continue > this thread in email. > > Dane > > On Feb 14, 2006, at 12:31 AM, Andre Merzky wrote: > > >Hi Dane, > > > >thanks for the remarks and corrections - I should have known > >that its not _that_ easy as I thought ;-) It would be > >perfect to have you at the Wednesday session! > > -- "So much time, so little to do..." -- Garfield

1 0

group mailing list..
by Shantenu Jha 17 Feb '06

17 Feb '06

I think Andre means GridRPC and not GrudRPC.... That aside my $0.02: Having one list seems the best way forward. Any objections (aesthetic or technical), shout now, or forever stay confined to saga-rg(a)ggf.org Shantenu Date: Fri, 17 Feb 2006 17:26:32 +0200 From: Andre Merzky <andre(a)merzky.net> To: SAGA RG <saga-rg(a)ggf.org> Subject: group mailing list... Hi, during GGF, we also discussed the use of the mailing list. In _theory_ we should have two mailing lists, one for the RG, and one for the new WG. However, we propose to continue to use this mailing list for both groups until the need for a separate WG mailing list arises. Additionally, we want to encourage people to subscribe to the GrudRPC and GridCPR mailing lists, where the respective discussions of the rpc and cpr design temas will take place (see meeting notes). gridrpc-wg(a)ggf.org gridcpr-wg(a)ggf.org Cheers, Andre. -- "So much time, so little to do..." -- Garfield

1 0

GGF 16 SAGA Session 4 Notes
by Shantenu Jha 16 Feb '06

16 Feb '06

GGF16 (Athens), SAGA-RG Session #4, Wed 13:30-15:00 http://wiki.cct.lsu.edu/saga/space/GGF16+Meeting+Notes SAGA-Interoperability Session ============================= - Presentation by Malcolm Illingworth (EPCC) on DEISA's Java implementation (JRA7 work package) - Second presentation by Andre Merzky on CCT, LSU and Vrije University's implementation using C++ - Third presentation/update by Steven Newhouse (SN) on OMII, UK implementation in Java. See SAGA Wiki and gridforge pages for a PDF of the first two presentations. https://forge.gridforum.org/projects/saga-rg/ http://cs.cct.lsu.edu/saga/ Here are the main points from OMII's implementation. - OMII's Java SAGA implementation - Will be synced upto the current SAGA spec (As of summer 2006) - OMII's implementation will be OpenSource - Java SAGA API into the BES space (OGSA) - Currently, implementation binds to low level infrastructure but will also bind to a Gridsam implementation Implementor's Feedback: - Wants task issue resolved - Streaming status unclear? - Some areas are less tested e.g., logical files - SN inquired about unit tests/compatability test by other implementations. There should be some scope for collaboration with at least the other Java implementation.

2 1

GGF16 SAGA session 3 Notes
by Shantenu Jha 16 Feb '06

16 Feb '06

GGF16 (Athens), Wed 15 Feb, SAGA Session #3, 10:30-12:00 http://wiki.cct.lsu.edu/saga/space/GGF16+Meeting+Notes Summary: Session devoted to discussing main aspects of the Requirement Document (in progress, close to delivery) as well as address the "next Top 10" on the issue list. Discussed the the structure of the evolving Requirement document , highlights of the document and issue list. Some feedback on the Requirement Document: - "Efficient data access... supported by the SAGA API" Need to say what is being said better Also add more text to the description.. " Data Access should be supported by the SAGA API which in a way does not preclude efficient access" - "Data Replication" Clarify. - "Information Services" Suprised (yet again!) by the number of Use Cases requesting IS This needs to be investigated further by Andre+Shantenu - "Streams" OK - "Events and Steering" OK Non-Functional Areas. ==================== - Transactions are difficult to do... - Gridlab: Pg 15 - Clarify in text remarks about whether it is for Middleware technology or middleware implementation. ** Deiter K offered to be persuaded to extract more Use Cases from ** EGEE/gLite. Jha to follow up** Other General comments on the Requirements Doc: ============================================= - Conclusions section requires work - Steven Newhouse: What are the next steps? Should be ready for submission/feedback in the next two weeks. - Suggestion for the 'doc snapshot' to be placed on the Wiki [ Probably not until the stable version is ready; till then please rely on CVS] ============================= ============================= Second half of SAGA session#3 Most responses/discussions have been entered into the Issue list, which can be found from the SAGA CVS respository. Issue list: ========= - issue 26 URL problem: any solution? allow any, 'any://'? Do we have an "upward compatibility" path with GFS? AM: We think we are fine.. SN: "Seems Dangerous to Expect that level of intelligence at the API Level". Needs further discussion Complexity vs completeness? - issue 26) why not on shell wildcards? windows does support "*" -- not as powerful as linux will there be wildcard specification in GFS? therefore defer to GFS solution - issue 50) Discussion from BES: Two stage cancel. AM: proposed a sub-state model - Issue 64/68 Agreement on the fact that it should be allowed to return immediately. - Issue 120) Accept Hartmut's solution! [see issue list] - Issue 110) Attach object's session to the error constructor.. - Issue 108) Some global error state... will help auditing etc., "is supported" as an attribute..

1 0

GGF16 SAGA Session 2 meeting notes
by Shantenu Jha 16 Feb '06

16 Feb '06

GGF16 (Athens), Tue 14 Feb, SAGA Session #2, http://wiki.cct.lsu.edu/saga/space/GGF16+Meeting+Notes Summary: The second SAGA session was devoted entirely to tackling the "Top 10" critical items on the issue list. The "Top 10" items were selected primarily by Andre in consultation with Hartmut and Shantenu. Further detailed responses/discussion to the "issues" can be found on the "issue list" document from the SAGA CVS repository. - issue 25] Do we need 'stat'? Why not just use unix/posix stipulations? AM: What is group in globus? Chris: Why not ACL? SHOULD be some abstract data type? Q: Do we have a notion of "permanent VO" Asked differnently, apart from security, is there a need Defer it from the saga-core spec(?!) and leave it till there is a grid file system *** Hartmut to be reminded to generate a good use case for logical Files *** - Issue number 63: option 1: handling task...? option 2: casting? option 3: user has to maintain the list option 4: add templates?! Add something which makes it easier for the user Need to think about inspection (independently) Q (Hartmut): Do we have a common base class? No. - Issue 71: Should we include resource decovery? service discovery? No. - Issue 78: enums and scoping: clashes in inheritance **** This is not a SAGA issue. - Issue 79: Should jobs implement the task interface? Have a job as it was; make interface more equal Have task inherit job interface - Yes, and its useful for some of the use cases received. - Issue 86: Job info object? Keep in mind that someday SAGA may be an inteface for OGSA? Job status as a different class Attributes let you to extend attrbute without modifying the interface Keep getters and setters to language bindgs, although one can replace the other.... - issue 99: Should package names be reflected in the API or be leaft for language binding discussion(s) See "issue list" for discussion - Issue 115: Review monitoring Should callbacks have an optional ttl (wait like) There is an application level trivial and simple solution e.g. timer -- supported by every OS Hence refrain from introducing at the API level..

1 0

GGF16 saga session 1 notes
by Shantenu Jha 15 Feb '06

15 Feb '06

SAGA-WG/RG meeting at GGF16, Session 1, Monday 13th, 17:30-19:00 Notes taken by Thilo Kielmann Andre opens the session, explains the IPR policy and passes around the sign-up sheet Agenda bashing -------------- The following agenda is proposed and agreed by the audience: 0. Quick recap of the re-chartering status. 1. Discuss and agree on: - SAGA-RG Charter - SAGA-Core WG Charter 2. Next steps for the new SAGA-RG (Discussions about SAGA API details and the documents in preparation will be held in the other SAGA meetings later this week.) 1. Rechartering --------------- The "old" SAGA-RG split into two separate groups. A research group (SAGA-RG), has been tasked with taking care of research-like aspects such as API look-and-feel and areas for API scope extension. A working group (SAGA-CORE-WG) will work towards an API standard and language bindings based on the work done so far. The rechartering is expected to be approved by GFSG the same day. [Note added: And so it was; see posting on saga-rg(a)ggf.org list] A) Research group Andre reads the charter to the audience. No objections were made. B) Working group Andre reads the charter to the audience. No objections were made. 2. Next steps for the new SAGA-RG --------------------------------- The discussion revolved around identifying areas/functionalities for API scope extensions. Obvious candidates for scope extension are GridRPC, GridCPR, and resource discovery. A start to be made at this GGF. It was emphasized that the poll for additional use cases is still open. It is expected that occasionally a new use-case-document will be produced that will cover those new use cases and the scope extensions resulting from them. The GridCPR group was discussed first: - Has use cases - Has architecture doc (may serve as an API requirements doc) - Both documents are already beyond public review, this gives us stable input - We should now form a design team to define a strawman SAGA-CPR API - Defining the API is GridCPR's next milestone anyway - We will try to use a SAGA look and feel - The gridcpr mailing list should be the right place to the design team - Tom Goodale is being volunteered (with Derek Simmel and Paul Stodghill) to draft a SAGA-CPR strawman - Timeline: have a session on that at GGF17 The GridRPC group was discussed next: - has already API and user group - needs/wants to align with SAGA still - SAGA-RPC should be established, but no volunteers were identified yet There was a discussion of the needs of ASTRO-RG: remote data access - remote, format-dependent data access - data formats, probably a very hard problem (HDF, netCDF) - also remote data bases? - maybe beyond SAGA scope Common look and feel is another issue for the new SAGA-RG. Possible topics covered by this: - security, but security group doesn't know yet either what to put into an API - quality of service, also little confidence among the SAGA folks how to get this right - service discovery, resouce discovery, life time management - lifetime of security handles - persistent information storage - application-level information service - application steering, needs auditing, tracking - currently has notification, we need self-describing events (-> steering?) - message communication on top of streaming Bottom line (rough consensus): - the current strawman API is close to completion (will be ready for the next ggf) - we will need further input (and use cases!) for extension topics The next SAGA session at GGF16 will cover more WG and API related topics. 19:00 session is closed

1 0

RE: [saga-rg] job states...
by Rajic, Hrabri 14 Feb '06

14 Feb '06

Other way around :-). DRMAA does not have explicit support for file transfers, so its state diagram is missing file staging. The original DRMAA-WG members' intent was to specify something that most of the DRM systems currently support, possibly a bit more to have a lower entry implementation bar. That has eliminated the pre and post execution stages. Cleaning up and shutting down stages were never discussed. Hrabri >-----Original Message----- >From: owner-saga-rg(a)ggf.org [mailto:owner-saga-rg@ggf.org] On Behalf Of >Thilo Kielmann >Sent: Tuesday, February 14, 2006 12:53 AM >To: Rajic, Hrabri >Cc: Simple API for Grid Applications WG >Subject: Re: [saga-rg] job states... > >Pardon me, BES is not for Grids??? > >Thilo > >On Mon, Feb 13, 2006 at 10:14:07AM -0800, Rajic, Hrabri wrote: >> >> They are not. Grid is a richer environment. >> >> Hrabri >> >> >-----Original Message----- >> >From: owner-saga-rg(a)ggf.org [mailto:owner-saga-rg@ggf.org] On Behalf Of >> >Thilo Kielmann >> >Sent: Sunday, February 12, 2006 1:09 AM >> >To: Andre Merzky >> >Cc: Christopher Smith; Simple API for Grid Applications WG >> >Subject: Re: [saga-rg] job states... >> > >> >Curious question: >> > >> >SAGA should align job states with both BES and DRMAA ? >> >Are they the same to begin with? >> > >> >Thilo >> > >> >On Sat, Feb 11, 2006 at 03:54:16AM +0100, Andre Merzky wrote: >> >> X-Original-To: kielmann@localhost >> >> Delivered-To: kielmann(a)localhost.cs.vu.nl >> >> Delivered-To: grdfm-saga-rg-outgoing(a)mailbouncer.mcs.anl.gov >> >> X-Original-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> >> Delivered-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> >> Date: Sat, 11 Feb 2006 03:54:16 +0100 >> >> From: Andre Merzky <andre(a)merzky.net> >> >> To: Christopher Smith <csmith(a)platform.com> >> >> Cc: Andre Merzky <andre(a)merzky.net>, >> >> Simple API for Grid Applications WG <saga-rg(a)ggf.org> >> >> Subject: Re: [saga-rg] job states... >> >> X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at >> >mailbouncer.mcs.anl.gov >> >> X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at >> >mailbouncer.mcs.anl.gov >> >> >> >> I agree - the file transfer state models are needed for SAGA. >> >> We don't have any actions on these states anyway. >> >> >> >> Andre. >> >> >> >> >> >> Quoting [Christopher Smith] (Feb 11 2006): >> >> > >> >> > Sure. >> >> > >> >> > As mentioned ... I think maybe supporting a subset of BES is ok. >> Much >> >of the >> >> > state model wrt file transfer state modelling I think is not >> required >> >for >> >> > SAGA. >> >> > >> >> > -- Chris >> >> > >> >> > >> >> > >> >> > On 10/2/06 18:46, "Andre Merzky" <andre(a)merzky.net> wrote: >> >> > >> >> > > Ok, then I'll do that in the strawman. I would appreciate >> >> > > if you could glance over it after commit, for a sanity >> >> > > check. >> >> > > >> >> > > Thanks, Andre. >> >> > > >> >> > > >> >> > > Quoting [Christopher Smith] (Feb 11 2006): >> >> > >> >> >> > >> It makes sense to keep the state models in sync. >> >> > >> >> >> > >> -- Chris >> >> > >> >> >> > >> >> >> > >> On 10/2/06 18:26, "Andre Merzky" <andre(a)merzky.net> wrote: >> >> > >> >> >> > >>> Quoting [Christopher Smith] (Feb 11 2006): >> >> > >>>> >> >> > >>>> What I meant by that comment is that where it is a subset, it >> >should >> >> > >>>> reflect >> >> > >>>> the BES terminology. I think that the number of states >> represented >> >is >> >> > >>>> enough >> >> > >>>> already. ;-) >> >> > >>> >> >> > >>> Would it make sense to just copy the BES state diagram? >> >> > >>> >> >> > >>> It did not exist when we (== you ;-) drafted the SAGA job >> >> > >>> states - if it would have been around then, we might have >> >> > >>> had copied it already. >> >> > >>> >> >> > >>> Apart from the SystemXXX/UserXXX states, and from Hold, >> >> > >>> it is not that much different from the SAGA model anyway. >> >> > >>> >> >> > >>> Cheers, Andre. >> >> > >>> >> >> > >>> >> >> > >>>> -- Chris >> >> > >>>> >> >> > >>>> >> >> > >>>> On 10/2/06 17:30, "Andre Merzky" <andre(a)merzky.net> wrote: >> >> > >>>> >> >> > >>>>> Hi Chris, >> >> > >>>>> >> >> > >>>>> many thanks for the answers! :-) >> >> > >>>>> >> >> > >>>>>> By the way ... I believe that the state diagram should at >> least >> >be a >> >> > >>>>>> subset >> >> > >>>>>> of the BES state diagram ... we should adopt the same names. >> >> > >>>>> >> >> > >>>>> I agree, kind of - I would say that the SAGA job state >> >> > >>>>> diagram should at _most_ be subset of the BES state diagram. >> >> > >>>>> It could be _S_implier :-) >> >> > >>>>> >> >> > >>>>> Cheers, Andre. >> >> > >>>>> >> >> > >>>>> >> >> > >>>>> Quoting [Christopher Smith] (Feb 10 2006): >> >> > >>>>>> Date: Fri, 10 Feb 2006 13:41:18 -0800 >> >> > >>>>>> Subject: Re: [saga-rg] job states... >> >> > >>>>>> From: Christopher Smith <csmith(a)platform.com> >> >> > >>>>>> To: Simple API for Grid Applications WG <saga-rg(a)ggf.org> >> >> > >>>>>> >> >> > >>>>>> On 4/2/06 11:18, "Andre Merzky" <andre(a)merzky.net> wrote: >> >> > >>>>>> >> >> > >>>>>> Ok ... I'll try to answer these, at least from my viewpoint. >> >> > >>>>>> >> >> > >>>>>>> >> >> > >>>>>>> I think that diagram is wrong, isn't it? Well, here are my >> >> > >>>>>>> questions: >> >> > >>>>>>> >> >> > >>>>>>> - if we submit a job, its immediately Queued - is that >> >> > >>>>>>> right? Should it be pending before (e.g. as long as >> the >> >> > >>>>>>> queuing request travels the middleware layers)? >> >> > >>>>>>> >> >> > >>>>>> To me, Queued is the same as Pending. Pending is probably a >> >better word >> >> > >>>>>> for >> >> > >>>>>> this. Can't remember where the Queued name came from, as LSF >> >uses PEND. >> >> > >>>>>> >> >> > >>>>>>> - can the hold and suspend states reached only from >> >> > >>>>>>> 'Running', or from elsewhere as well? >> >> > >>>>>>> >> >> > >>>>>> You can only go into a Hold state from Pending, I think, or >> >directly into >> >> > >>>>>> Hold on submission. >> >> > >>>>>> >> >> > >>>>>>> - What is the difference between 'Hold' and 'Suspend'? >> >> > >>>>>>> >> >> > >>>>>> A Hold state tells the scheduler/broker not to consider this >> job >> >for >> >> > >>>>>> scheduling/dispatch until the hold is explicitly released. >> >> > >>>>>> >> >> > >>>>>>> - Are there signals defined (apart from KILL) which >> shange >> >> > >>>>>>> the job state? I guess that is not as simple as saying >> >> > >>>>>>> SUSP does suspend - that state is probably defined by >> >> > >>>>>>> the scheduler, not by the OS... >> >> > >>>>>>> >> >> > >>>>>> Right ... this is implementation dependent on the mechanism >> used >> >to >> >> > >>>>>> suspend >> >> > >>>>>> a job (might be a signal, might be some other mechanism). >> What >> >is >> >> > >>>>>> important >> >> > >>>>>> is that there is an operation to initiate the state >> transition. >> >> > >>>>>> >> >> > >>>>>>> - What is the use case for distinguishing between >> UserHold >> >> > >>>>>>> and SystemHold, or between UserSuspend and >> >> > >>>>>>> SystemSuspend? >> >> > >>>>>>> >> >> > >>>>>> If I preempt workload, the system will put it into a >> >SystemSuspend state >> >> > >>>>>> that a user cannot cause a switch out of, otherwise a system >> may >> >become >> >> > >>>>>> oversubscribed due to the preempted and preempting jobs >> running >> >at the >> >> > >>>>>> same >> >> > >>>>>> time. A UserSuspend can be entered and exited by the user, >> and >> >is often >> >> > >>>>>> used >> >> > >>>>>> to hold processing to check progress, etc. >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>>> By the way ... I believe that the state diagram should at >> least >> >be a >> >> > >>>>>> subset >> >> > >>>>>> of the BES state diagram ... we should adopt the same names. >> >> > >>>>>> >> >> > >>>>>> -- Chris >> >> > >>>>> >> >> > >>>>> >> >> > >>> >> >> > >>> >> >> > > >> >> > > >> >> >> >> >> >> >> >> -- >> >> "So much time, so little to do..." -- Garfield >> >> >> > >> > >> > >> >-- >> >Thilo Kielmann >> >http://www.cs.vu.nl/~kielmann/ > > > >-- >Thilo Kielmann >http://www.cs.vu.nl/~kielmann/

1 0

Fwd (andre@merzky.net): Fwd (s.jha@ucl.ac.uk): Re: OGSA is aligned...
by Andre Merzky 14 Feb '06

14 Feb '06

Hi SAGA group, We finally managed to sit together for an hour and discuss the relation between SAGA and OGSA in some detail. Thanks to the OGSA folks for making the time! This email tries to capture the points and results of that discussion. It is rather detailed for two reasons: - we want the OGSA folks to check, and 'officially' agree to these points (they did by now) - we want to use these points as input to our middleware alignment document Observations: - There are a number of paradigms which span OGSA and are potentially apparent at the application level in some use cases (TODO: recheck OGSA use case document) Amongst these paradigms are, most notably, - notification / events - asynchroneous / decoupled operations - state management - life-time management -- notification / events: SAGA thinks we are fine here, as we have A) a generic event and monitoring model, and B) most events can be captured and reacted on on implmentation level. -- asynchroneous / decoupled operations: SAGA thinks we are fine here as well, as A) the API is asynchroneous as well, and B) asynchroneous middleware calls can be made synchroneous with a blocking wait (that has drawbacks though, but we can live with those in our use cases). -- state management: SAGA thinks we are fine here as well, as A) essential state information (file pointer, job state, stream state) are exposed on API level already B) other state information can be kept in the API implementation on client side, or on intermediate stateful services, depending on the system the API is implemented on -- life-time management: This was the most interesting point I think. The use cases are: - an application runs for 3 months, and expects a file pointer to be available for that time - a remote file op implies a log file, but a network failure prevents the lock from beeing removed. W/o life-time management support, that lock lingers (can't distinguish between a network drop and application being busy). Our discussion came to the conclusion, that: - sensible life-time defaults in the implementation, or out of bound, can probably catch most problems - a generic application life-time attribute to a session handle should catch most of the more difficult cases (long running applications etc). - per 80:20 rule, that should be sufficient for SAGA until more specific use cases are known. - more detailed use cases might motivate life-time attributes (contexts) for SAGA object. Summary: At the end, we agreed with Steven's remark that OGSA and other middleware properties should ideally NOT pop up on application level at all. Having said that, there might be use cases where an exposure of life-time, state, discovery etc. might be beneficial for application. Also it might not be obvious what is a middleware property ("one middleware's property might be another applications") ( Comment from Steven: If stuff has to pop up it should do so in an infrastructure neutral way - i.e. it is a lifetime request (not a setting of a WSRF resource property). Also if requesting a specific lifetime there should be an option for the layer to throw a 'NotSupportedExcepetion" ) SAGA should wait for more detailed use cases before pursuing that topic any further, for now we see NO point of conflict between SAGA and OGSA based middleware. Life-time management might be a simple and local addition to session properties, and should be considered for inclusion into the API, after cross checking with the available OGSA use cases. Cheers, Andre et.al. -- "So much time, so little to do..." -- Garfield

1 0

Re: SAGA and Security
by Andre Merzky 14 Feb '06

14 Feb '06

We have more sessions, on Wednesday on 10:30am and another one on 1:30pm - jsut in case you find time. Otherwise email if fine of course! :-) Thanks, Andre. Quoting [Dane Skow] (Feb 14 2006): > > Groan, I just realized that my presentation at the GT4 workshop is at > 10:30 session on Wednesday AM so a 3-way conflict. I don't think > either Olle or I would be able to make that session. I think Anne's > case below is an interesting example to walk through. Let's continue > this thread in email. > > Dane > > On Feb 14, 2006, at 12:31 AM, Andre Merzky wrote: > > >Hi Dane, > > > >thanks for the remarks and corrections - I should have known > >that its not _that_ easy as I thought ;-) It would be > >perfect to have you at the Wednesday session! > > -- "So much time, so little to do..." -- Garfield

1 0

RE: [saga-rg] job states...
by Rajic, Hrabri 14 Feb '06

14 Feb '06

They are not. Grid is a richer environment. Hrabri >-----Original Message----- >From: owner-saga-rg(a)ggf.org [mailto:owner-saga-rg@ggf.org] On Behalf Of >Thilo Kielmann >Sent: Sunday, February 12, 2006 1:09 AM >To: Andre Merzky >Cc: Christopher Smith; Simple API for Grid Applications WG >Subject: Re: [saga-rg] job states... > >Curious question: > >SAGA should align job states with both BES and DRMAA ? >Are they the same to begin with? > >Thilo > >On Sat, Feb 11, 2006 at 03:54:16AM +0100, Andre Merzky wrote: >> X-Original-To: kielmann@localhost >> Delivered-To: kielmann(a)localhost.cs.vu.nl >> Delivered-To: grdfm-saga-rg-outgoing(a)mailbouncer.mcs.anl.gov >> X-Original-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> Delivered-To: grdfm-saga-rg(a)mailbouncer.mcs.anl.gov >> Date: Sat, 11 Feb 2006 03:54:16 +0100 >> From: Andre Merzky <andre(a)merzky.net> >> To: Christopher Smith <csmith(a)platform.com> >> Cc: Andre Merzky <andre(a)merzky.net>, >> Simple API for Grid Applications WG <saga-rg(a)ggf.org> >> Subject: Re: [saga-rg] job states... >> X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at >mailbouncer.mcs.anl.gov >> X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at >mailbouncer.mcs.anl.gov >> >> I agree - the file transfer state models are needed for SAGA. >> We don't have any actions on these states anyway. >> >> Andre. >> >> >> Quoting [Christopher Smith] (Feb 11 2006): >> > >> > Sure. >> > >> > As mentioned ... I think maybe supporting a subset of BES is ok. Much >of the >> > state model wrt file transfer state modelling I think is not required >for >> > SAGA. >> > >> > -- Chris >> > >> > >> > >> > On 10/2/06 18:46, "Andre Merzky" <andre(a)merzky.net> wrote: >> > >> > > Ok, then I'll do that in the strawman. I would appreciate >> > > if you could glance over it after commit, for a sanity >> > > check. >> > > >> > > Thanks, Andre. >> > > >> > > >> > > Quoting [Christopher Smith] (Feb 11 2006): >> > >> >> > >> It makes sense to keep the state models in sync. >> > >> >> > >> -- Chris >> > >> >> > >> >> > >> On 10/2/06 18:26, "Andre Merzky" <andre(a)merzky.net> wrote: >> > >> >> > >>> Quoting [Christopher Smith] (Feb 11 2006): >> > >>>> >> > >>>> What I meant by that comment is that where it is a subset, it >should >> > >>>> reflect >> > >>>> the BES terminology. I think that the number of states represented >is >> > >>>> enough >> > >>>> already. ;-) >> > >>> >> > >>> Would it make sense to just copy the BES state diagram? >> > >>> >> > >>> It did not exist when we (== you ;-) drafted the SAGA job >> > >>> states - if it would have been around then, we might have >> > >>> had copied it already. >> > >>> >> > >>> Apart from the SystemXXX/UserXXX states, and from Hold, >> > >>> it is not that much different from the SAGA model anyway. >> > >>> >> > >>> Cheers, Andre. >> > >>> >> > >>> >> > >>>> -- Chris >> > >>>> >> > >>>> >> > >>>> On 10/2/06 17:30, "Andre Merzky" <andre(a)merzky.net> wrote: >> > >>>> >> > >>>>> Hi Chris, >> > >>>>> >> > >>>>> many thanks for the answers! :-) >> > >>>>> >> > >>>>>> By the way ... I believe that the state diagram should at least >be a >> > >>>>>> subset >> > >>>>>> of the BES state diagram ... we should adopt the same names. >> > >>>>> >> > >>>>> I agree, kind of - I would say that the SAGA job state >> > >>>>> diagram should at _most_ be subset of the BES state diagram. >> > >>>>> It could be _S_implier :-) >> > >>>>> >> > >>>>> Cheers, Andre. >> > >>>>> >> > >>>>> >> > >>>>> Quoting [Christopher Smith] (Feb 10 2006): >> > >>>>>> Date: Fri, 10 Feb 2006 13:41:18 -0800 >> > >>>>>> Subject: Re: [saga-rg] job states... >> > >>>>>> From: Christopher Smith <csmith(a)platform.com> >> > >>>>>> To: Simple API for Grid Applications WG <saga-rg(a)ggf.org> >> > >>>>>> >> > >>>>>> On 4/2/06 11:18, "Andre Merzky" <andre(a)merzky.net> wrote: >> > >>>>>> >> > >>>>>> Ok ... I'll try to answer these, at least from my viewpoint. >> > >>>>>> >> > >>>>>>> >> > >>>>>>> I think that diagram is wrong, isn't it? Well, here are my >> > >>>>>>> questions: >> > >>>>>>> >> > >>>>>>> - if we submit a job, its immediately Queued - is that >> > >>>>>>> right? Should it be pending before (e.g. as long as the >> > >>>>>>> queuing request travels the middleware layers)? >> > >>>>>>> >> > >>>>>> To me, Queued is the same as Pending. Pending is probably a >better word >> > >>>>>> for >> > >>>>>> this. Can't remember where the Queued name came from, as LSF >uses PEND. >> > >>>>>> >> > >>>>>>> - can the hold and suspend states reached only from >> > >>>>>>> 'Running', or from elsewhere as well? >> > >>>>>>> >> > >>>>>> You can only go into a Hold state from Pending, I think, or >directly into >> > >>>>>> Hold on submission. >> > >>>>>> >> > >>>>>>> - What is the difference between 'Hold' and 'Suspend'? >> > >>>>>>> >> > >>>>>> A Hold state tells the scheduler/broker not to consider this job >for >> > >>>>>> scheduling/dispatch until the hold is explicitly released. >> > >>>>>> >> > >>>>>>> - Are there signals defined (apart from KILL) which shange >> > >>>>>>> the job state? I guess that is not as simple as saying >> > >>>>>>> SUSP does suspend - that state is probably defined by >> > >>>>>>> the scheduler, not by the OS... >> > >>>>>>> >> > >>>>>> Right ... this is implementation dependent on the mechanism used >to >> > >>>>>> suspend >> > >>>>>> a job (might be a signal, might be some other mechanism). What >is >> > >>>>>> important >> > >>>>>> is that there is an operation to initiate the state transition. >> > >>>>>> >> > >>>>>>> - What is the use case for distinguishing between UserHold >> > >>>>>>> and SystemHold, or between UserSuspend and >> > >>>>>>> SystemSuspend? >> > >>>>>>> >> > >>>>>> If I preempt workload, the system will put it into a >SystemSuspend state >> > >>>>>> that a user cannot cause a switch out of, otherwise a system may >become >> > >>>>>> oversubscribed due to the preempted and preempting jobs running >at the >> > >>>>>> same >> > >>>>>> time. A UserSuspend can be entered and exited by the user, and >is often >> > >>>>>> used >> > >>>>>> to hold processing to check progress, etc. >> > >>>>>> >> > >>>>>> >> > >>>>>> By the way ... I believe that the state diagram should at least >be a >> > >>>>>> subset >> > >>>>>> of the BES state diagram ... we should adopt the same names. >> > >>>>>> >> > >>>>>> -- Chris >> > >>>>> >> > >>>>> >> > >>> >> > >>> >> > > >> > > >> >> >> >> -- >> "So much time, so little to do..." -- Garfield >> > > > >-- >Thilo Kielmann >http://www.cs.vu.nl/~kielmann/

2 1