Hi all, As Marvin mentioned in his email on extensibility and composition of protocols, I'd like to present this short document describing, in more detail, a model for extending the state model of jobs from a "base profile". The document is based on some discussions that Marvin and I have been having about scheduler interoperability, and is very close to the model that has been proposed within the SAGA working group. Please review Marvin's email about extensibility to get a feel for the "context" of the model. It is my intention to show how the ESI and BES state models can be represented using this approach. I'll try to get that done before Tokyo, but might just need to present some slides at GGF17, assuming I'm given a few minutes to do so. This is very much a work in progress, so I'm looking forward to feedback.... A note for the SAGA group .... I'd still like to see "pending" in the base job state diagram ... don't know why it disappeared. ;-) -- Chris
Quoting [Christopher Smith] (May 02 2006):
Hi all,
As Marvin mentioned in his email on extensibility and composition of protocols, I'd like to present this short document describing, in more detail, a model for extending the state model of jobs from a "base profile". The document is based on some discussions that Marvin and I have been having about scheduler interoperability, and is very close to the model that has been proposed within the SAGA working group. Please review Marvin's email about extensibility to get a feel for the "context" of the model.
It is my intention to show how the ESI and BES state models can be represented using this approach. I'll try to get that done before Tokyo, but might just need to present some slides at GGF17, assuming I'm given a few minutes to do so.
This is very much a work in progress, so I'm looking forward to feedback....
A note for the SAGA group .... I'd still like to see "pending" in the base job state diagram ... don't know why it disappeared. ;-)
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state. So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added. Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram. Well, thats my opinion at least ;-) I know that you see Pending as crucial. IIUC, that funds (partly) on the fact that most/all Queuing systems support pending. However, SAGA is right now not about submissing to queues in the first place, so I am not sure if that is relevant (right now). Could we discuss that in Tokyo? Some (real) use cases would be very convincing. Thanks, best regards, Andre.
-- Chris
-- "So much time, so little to do..." -- Garfield
I would suggest to spend some 20 minutes of a SAGA session on this. Does anyone (Andre? ;-) know which session this could be? Thilo On Wed, May 03, 2006 at 02:36:04PM +0200, Andre Merzky wrote:
X-Original-To: kielmann@localhost Delivered-To: kielmann@localhost.cs.vu.nl Delivered-To: grdfm-saga-rg-outgoing@mailbouncer.mcs.anl.gov X-Original-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Delivered-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Date: Wed, 3 May 2006 14:36:04 +0200 From: Andre Merzky <andre@merzky.net> To: Christopher Smith <csmith@platform.com> Cc: "ogsa-wg@ggf.org" <ogsa-wg@ggf.org>, ogsa-bes-wg@ggf.org, SAGA RG <saga-rg@ggf.org> Subject: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs
Quoting [Christopher Smith] (May 02 2006):
Hi all,
As Marvin mentioned in his email on extensibility and composition of protocols, I'd like to present this short document describing, in more detail, a model for extending the state model of jobs from a "base profile". The document is based on some discussions that Marvin and I have been having about scheduler interoperability, and is very close to the model that has been proposed within the SAGA working group. Please review Marvin's email about extensibility to get a feel for the "context" of the model.
It is my intention to show how the ESI and BES state models can be represented using this approach. I'll try to get that done before Tokyo, but might just need to present some slides at GGF17, assuming I'm given a few minutes to do so.
This is very much a work in progress, so I'm looking forward to feedback....
A note for the SAGA group .... I'd still like to see "pending" in the base job state diagram ... don't know why it disappeared. ;-)
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state.
So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added.
Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram.
Well, thats my opinion at least ;-) I know that you see Pending as crucial. IIUC, that funds (partly) on the fact that most/all Queuing systems support pending. However, SAGA is right now not about submissing to queues in the first place, so I am not sure if that is relevant (right now).
Could we discuss that in Tokyo? Some (real) use cases would be very convincing.
Thanks, best regards,
Andre.
-- Chris
-- "So much time, so little to do..." -- Garfield
-- Thilo Kielmann http://www.cs.vu.nl/~kielmann/
On 03/5/06 05:36, "Andre Merzky" <andre@merzky.net> wrote:
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state.
So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added.
The state that a job enters after a create_job operation is 'pending', so the create_job operation is the method that puts a job in this state. This state is required for queuing systems ... I can't even imagine how it got dropped from the state diagram. I think you're trying to map the task state model too closely. ;-)
Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram.
This I agree with. -- Chris
Quoting [Christopher Smith] (May 03 2006):
On 03/5/06 05:36, "Andre Merzky" <andre@merzky.net> wrote:
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state.
So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added.
The state that a job enters after a create_job operation is 'pending', so the create_job operation is the method that puts a job in this state.
Right, I understand that from the queuing point I think. However, from the application level it does not matter, as it cannot actively change that state. We have no notion of queue states at all. Well, let me put it differently: what do we gain by moving the state to the higher level? It complicates the state diagram (not much, that is not the point), but does not allow to perform any new state transition from application level. As said, I would compare this to Suspended state: its clearly an important state in many systems, but as long as the API does not expose means to suspend/resume a job, the state is meaningless, and merily informative (e.g. the job gut suspended by a 3rd party, or by the backend). That information is exposed however, through the job state details - same holds for hols (ha!) I think.
This state is required for queuing systems ... I can't even imagine how it got dropped from the state diagram. I think you're trying to map the task state model too closely. ;-)
Nono, that has nothing to do with the task states - they could easily leave that state out. Well, its very convenient to have the same model here, thats for sure ;-)
Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram.
This I agree with.
As said, it might well be that SAGA is broken in the respect that we doesn't allow job.hold (), or submit into hold. Well, it wasn't in the use-cases ... :-P Cheers, Andre.
-- Chris
-- "So much time, so little to do..." -- Garfield
Chris and Andre, I think we really should resolve this issue face-to-face next week at GGF17. That will save a lot of email typing time... Andre, do we have a slot in one of our sessions? Thilo On Wed, May 03, 2006 at 06:08:46PM +0200, Andre Merzky wrote:
X-Original-To: kielmann@localhost Delivered-To: kielmann@localhost.cs.vu.nl Delivered-To: grdfm-saga-rg-outgoing@mailbouncer.mcs.anl.gov X-Original-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Delivered-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Date: Wed, 3 May 2006 18:08:46 +0200 From: Andre Merzky <andre@merzky.net> To: Christopher Smith <csmith@platform.com> Cc: Andre Merzky <andre@merzky.net>, "ogsa-wg@ggf.org" <ogsa-wg@ggf.org>, ogsa-bes-wg@ggf.org, SAGA RG <saga-rg@ggf.org> Subject: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs
Quoting [Christopher Smith] (May 03 2006):
On 03/5/06 05:36, "Andre Merzky" <andre@merzky.net> wrote:
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state.
So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added.
The state that a job enters after a create_job operation is 'pending', so the create_job operation is the method that puts a job in this state.
Right, I understand that from the queuing point I think. However, from the application level it does not matter, as it cannot actively change that state. We have no notion of queue states at all.
Well, let me put it differently: what do we gain by moving the state to the higher level? It complicates the state diagram (not much, that is not the point), but does not allow to perform any new state transition from application level.
As said, I would compare this to Suspended state: its clearly an important state in many systems, but as long as the API does not expose means to suspend/resume a job, the state is meaningless, and merily informative (e.g. the job gut suspended by a 3rd party, or by the backend). That information is exposed however, through the job state details - same holds for hols (ha!) I think.
This state is required for queuing systems ... I can't even imagine how it got dropped from the state diagram. I think you're trying to map the task state model too closely. ;-)
Nono, that has nothing to do with the task states - they could easily leave that state out. Well, its very convenient to have the same model here, thats for sure ;-)
Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram.
This I agree with.
As said, it might well be that SAGA is broken in the respect that we doesn't allow job.hold (), or submit into hold. Well, it wasn't in the use-cases ... :-P
Cheers, Andre.
-- Chris
-- "So much time, so little to do..." -- Garfield
-- Thilo Kielmann http://www.cs.vu.nl/~kielmann/
I agree ... we can discuss this next week. -- Chris On 03/5/06 10:09, "Thilo Kielmann" <kielmann@cs.vu.nl> wrote:
Chris and Andre,
I think we really should resolve this issue face-to-face next week at GGF17. That will save a lot of email typing time...
Andre, do we have a slot in one of our sessions?
Thilo
On Wed, May 03, 2006 at 06:08:46PM +0200, Andre Merzky wrote:
X-Original-To: kielmann@localhost Delivered-To: kielmann@localhost.cs.vu.nl Delivered-To: grdfm-saga-rg-outgoing@mailbouncer.mcs.anl.gov X-Original-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Delivered-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Date: Wed, 3 May 2006 18:08:46 +0200 From: Andre Merzky <andre@merzky.net> To: Christopher Smith <csmith@platform.com> Cc: Andre Merzky <andre@merzky.net>, "ogsa-wg@ggf.org" <ogsa-wg@ggf.org>, ogsa-bes-wg@ggf.org, SAGA RG <saga-rg@ggf.org> Subject: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs
Quoting [Christopher Smith] (May 03 2006):
On 03/5/06 05:36, "Andre Merzky" <andre@merzky.net> wrote:
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state.
So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added.
The state that a job enters after a create_job operation is 'pending', so the create_job operation is the method that puts a job in this state.
Right, I understand that from the queuing point I think. However, from the application level it does not matter, as it cannot actively change that state. We have no notion of queue states at all.
Well, let me put it differently: what do we gain by moving the state to the higher level? It complicates the state diagram (not much, that is not the point), but does not allow to perform any new state transition from application level.
As said, I would compare this to Suspended state: its clearly an important state in many systems, but as long as the API does not expose means to suspend/resume a job, the state is meaningless, and merily informative (e.g. the job gut suspended by a 3rd party, or by the backend). That information is exposed however, through the job state details - same holds for hols (ha!) I think.
This state is required for queuing systems ... I can't even imagine how it got dropped from the state diagram. I think you're trying to map the task state model too closely. ;-)
Nono, that has nothing to do with the task states - they could easily leave that state out. Well, its very convenient to have the same model here, thats for sure ;-)
Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram.
This I agree with.
As said, it might well be that SAGA is broken in the respect that we doesn't allow job.hold (), or submit into hold. Well, it wasn't in the use-cases ... :-P
Cheers, Andre.
-- Chris
-- "So much time, so little to do..." -- Garfield
Chris, we should have space in the first saga-core-wg session, on Thursday 3:45 pm. Does that collide with some other session for you (or other interested people)? An alternative would be Friday 11:00 am. Cheers, Andre. PS.: Thanks for the opportunity to announce our sessions on three lists! Har har >:-) Quoting [Christopher Smith] (May 03 2006):
Date: Wed, 03 May 2006 10:45:29 -0700 Subject: Re: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs From: Christopher Smith <csmith@platform.com> To: Thilo Kielmann <kielmann@cs.vu.nl>, Andre Merzky <andre@merzky.net> CC: "ogsa-wg@ggf.org" <ogsa-wg@ggf.org>, ogsa-bes-wg@ggf.org, SAGA RG <saga-rg@ggf.org>
I agree ... we can discuss this next week.
-- Chris
On 03/5/06 10:09, "Thilo Kielmann" <kielmann@cs.vu.nl> wrote:
Chris and Andre,
I think we really should resolve this issue face-to-face next week at GGF17. That will save a lot of email typing time...
Andre, do we have a slot in one of our sessions?
Thilo
On Wed, May 03, 2006 at 06:08:46PM +0200, Andre Merzky wrote:
X-Original-To: kielmann@localhost Delivered-To: kielmann@localhost.cs.vu.nl Delivered-To: grdfm-saga-rg-outgoing@mailbouncer.mcs.anl.gov X-Original-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Delivered-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Date: Wed, 3 May 2006 18:08:46 +0200 From: Andre Merzky <andre@merzky.net> To: Christopher Smith <csmith@platform.com> Cc: Andre Merzky <andre@merzky.net>, "ogsa-wg@ggf.org" <ogsa-wg@ggf.org>, ogsa-bes-wg@ggf.org, SAGA RG <saga-rg@ggf.org> Subject: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs
Quoting [Christopher Smith] (May 03 2006):
On 03/5/06 05:36, "Andre Merzky" <andre@merzky.net> wrote:
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state.
So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added.
The state that a job enters after a create_job operation is 'pending', so the create_job operation is the method that puts a job in this state.
Right, I understand that from the queuing point I think. However, from the application level it does not matter, as it cannot actively change that state. We have no notion of queue states at all.
Well, let me put it differently: what do we gain by moving the state to the higher level? It complicates the state diagram (not much, that is not the point), but does not allow to perform any new state transition from application level.
As said, I would compare this to Suspended state: its clearly an important state in many systems, but as long as the API does not expose means to suspend/resume a job, the state is meaningless, and merily informative (e.g. the job gut suspended by a 3rd party, or by the backend). That information is exposed however, through the job state details - same holds for hols (ha!) I think.
This state is required for queuing systems ... I can't even imagine how it got dropped from the state diagram. I think you're trying to map the task state model too closely. ;-)
Nono, that has nothing to do with the task states - they could easily leave that state out. Well, its very convenient to have the same model here, thats for sure ;-)
Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram.
This I agree with.
As said, it might well be that SAGA is broken in the respect that we doesn't allow job.hold (), or submit into hold. Well, it wasn't in the use-cases ... :-P
Cheers, Andre.
-- Chris
-- "So much time, so little to do..." -- Garfield
-- "So much time, so little to do..." -- Garfield
The Thursday session works, as there is a collision with JSDL on Friday. -- Chris On 04/5/06 07:38, "Andre Merzky" <andre@merzky.net> wrote:
Chris,
we should have space in the first saga-core-wg session, on Thursday 3:45 pm. Does that collide with some other session for you (or other interested people)? An alternative would be Friday 11:00 am.
Cheers, Andre.
PS.: Thanks for the opportunity to announce our sessions on three lists! Har har >:-)
Quoting [Christopher Smith] (May 03 2006):
Date: Wed, 03 May 2006 10:45:29 -0700 Subject: Re: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs From: Christopher Smith <csmith@platform.com> To: Thilo Kielmann <kielmann@cs.vu.nl>, Andre Merzky <andre@merzky.net> CC: "ogsa-wg@ggf.org" <ogsa-wg@ggf.org>, ogsa-bes-wg@ggf.org, SAGA RG <saga-rg@ggf.org>
I agree ... we can discuss this next week.
-- Chris
On 03/5/06 10:09, "Thilo Kielmann" <kielmann@cs.vu.nl> wrote:
Chris and Andre,
I think we really should resolve this issue face-to-face next week at GGF17. That will save a lot of email typing time...
Andre, do we have a slot in one of our sessions?
Thilo
On Wed, May 03, 2006 at 06:08:46PM +0200, Andre Merzky wrote:
X-Original-To: kielmann@localhost Delivered-To: kielmann@localhost.cs.vu.nl Delivered-To: grdfm-saga-rg-outgoing@mailbouncer.mcs.anl.gov X-Original-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Delivered-To: grdfm-saga-rg@mailbouncer.mcs.anl.gov Date: Wed, 3 May 2006 18:08:46 +0200 From: Andre Merzky <andre@merzky.net> To: Christopher Smith <csmith@platform.com> Cc: Andre Merzky <andre@merzky.net>, "ogsa-wg@ggf.org" <ogsa-wg@ggf.org>, ogsa-bes-wg@ggf.org, SAGA RG <saga-rg@ggf.org> Subject: [saga-rg] Re: [ogsa-bes-wg] Extensible state model for jobs
Quoting [Christopher Smith] (May 03 2006):
On 03/5/06 05:36, "Andre Merzky" <andre@merzky.net> wrote:
Right now its absent because only those states which can be actually reached by SAGA method calls are present. Now, as you said earlier, it can be assumed a flaw that SAGA has no means to put a job into 'Pending' state.
So, if there are some use cases or some agreement that the API should allow to move a job into 'Pending' state, the method/attribute will be added, and the state will be added.
The state that a job enters after a create_job operation is 'pending', so the create_job operation is the method that puts a job in this state.
Right, I understand that from the queuing point I think. However, from the application level it does not matter, as it cannot actively change that state. We have no notion of queue states at all.
Well, let me put it differently: what do we gain by moving the state to the higher level? It complicates the state diagram (not much, that is not the point), but does not allow to perform any new state transition from application level.
As said, I would compare this to Suspended state: its clearly an important state in many systems, but as long as the API does not expose means to suspend/resume a job, the state is meaningless, and merily informative (e.g. the job gut suspended by a 3rd party, or by the backend). That information is exposed however, through the job state details - same holds for hols (ha!) I think.
This state is required for queuing systems ... I can't even imagine how it got dropped from the state diagram. I think you're trying to map the task state model too closely. ;-)
Nono, that has nothing to do with the task states - they could easily leave that state out. Well, its very convenient to have the same model here, thats for sure ;-)
Similar reasoning holds for 'Suspend' - as long as we don't have suspend/resume methods, we would not like to expose a 'Suspended' state on the higher level state diagram.
This I agree with.
As said, it might well be that SAGA is broken in the respect that we doesn't allow job.hold (), or submit into hold. Well, it wasn't in the use-cases ... :-P
Cheers, Andre.
-- Chris
-- "So much time, so little to do..." -- Garfield
participants (3)
-
Andre Merzky -
Christopher Smith -
Thilo Kielmann