Re: [ogsa-bes-wg] Proposed changes to Spec.
Darren: I would like to argue against your proposed operations on the grounds of fragility and scalability. Your operation definitions require that a client keep track of the set of activities to be monitored and controlled. That's a big burden to place on the client. I suggest (as we have also suggested in earlier communications to BES-WG) that we should instead allow the user to specify a set of activities that are to be monitored or controlled in terms of their properties, not names. E.g., "activities that belong to me", "activities that involve executable Foo", "activities that have been running for over 2 hours." A natural way of modeling this is to represent the set of activities associated with your jobs as a WS-ServiceGroup Service Group, and then use queries on the service group to specify the activities to which an operation applies. Regards -- Ian. PS: As noted in previous emails and conversations, I also believe that the use of WS-Names rather than vanilla EPRs is inappropriate. I also believe that it is inconsistent with the OGSA policy that WS-Names cannot be mandated in OGSA-related specifications. At 08:52 AM 10/26/2005 -0600, Darren Pulsipher wrote:
I have spent some considerable time with the document. I know far more than I planned, but it was good time spent. I think we should discuss these items in our meeting on Thursday. I have posted some changes to the document but would like to discuss these items before I make these changes to them.
There are some fundamental problems/questions that we need to resolve.
1. How do we resume from where we left off. There is no Resume anymore so we are putting the responsibility to the submitter or controller to keep track of states. We can only use RequestActivityStateChanges to suspend something and if I want to suspend 1000s of activities that means I would need to first do a GetActivityState save the states and then call RequestActivityStateChanges without any of the activities changing state. That is not going to happen. So I propose we do the following: - Add the following interfaces: SuspendActivites(WS-Name[] activityIdentifiers) ResumeActivities(WS-Name[] activityIdentifiers)
Or we add one interface PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers)
2. Another problem is how do we terminate an activity. The States that are available are "Terminated" and "ShuttingDown". According to the state net the activity is moved to the "ShuttingDown" and then to "Terminated" so requesting RequestActivityStateChanges for activities with "ShuttingDown" means it should move to the ShuttingDown state but it will move the activities to the "Terminated" state. This can be very confusing. So I suggest we add another interface. - Add the following interface: TerminateActivities(WS-Name[] activityIdentifiers)
Or add another possible argument to the interface. PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] activityIdentifiers)
3. Another question that is not clear in the spec is if an action (staging in or staging out) moves into an exception state does that mean that the activity moves into the exception state right away or does it mean that it ever moves into the exception state. Does it mean that if one of the actions moves into an exception state that all other actions are halted and moved to the exception state and then the activity? - I proposed that actions state have no impact on the activity state. If an action has an exception or is terminated then it should not have any impact on the activity what so every.
4. Another not so important change but it might help clarify some confusion and allow for extendibility later on would be to get rid of the "ExecutionPending" and "ExecutionComplete" and make the "Running" state a composite state like the "StagingIn" and "StagingOut" states. This will give the ability to handle complex multiple application execution definitions that the JSDL group is looking at for the next rev of the Job Definition. This will allow for multiple Application actions. This is not workflow. Nor should it be considered the start of workflow. This just allows for extendibility easily.
Darren Pulsipher
_______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org
Ian, My understanding of the OGSA policy is that OGSA does not mandate WS-Names . instead it encourages them WHERE THEY ARE APPROPRIATE. That decision is up to individual working groups, and perhaps a design team working on a profile. The policy, in my mind, is NOT that WS-Names may not be mandated by OGSA specifications. In the case of OGSA-BES, at the last GGF in Boston the working group discussed at great length the pro's and cons of WS-Names, opaque EPR's, and abstract names (i.e., uri's with a few properties). The overwhelming majority preferred WS-Names (an explicit, comparable return value) (8 to 2). A further discussion was held as to whether the WS-Names should have both abstract names AND resolvers, and the vote was 4 to 1 for having a resolver, with many people indifferent. Thus, OGSA-BES will have* WS-Names as return values and input parameters. We also discussed "eventing" at some length. I have not yet typed in my notes, the three high order bits were: 1) The container is the right place to emit the events 2) Clients must explicitly subscribe to events ("use filtering from the container, not specific registration" . I'm not sure what that means, I think we will need to discuss this in a call and remind me.) 3) Events are state transitions of the activities Andrew *Of course the document still has to go into public comment, get revised, and hopefully some day be approved by the GFSG. What is really in the document may change between now and then. _____ From: owner-ogsa-bes-wg@ggf.org [mailto:owner-ogsa-bes-wg@ggf.org] On Behalf Of Ian Foster Sent: Wednesday, October 26, 2005 11:40 AM To: Darren Pulsipher; ogsa-bes-wg@ggf.org Subject: Re: [ogsa-bes-wg] Proposed changes to Spec. Darren: I would like to argue against your proposed operations on the grounds of fragility and scalability. Your operation definitions require that a client keep track of the set of activities to be monitored and controlled. That's a big burden to place on the client. I suggest (as we have also suggested in earlier communications to BES-WG) that we should instead allow the user to specify a set of activities that are to be monitored or controlled in terms of their properties, not names. E.g., "activities that belong to me", "activities that involve executable Foo", "activities that have been running for over 2 hours." A natural way of modeling this is to represent the set of activities associated with your jobs as a WS-ServiceGroup Service Group, and then use queries on the service group to specify the activities to which an operation applies. Regards -- Ian. PS: As noted in previous emails and conversations, I also believe that the use of WS-Names rather than vanilla EPRs is inappropriate. I also believe that it is inconsistent with the OGSA policy that WS-Names cannot be mandated in OGSA-related specifications. At 08:52 AM 10/26/2005 -0600, Darren Pulsipher wrote: I have spent some considerable time with the document. I know far more than I planned, but it was good time spent. I think we should discuss these items in our meeting on Thursday. I have posted some changes to the document but would like to discuss these items before I make these changes to them. There are some fundamental problems/questions that we need to resolve. 1. How do we resume from where we left off. There is no Resume anymore so we are putting the responsibility to the submitter or controller to keep track of states. We can only use RequestActivityStateChanges to suspend something and if I want to suspend 1000s of activities that means I would need to first do a GetActivityState save the states and then call RequestActivityStateChanges without any of the activities changing state. That is not going to happen. So I propose we do the following: - Add the following interfaces: SuspendActivites(WS-Name[] activityIdentifiers) ResumeActivities(WS-Name[] activityIdentifiers) Or we add one interface PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers) 2. Another problem is how do we terminate an activity. The States that are available are "Terminated" and "ShuttingDown". According to the state net the activity is moved to the "ShuttingDown" and then to "Terminated" so requesting RequestActivityStateChanges for activities with "ShuttingDown" means it should move to the ShuttingDown state but it will move the activities to the "Terminated" state. This can be very confusing. So I suggest we add another interface. - Add the following interface: TerminateActivities(WS-Name[] activityIdentifiers) Or add another possible argument to the interface. PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] activityIdentifiers) 3. Another question that is not clear in the spec is if an action (staging in or staging out) moves into an exception state does that mean that the activity moves into the exception state right away or does it mean that it ever moves into the exception state. Does it mean that if one of the actions moves into an exception state that all other actions are halted and moved to the exception state and then the activity? - I proposed that actions state have no impact on the activity state. If an action has an exception or is terminated then it should not have any impact on the activity what so every. 4. Another not so important change but it might help clarify some confusion and allow for extendibility later on would be to get rid of the "ExecutionPending" and "ExecutionComplete" and make the "Running" state a composite state like the "StagingIn" and "StagingOut" states. This will give the ability to handle complex multiple application execution definitions that the JSDL group is looking at for the next rev of the Job Definition. This will allow for multiple Application actions. This is not workflow. Nor should it be considered the start of workflow. This just allows for extendibility easily. Darren Pulsipher _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org http://www.globus.org/
Andrew: We should ask the OGSA-WG to speak definitively on the first issue, as there seems to be confusion among participants as to what the policy means. Regarding OGSA-BES's use of WS-Name: I appreciate the information on the decision procedure. More generally, I guess this exchange reflects a common tension between the relative importance of the views of "insiders" vs. "outsiders" in GGF WGs. I know that quite a few people (myself included) started off wanting to engage as insiders in OGSA-BES, but after various unsatisfactory interactions (e.g., discussions like this one, where the response to a concern is to be told that it will be disregarded), ended up becoming outsiders. The OGSA-BES is of course free to work as it wants, but excluding the views of so many people also greatly reduces the likelihood that whatever specifications OGSA-BES produces will be widely adopted. I don't understand the discussion of events: I didn't speak to eventing in my email, but about the right way of specifying the activities to which an operation should apply. Regards, Ian. At 01:05 PM 10/26/2005 -0400, Andrew Grimshaw wrote:
Ian,
My understanding of the OGSA policy is that OGSA does not mandate WS-Names & instead it encourages them WHERE THEY ARE APPROPRIATE. That decision is up to individual working groups, and perhaps a design team working on a profile. The policy, in my mind, is NOT that WS-Names may not be mandated by OGSA specifications.
In the case of OGSA-BES, at the last GGF in Boston the working group discussed at great length the pros and cons of WS-Names, opaque EPRs, and abstract names (i.e., uris with a few properties). The overwhelming majority preferred WS-Names (an explicit, comparable return value) (8 to 2). A further discussion was held as to whether the WS-Names should have both abstract names AND resolvers, and the vote was 4 to 1 for having a resolver, with many people indifferent.
Thus, OGSA-BES will have* WS-Names as return values and input parameters.
We also discussed eventingat some length. I have not yet typed in my notes, the three high order bits were:
1) The container is the right place to emit the events
2) Clients must explicitly subscribe to events (use filtering from the container, not specific registration& Im not sure what that means, I think we will need to discuss this in a call and remind me.)
3) Events are state transitions of the activities
Andrew
*Of course the document still has to go into public comment, get revised, and hopefully some day be approved by the GFSG. What is really in the document may change between now and then.
---------- From: owner-ogsa-bes-wg@ggf.org [mailto:owner-ogsa-bes-wg@ggf.org] On Behalf Of Ian Foster Sent: Wednesday, October 26, 2005 11:40 AM To: Darren Pulsipher; ogsa-bes-wg@ggf.org Subject: Re: [ogsa-bes-wg] Proposed changes to Spec.
Darren:
I would like to argue against your proposed operations on the grounds of fragility and scalability. Your operation definitions require that a client keep track of the set of activities to be monitored and controlled. That's a big burden to place on the client.
I suggest (as we have also suggested in earlier communications to BES-WG) that we should instead allow the user to specify a set of activities that are to be monitored or controlled in terms of their properties, not names. E.g., "activities that belong to me", "activities that involve executable Foo", "activities that have been running for over 2 hours."
A natural way of modeling this is to represent the set of activities associated with your jobs as a WS-ServiceGroup Service Group, and then use queries on the service group to specify the activities to which an operation applies.
Regards -- Ian.
PS: As noted in previous emails and conversations, I also believe that the use of WS-Names rather than vanilla EPRs is inappropriate. I also believe that it is inconsistent with the OGSA policy that WS-Names cannot be mandated in OGSA-related specifications.
At 08:52 AM 10/26/2005 -0600, Darren Pulsipher wrote:
I have spent some considerable time with the document. I know far more than I planned, but it was good time spent. I think we should discuss these items in our meeting on Thursday. I have posted some changes to the document but would like to discuss these items before I make these changes to them.
There are some fundamental problems/questions that we need to resolve.
1. How do we resume from where we left off. There is no Resume anymore so we are putting the responsibility to the submitter or controller to keep track of states. We can only use RequestActivityStateChanges to suspend something and if I want to suspend 1000s of activities that means I would need to first do a GetActivityState save the states and then call RequestActivityStateChanges without any of the activities changing state. That is not going to happen. So I propose we do the following: - Add the following interfaces: SuspendActivites(WS-Name[] activityIdentifiers) ResumeActivities(WS-Name[] activityIdentifiers)
Or we add one interface PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers)
2. Another problem is how do we terminate an activity. The States that are available are "Terminated" and "ShuttingDown". According to the state net the activity is moved to the "ShuttingDown" and then to "Terminated" so requesting RequestActivityStateChanges for activities with "ShuttingDown" means it should move to the ShuttingDown state but it will move the activities to the "Terminated" state. This can be very confusing. So I suggest we add another interface. - Add the following interface: TerminateActivities(WS-Name[] activityIdentifiers)
Or add another possible argument to the interface. PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] activityIdentifiers)
3. Another question that is not clear in the spec is if an action (staging in or staging out) moves into an exception state does that mean that the activity moves into the exception state right away or does it mean that it ever moves into the exception state. Does it mean that if one of the actions moves into an exception state that all other actions are halted and moved to the exception state and then the activity? - I proposed that actions state have no impact on the activity state. If an action has an exception or is terminated then it should not have any impact on the activity what so every.
4. Another not so important change but it might help clarify some confusion and allow for extendibility later on would be to get rid of the "ExecutionPending" and "ExecutionComplete" and make the "Running" state a composite state like the "StagingIn" and "StagingOut" states. This will give the ability to handle complex multiple application execution definitions that the JSDL group is looking at for the next rev of the Job Definition. This will allow for multiple Application actions. This is not workflow. Nor should it be considered the start of workflow. This just allows for extendibility easily.
Darren Pulsipher
_______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org
_______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org
Ian, We did not disregard the comment at all. It was discussed at length in Boston by a room with 15-20 people, from a very diverse set of organizations and backgrounds. Pro's and con's were presented, and consensus was reached. We are not excluding any views - certainly not "of so many people". However like any consensus driven organization, the WG does not adopt every view expressed. The attention to WS-Names in BES - i.e., the use of EPR's with an explicit decoration of an element called AbstractName of type URI/IRI perplexes me. The URI/IRI is close to a free form string - with no internal structure being mandated whatsoever. Adding another element to an XML structure is simple (the X stands for extensible). Presumably Globus uses some information in the EPR to identify in some way the endpoint service being referenced by an EPR. What WS-Names do is give the client a way to determine if two EPR's "point" to the "same" endpoint. Constructing the AbstractName can be trivial - concatenate the string elements that are currently being used to identify the endpoint. Andrew _____ From: owner-ogsa-bes-wg@ggf.org [mailto:owner-ogsa-bes-wg@ggf.org] On Behalf Of Ian Foster Sent: Wednesday, October 26, 2005 1:44 PM To: Andrew Grimshaw; 'Darren Pulsipher'; ogsa-bes-wg@ggf.org Subject: RE: [ogsa-bes-wg] Proposed changes to Spec. Andrew: We should ask the OGSA-WG to speak definitively on the first issue, as there seems to be confusion among participants as to what the policy means. Regarding OGSA-BES's use of WS-Name: I appreciate the information on the decision procedure. More generally, I guess this exchange reflects a common tension between the relative importance of the views of "insiders" vs. "outsiders" in GGF WGs. I know that quite a few people (myself included) started off wanting to engage as insiders in OGSA-BES, but after various unsatisfactory interactions (e.g., discussions like this one, where the response to a concern is to be told that it will be disregarded), ended up becoming outsiders. The OGSA-BES is of course free to work as it wants, but excluding the views of so many people also greatly reduces the likelihood that whatever specifications OGSA-BES produces will be widely adopted. I don't understand the discussion of events: I didn't speak to eventing in my email, but about the right way of specifying the activities to which an operation should apply. Regards, Ian. At 01:05 PM 10/26/2005 -0400, Andrew Grimshaw wrote: Ian, My understanding of the OGSA policy is that OGSA does not mandate WS-Names & instead it encourages them WHERE THEY ARE APPROPRIATE. That decision is up to individual working groups, and perhaps a design team working on a profile. The policy, in my mind, is NOT that WS-Names may not be mandated by OGSA specifications. In the case of OGSA-BES, at the last GGF in Boston the working group discussed at great length the pros and cons of WS-Names, opaque EPRs, and abstract names (i.e., uris with a few properties). The overwhelming majority preferred WS-Names (an explicit, comparable return value) (8 to 2). A further discussion was held as to whether the WS-Names should have both abstract names AND resolvers, and the vote was 4 to 1 for having a resolver, with many people indifferent. Thus, OGSA-BES will have* WS-Names as return values and input parameters. We also discussed eventingat some length. I have not yet typed in my notes, the three high order bits were: 1) The container is the right place to emit the events 2) Clients must explicitly subscribe to events (use filtering from the container, not specific registration& Im not sure what that means, I think we will need to discuss this in a call and remind me.) 3) Events are state transitions of the activities Andrew *Of course the document still has to go into public comment, get revised, and hopefully some day be approved by the GFSG. What is really in the document may change between now and then. _____ From: owner-ogsa-bes-wg@ggf.org [mailto:owner-ogsa-bes-wg@ggf.org] On Behalf Of Ian Foster Sent: Wednesday, October 26, 2005 11:40 AM To: Darren Pulsipher; ogsa-bes-wg@ggf.org Subject: Re: [ogsa-bes-wg] Proposed changes to Spec. Darren: I would like to argue against your proposed operations on the grounds of fragility and scalability. Your operation definitions require that a client keep track of the set of activities to be monitored and controlled. That's a big burden to place on the client. I suggest (as we have also suggested in earlier communications to BES-WG) that we should instead allow the user to specify a set of activities that are to be monitored or controlled in terms of their properties, not names. E.g., "activities that belong to me", "activities that involve executable Foo", "activities that have been running for over 2 hours." A natural way of modeling this is to represent the set of activities associated with your jobs as a WS-ServiceGroup Service Group, and then use queries on the service group to specify the activities to which an operation applies. Regards -- Ian. PS: As noted in previous emails and conversations, I also believe that the use of WS-Names rather than vanilla EPRs is inappropriate. I also believe that it is inconsistent with the OGSA policy that WS-Names cannot be mandated in OGSA-related specifications. At 08:52 AM 10/26/2005 -0600, Darren Pulsipher wrote: I have spent some considerable time with the document. I know far more than I planned, but it was good time spent. I think we should discuss these items in our meeting on Thursday. I have posted some changes to the document but would like to discuss these items before I make these changes to them. There are some fundamental problems/questions that we need to resolve. 1. How do we resume from where we left off. There is no Resume anymore so we are putting the responsibility to the submitter or controller to keep track of states. We can only use RequestActivityStateChanges to suspend something and if I want to suspend 1000s of activities that means I would need to first do a GetActivityState save the states and then call RequestActivityStateChanges without any of the activities changing state. That is not going to happen. So I propose we do the following: - Add the following interfaces: SuspendActivites(WS-Name[] activityIdentifiers) ResumeActivities(WS-Name[] activityIdentifiers) Or we add one interface PerformOnActivites("Suspend" | "Resume", WS-Name[] activityIdentifiers) 2. Another problem is how do we terminate an activity. The States that are available are "Terminated" and "ShuttingDown". According to the state net the activity is moved to the "ShuttingDown" and then to "Terminated" so requesting RequestActivityStateChanges for activities with "ShuttingDown" means it should move to the ShuttingDown state but it will move the activities to the "Terminated" state. This can be very confusing. So I suggest we add another interface. - Add the following interface: TerminateActivities(WS-Name[] activityIdentifiers) Or add another possible argument to the interface. PerformOnActivites("Suspend" | "Resume" | "Terminate", WS-Name[] activityIdentifiers) 3. Another question that is not clear in the spec is if an action (staging in or staging out) moves into an exception state does that mean that the activity moves into the exception state right away or does it mean that it ever moves into the exception state. Does it mean that if one of the actions moves into an exception state that all other actions are halted and moved to the exception state and then the activity? - I proposed that actions state have no impact on the activity state. If an action has an exception or is terminated then it should not have any impact on the activity what so every. 4. Another not so important change but it might help clarify some confusion and allow for extendibility later on would be to get rid of the "ExecutionPending" and "ExecutionComplete" and make the "Running" state a composite state like the "StagingIn" and "StagingOut" states. This will give the ability to handle complex multiple application execution definitions that the JSDL group is looking at for the next rev of the Job Definition. This will allow for multiple Application actions. This is not workflow. Nor should it be considered the start of workflow. This just allows for extendibility easily. Darren Pulsipher _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org http://www.globus.org/ _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org http://www.globus.org/
participants (2)
-
Andrew Grimshaw -
Ian Foster