Scoping for Job Creation and Management
Folks, Thanks to Ian for the snippet of GRAM. I'll use it as a basis for my action item to start the scoping exercise for this proposed BoF. 1) I would exclude the delegation stuff. This seems to layer well with GRAM (e.g. it is optional) and therefore should be a separate specification. The combined GRAM and delegation usage pattern would be the subject of a Profile. 2) I believe that the ManagedJobFactory::createManagedJob concept is in scope. I think it should take an open content job description so that the current Globus XML description, JSDL, a "run BLAST" thing , and even run Unicore AJO would all work. The Profile might later mandate that for interoperability only JSDL applies. 3) Now that we have a manageable-job, we need operations on it. From Ian's list, I only have a problem with the implied linkage between release and something the the job description (this is an inhibitor to composition, but that discussion it for the new WG). Some might want other control functions: hold, abort (stops execution but doesn't destroy the resource). This specification should be interoperable from the outset, e.g. no extensibility elements. 4) I agree that the data transfer aspects are out of scope, as is provisioning. A JSDL document (as well as the GRAM equivalent) tells the job container where to get the input files, where to put the output files, and what the source for the executable is. How this happens is not the problem of this interface. 5) The specification should be a WSDL interface and associated semantic description. This doesn't mean that only WSDL based presentations are allowed at the BoF or at the WG, just that the eventual output would be a WSDL spec. 6) The new specification must align and compose with the OGSA Base Profile. There some interesting questions here. The GRAM and Unicore interfaces certainly assume the WSRF infrastructure, but it is clearly possible to build job management web services without WSRF. The WG should discuss the extent to which composition (as opposed to extension) it possible without duplication of function. [Just to clarify: We should not include a 'cancel' operation that does the same function as the Resource Lifetime 'destroy' operation, but if the interface makes sense without either, then the dependancy on lifetime could be relaxed.] It should come as no surprise that I would like to see the whole Basic Profile exploited as much as possible, but that's a discussion for later. 7) Job naming, job migration, job scheduling, scheduling candidates, job placement, job restart/recovery, are all out of scope. Thoughts? Do we have a volunteer to start drafting a BoF Call and Draft Charter?
(Need to verify operation names.)DelegationFactory::createDelegation This (optional) step allows a client to delegate credentials that will be required for correct operation of WS GRAM, RFT, or the user's job process. Such credentials are only used when referenced in the subsequent job request and under the condition that WS GRAM or RFT is configured to make use of the DelegationFactory, respectively. Delegation::refresh This (optional) step allows a client to update the credentials already established for use with the previous createDelegation step.
ManagedJobFactory::createManagedJob This required step establishes the stateful ManagedJob resource which implements the job processing described in the input request.
ManagedJob::release This (optional) step allows the ManagedJob to continue through a state in its lifecycle where it was previously held or scheduled to be held according to details of the original job request. It is a fault to try to release a hold that was not set in the job request or that was already released.
ManagedJob::setTerminationTime This (optional) step allows the client to reschedule automatic termination to be different than was originally set during creation of the ManagedJob resource.
ManagedJob::destroy This (optional) step allows the client to explicitly abort a job and destroy the ManagedJob resource, in the event that the scheduled automatic termination time is not adequate. ManagedJob::subscribe This (optional) step allows a client to subscribe for notifications of status (and particularly lifecycle status) of the ManagedJob resource. For responsiveness, it is possible to establish an initial subscription in the createJob() operation without an additional round-trip communication to the newly created job.
ManagedJob::queryProperty and queryMultipleProperties These (optional) steps allow a client to query the status (and particularly lifecycle status) of the ManagedJob resource. --
Take care: Dr. David Snelling < David . Snelling . UK . Fujitsu . com > Fujitsu Laboratories of Europe Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE +44-208-606-4649 (Office) +44-208-606-4539 (Fax) +44-7768-807526 (Mobile)
David: This seems reasonable to me at first glance. The interesting thing is that there is not that much there, if one assumes WSRF/WSN. Ian. At 11:09 AM 1/18/2005 +0000, David Snelling wrote:
Folks,
Thanks to Ian for the snippet of GRAM. I'll use it as a basis for my action item to start the scoping exercise for this proposed BoF.
1) I would exclude the delegation stuff. This seems to layer well with GRAM (e.g. it is optional) and therefore should be a separate specification. The combined GRAM and delegation usage pattern would be the subject of a Profile.
2) I believe that the ManagedJobFactory::createManagedJob concept is in scope. I think it should take an open content job description so that the current Globus XML description, JSDL, a "run BLAST" thing , and even run Unicore AJO would all work. The Profile might later mandate that for interoperability only JSDL applies.
3) Now that we have a manageable-job, we need operations on it. From Ian's list, I only have a problem with the implied linkage between release and something the the job description (this is an inhibitor to composition, but that discussion it for the new WG). Some might want other control functions: hold, abort (stops execution but doesn't destroy the resource). This specification should be interoperable from the outset, e.g. no extensibility elements.
4) I agree that the data transfer aspects are out of scope, as is provisioning. A JSDL document (as well as the GRAM equivalent) tells the job container where to get the input files, where to put the output files, and what the source for the executable is. How this happens is not the problem of this interface.
5) The specification should be a WSDL interface and associated semantic description. This doesn't mean that only WSDL based presentations are allowed at the BoF or at the WG, just that the eventual output would be a WSDL spec.
6) The new specification must align and compose with the OGSA Base Profile. There some interesting questions here. The GRAM and Unicore interfaces certainly assume the WSRF infrastructure, but it is clearly possible to build job management web services without WSRF. The WG should discuss the extent to which composition (as opposed to extension) it possible without duplication of function. [Just to clarify: We should not include a 'cancel' operation that does the same function as the Resource Lifetime 'destroy' operation, but if the interface makes sense without either, then the dependancy on lifetime could be relaxed.] It should come as no surprise that I would like to see the whole Basic Profile exploited as much as possible, but that's a discussion for later.
7) Job naming, job migration, job scheduling, scheduling candidates, job placement, job restart/recovery, are all out of scope.
Thoughts?
Do we have a volunteer to start drafting a BoF Call and Draft Charter?
(Need to verify operation names.)DelegationFactory::createDelegation This (optional) step allows a client to delegate credentials that will be required for correct operation of WS GRAM, RFT, or the user's job process. Such credentials are only used when referenced in the subsequent job request and under the condition that WS GRAM or RFT is configured to make use of the DelegationFactory, respectively.
Delegation::refresh This (optional) step allows a client to update the credentials already established for use with the previous createDelegation step.
ManagedJobFactory::createManagedJob This required step establishes the stateful ManagedJob resource which implements the job processing described in the input request.
ManagedJob::release This (optional) step allows the ManagedJob to continue through a state in its lifecycle where it was previously held or scheduled to be held according to details of the original job request. It is a fault to try to release a hold that was not set in the job request or that was already released.
ManagedJob::setTerminationTime This (optional) step allows the client to reschedule automatic termination to be different than was originally set during creation of the ManagedJob resource.
ManagedJob::destroy This (optional) step allows the client to explicitly abort a job and destroy the ManagedJob resource, in the event that the scheduled automatic termination time is not adequate.
ManagedJob::subscribe This (optional) step allows a client to subscribe for notifications of status (and particularly lifecycle status) of the ManagedJob resource. For responsiveness, it is possible to establish an initial subscription in the createJob() operation without an additional round-trip communication to the newly created job.
ManagedJob::queryProperty and queryMultipleProperties These (optional) steps allow a client to query the status (and particularly lifecycle status) of the ManagedJob resource. --
Take care:
Dr. David Snelling < David . Snelling . UK . Fujitsu . com > Fujitsu Laboratories of Europe Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE
+44-208-606-4649 (Office) +44-208-606-4539 (Fax) +44-7768-807526 (Mobile)
_______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org
Ian, On 19 Jan 2005, at 1:32, Ian Foster wrote:
This seems reasonable to me at first glance.
The interesting thing is that there is not that much there, if one assumes WSRF/WSN.
Yes I know. If you assume WSRF and WSN and take an aggressive line we have a minimalist solution: MJobFactory_create (JSDL_Doucment, Initial_TT) -> EPR MJob implements WSRF-RP, WSRF-LT, WS-BN MJob_status as a resource property And you're done. Subscribe to value change notification on 'status' and you pretty much can do all of GRAM, minus the Globus specific bits. I don't think I would go for this minimal, but it was an interesting exercise. NB: I know JSDL pretty well and there's a lot there, which helps. -- Take care: Dr. David Snelling < David . Snelling . UK . Fujitsu . com > Fujitsu Laboratories of Europe Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE +44-208-606-4649 (Office) +44-208-606-4539 (Fax) +44-7768-807526 (Mobile)
David: As you know JSDL pretty well (certainly better than I, sorry to say), can you comment on whether the GT4 GRAM JDL (described at http://www-unix.globus.org/toolkit/docs/development/3.9.4/execution/wsgram/s...) can be rendered in JSDL? If the answer is yes, then I'd say first of all "good" and second "well that means we have work to do in terms of defining a standard vocabulary." One more thought regarding your earlier email concerning scoping: you proposed that we take delegation off the table. However, one reason for delegating credentials is so that the job manager host can initiate staging commands. Ian. At 10:32 AM 1/19/2005 +0000, David Snelling wrote:
Ian,
On 19 Jan 2005, at 1:32, Ian Foster wrote:
This seems reasonable to me at first glance.
The interesting thing is that there is not that much there, if one assumes WSRF/WSN.
Yes I know. If you assume WSRF and WSN and take an aggressive line we have a minimalist solution:
MJobFactory_create (JSDL_Doucment, Initial_TT) -> EPR
MJob implements WSRF-RP, WSRF-LT, WS-BN MJob_status as a resource property
And you're done.
Subscribe to value change notification on 'status' and you pretty much can do all of GRAM, minus the Globus specific bits. I don't think I would go for this minimal, but it was an interesting exercise. NB: I know JSDL pretty well and there's a lot there, which helps.
--
Take care:
Dr. David Snelling < David . Snelling . UK . Fujitsu . com > Fujitsu Laboratories of Europe Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE
+44-208-606-4649 (Office) +44-208-606-4539 (Fax) +44-7768-807526 (Mobile)
_______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org
Ian, I want to answer these separately, so a few mails over the next few hours, as I should be pay attention in a meeting. On 20 Jan 2005, at 0:40, Ian Foster wrote:
As you know JSDL pretty well (certainly better than I, sorry to say), can you comment on whether the GT4 GRAM JDL (described at http://www-unix.globus.org/toolkit/docs/development/3.9.4/execution/ wsgram/schemas/mjs_job_description.html) can be rendered in JSDL?
If the answer is yes, then I'd say first of all "good" and second "well that means we have work to do in terms of defining a standard vocabulary."
Later, I want to do a short summary of JSDL ant the same time and I still need to finish looking at JDL.
One more thought regarding your earlier email concerning scoping: you proposed that we take delegation off the table. However, one reason for delegating credentials is so that the job manager host can initiate staging commands.
This is a simpler answer. I would leave delegation off the scope of this group simply to separate concerns. Whatever the security mechanisms in place at the job management service, they should be composable with the job management interface. I know that some form of delegation is needed to manage the file staging, but a delegation service is not the only solution, so I am loathe to bind the two together is a specification. For example, the Unicore implementation would not require and delegation service. Lastly, if we need such a service, it should be done in the security area and be more widely applicable. -- Take care: Dr. David Snelling < David . Snelling . UK . Fujitsu . com > Fujitsu Laboratories of Europe Hayes Park Central Hayes End Road Hayes, Middlesex UB4 8FE +44-208-606-4649 (Office) +44-208-606-4539 (Fax) +44-7768-807526 (Mobile)
participants (2)
-
David Snelling -
Ian Foster