Resource Model Discussion during OGSA telecon
Hi! I just read the protocol of last nights OGSA-telecon: " * Resource model and BES discussion - It is unclear how or where the resource modelling work should continue now that BES has put it out of scope. - One option is to do this work in the context of RSS or perhaps even spawn a new group dedicated to resource modelling. The coordination problem is not insignificant, however. It is not yet clear how to proceed with this work. - BES put this work out of scope on the assumption that selection is over by the time a request reaches the BES container. (Also due to the fear that if the resource modelling work was taken on then it would cause the overall BES work to stall.) - Since RSS deals with resource selection it is probably a better (right?) place to do this modelling effort. - There is a concern that leaving this modelling work for later may cause rework of specs. This is not intended to stop ongoing work until the resource modelling is finished but to make sure people are aware of the risk. - BES is not doing modelling so the problem may not be that big - RSS would have to tackle this work---so it is probably going to be tackled relatively soon The consensus is that this topic needs more discussion. - Agreed that interested parties should discuss this offline, followed by discussing it on the BES call this week, followed by discussing it again in the OGSA call next Monday. - This activity is part of the coordination between OGSA & OGSA-BES WG " I think this is an very important discussion. In Chicago, we decided that the defintion of an information system is out of scope, but what about the underlying resource model? Basically, we need to identify which resource is capable of executing a job described in JSDL (Candidate Set Generator) and which resource to choose from this set (Execution Planning Service). While the first part is rather easy to answer (we need all the attributes used in the JSDL), the second part seems to be a kind of Pandora's box: The attributes vary greatly with the implementation of the scheduler. While something like load or queue length of the resource seems to be common, we can also have "network utilization", "existing reservations", "availability between 2 and 3 p.m. tomorrow"... In addition, the OGSA Information System session in Chicago revealed some other aspects: What about information necessary to manage resource (e.g. uptime, status, admin contact, ...) There are existing models like CIM which has been investigated by Fred Maciel. The design of an information system has been investigated by Abdeslem Djaoui. This seems to be a lot of work. So before making a statement on the OGSA mailing list, I would like to collect some opinions. What do you think about the resource model? Should we deal with it? Is it out of scope? Do we have the knowledge to deal with it? Can we - if it is out of scope - provide input to another team? Cheers, Mathias
Mathias Dalheimer wrote:
I just read the protocol of last nights OGSA-telecon: [...] I think this is an very important discussion. In Chicago, we decided that the defintion of an information system is out of scope, but what about the underlying resource model? Basically, we need to identify which resource is capable of executing a job described in JSDL (Candidate Set Generator) and which resource to choose from this set (Execution Planning Service). [...]
We really need to decide what our approach on all this is, and we need to decide *soon* before other people make us do work we don't want to. Let us start by examining the single-job case. In this case, the CSG must take an atomic job description and return a set of candidate ways of executing the job, and the EPS must select between these candidates according to some criteria. This means that we need an information model about the resources available, an information model about the atomic job requests, an information model that describes the "fitness" of a particular job candidate, and an information model of the selection criteria. Let's start by trying to leverage what's out there already. :^) Suggest (but don't require) CIM as the resource information model; that will play well politically in any case. Require JSDL+ (where the + stands for whatever bits other groups or particular schedulers require) for the atomic job descriptions. The fitness descriptor info-model we'll have to define (hopefully extensibly), and I'm totally unsure about what to do on selection criteria; some implementations might not want to use user-configurable selectors. Now, let's think about the more complex multi-job case. The problem here is that we don't have a standardized job workflow language. (The closest thing to a standard is BPEL, but that operates at a completely different level.) This makes life hard; without a standardized job workflow language, we can't say how a multi-job would be expressed to the EPS in the first place. On the other hand, for the CSG it is really much simpler; supporting this case really boils down to just being able to handle a vector of JSDL job descriptions instead of just one at a time (experience shows that this sort of bunching is a Good Idea), and that's a trivial extension. I don't know how scheduling and reservation interact with all this. The other big question is what about data and network handling? This matters for the EPS (and I don't know if the CSG will be reusable for this sort of task) but our draft charter just talks about computational job handling...
This seems to be a lot of work. So before making a statement on the OGSA mailing list, I would like to collect some opinions. What do you think about the resource model? Should we deal with it? Is it out of scope? Do we have the knowledge to deal with it? Can we - if it is out of scope - provide input to another team?
My feeling is "resource info model is out of scope" since the CSG can insulate the EPS from a whole host of different resource info models in the first place. We're not designing the implementation of the service. While we could provide input to people working on an RIM, I do not feel that it should be an explicit goal of our WG; there most certainly isn't a single strategy for candidate set generation... (And I'm not sure if I've answered the question.) Donal.
Hi all, I agree with Donal that resource modeling thing is something that we should avoid handling in the RSS because of the same concern that BES group has that "if the the modeling work was taken on then it would cause the overall BES work to stall." We'd rather focus on coming up with the definition of interface and protocols of EPS and CSG as in our charter draft. My gut is telling me that it might be a good idea to start our work by making it as simple as possible. I mean, we might want to deal with only single-job case in this very beginning. If we could come up with something more clear about what to do with the single-job case, then we might have some idea as to what to do with the multi-job case such as workflows, master/workers. Let's focus on a single-job case, which, I think, gives us freedom to exploit the JSDL v1 specification for the job description, especially for job's resource requirement description. As Donal pointed out, It seems that there are three things we have to think hard right now: (Assuming our focus is on the single-job case) 1) Resource modeling for job requirement: We may want to take a look at the JSDL resource part. I mean, we may want to restrict ourselves on job's resource requirements supported *only* by JSDL v1 spec. This restriction would make our lives much easier. We don't have to deal with the whole resource modeling thing. 2) Resource capability modeling As a start point, we may want to take a look at Globus MDS and Unicore IDB Resource modeling. How do computing resources publish their capabilities in the Globus and Unicore? What kinds of resource capabilities attributes are used in Globus, Unicore and Condor? We could come up with the most commonly used resource capabilities attributes. 3) Selection Criteria Now that we have the ways of describing 1) job's resource requirements and 2) resource capabilities, the CSG can simply generates a list of resources available for the execution of job with simple, dumb matchmaking mechanism only based on resource requirements and resource capabilities. As a selection criteria used for in the EPS, I am suggesting a ranking mechanism. To this end, we need to come up with some mechanism for making it possible to assign weight (or preference) to each of resource requirement attributes. With this simple weight mechanism, EPS would be able to choose the best resource that fit the job's requirement among the resources in the candidate set. For this ranking mechanism, we may want to investigate the Condor's matchmaking algorithm. Standardization of the condor's proprietary matchmaking algorithm in the OGSA RSS context would, if we can, be certainly our contribution to the Grid community. Regards, Soonwook -----Original Message----- From: owner-ogsa-rss-bof@ggf.org [mailto:owner-ogsa-rss-bof@ggf.org] On Behalf Of Donal K. Fellows Sent: Tuesday, July 12, 2005 5:47 PM To: ogsa-rss-bof@ggf.org Subject: Re: [ogsa-rss-bof] Resource Model Discussion during OGSA telecon Mathias Dalheimer wrote:
I just read the protocol of last nights OGSA-telecon: [...] I think this is an very important discussion. In Chicago, we decided that the defintion of an information system is out of scope, but what about the underlying resource model? Basically, we need to identify which resource is capable of executing a job described in JSDL (Candidate Set Generator) and which resource to choose from this set (Execution Planning Service). [...]
We really need to decide what our approach on all this is, and we need to decide *soon* before other people make us do work we don't want to. Let us start by examining the single-job case. In this case, the CSG must take an atomic job description and return a set of candidate ways of executing the job, and the EPS must select between these candidates according to some criteria. This means that we need an information model about the resources available, an information model about the atomic job requests, an information model that describes the "fitness" of a particular job candidate, and an information model of the selection criteria. Let's start by trying to leverage what's out there already. :^) Suggest (but don't require) CIM as the resource information model; that will play well politically in any case. Require JSDL+ (where the + stands for whatever bits other groups or particular schedulers require) for the atomic job descriptions. The fitness descriptor info-model we'll have to define (hopefully extensibly), and I'm totally unsure about what to do on selection criteria; some implementations might not want to use user-configurable selectors. Now, let's think about the more complex multi-job case. The problem here is that we don't have a standardized job workflow language. (The closest thing to a standard is BPEL, but that operates at a completely different level.) This makes life hard; without a standardized job workflow language, we can't say how a multi-job would be expressed to the EPS in the first place. On the other hand, for the CSG it is really much simpler; supporting this case really boils down to just being able to handle a vector of JSDL job descriptions instead of just one at a time (experience shows that this sort of bunching is a Good Idea), and that's a trivial extension. I don't know how scheduling and reservation interact with all this. The other big question is what about data and network handling? This matters for the EPS (and I don't know if the CSG will be reusable for this sort of task) but our draft charter just talks about computational job handling...
This seems to be a lot of work. So before making a statement on the OGSA mailing list, I would like to collect some opinions. What do you think about the resource model? Should we deal with it? Is it out of scope? Do we have the knowledge to deal with it? Can we - if it is out of scope - provide input to another team?
My feeling is "resource info model is out of scope" since the CSG can insulate the EPS from a whole host of different resource info models in the first place. We're not designing the implementation of the service. While we could provide input to people working on an RIM, I do not feel that it should be an explicit goal of our WG; there most certainly isn't a single strategy for candidate set generation... (And I'm not sure if I've answered the question.) Donal.
participants (3)
-
Donal K. Fellows -
Mathias Dalheimer -
Soonwook Hwang