Teleconference minutes - 2 November 2005
Minutes attached. https://forge.gridforum.org/projects/ogsa-wg/document/minutes-20051102/en/1 -- Andreas Savva Fujitsu Laboratories Ltd OGSA Teleconference - 2 November 2005 ===================================== * Participants Jay Unger (IBM) Jem Treadwell (HP) Ellen Stokes (IBM) Chris Smith (Platform) Andreas Savva (Fujitsu) Steven Newhouse (OMII) Mark Morgan (UVa) Steve McGough (Imperial) Tom Maguire (EMC) Fred Maciel (Hitachi) Hiro Kishimoto (Fujitsu) Andrew Grimshaw (UVa) Dave Berry (NeSC) Michael Behrens (R2AD, LLC) Minutes: Andreas Savva * October 26 minutes approved with no changes * Information Model discussion - Postponed for next Wednesday - Action: Andrew to identify the attributes needed for BES container * EMS Sequence review - Reviewed sequence diagram and text sent to the list http://www-unix.gridforum.org/mail_archive/ogsa-wg/2005/11/msg00003.html The diagram has not changed since the last review. There is additional text explaining the steps. - An open issue is the meaning of the 'abstract JSDL document'. - In the simplest case a document that does not specify a particular resource (host) for execution. - A more abstract case is that of specifyin only an application name and data (input, output) and leaving the rest to the system. - Taking the OS as an example, if the OS element is defined then only that specific OS can be used. If left undefined then any OS can be used. There is no in between choice with JSDL 1.0. The expectation is that JSDL should be combined with some other language to provide more flexible descriptions. - (WS-Agreement is one option but the assumption is that it is not used in this sequence. ) - Other existing work on preferences or selection policy? - EGEE uses Condor classads (with EGEE extensions) - Globus (no, since no there is no scheduler) - Can the Policy document may be empty? Then would just get whichever resource the EPS considers good enough. - Job management is out of scope of this scenario - How about job termination? It was discussed last time and is out of scope at this stage. - But it is a good idea to add the simplest case of 'control' returning to JM at the end of the job. - Agreed that this should be a separate sequence diagram. * Naming discussion - There was substantial confusion with the statements made at the last F2F. - Confirmed that the gist of the statement was: "In OGSA pointers are EPRs that may or may not be WS-Names." - The additional question is whether individuals WGs, because of the service specifications they are defining, can choose to require WS-Names (their reasons for doing so may vary). - Why does a service require EPR comparison? Does it fall in the category of "Nice but..." - One example is to distinguish EPRs for identity purposes. - And one common usage may be to enable additional functionality for monitoring, etc. - The expectation is that even if some OGSA services mint WS-Names and other OGSA services deal with them, those other OGSA services may also have to tolerate vanilla EPRs because there may be non-OGSA services in the system. - Does the requirement for WS-Names have to be part of the interface signature or is it more of a compliance statement that should be in a profile instead? - Is it an optimization? Can there be implemenations that functionally work without this feature, even if sub-optimally? So to make this feature mandatory becomes an overstatement. - It is also putting the onus on the client to manage the identities vs requiring the service to provide an operation. - Filesystem traversal as one(another) case where such an optimization (WS-Name) is useful. - The logging use case mentioned earlier is a commonly cited one. But in Jay's experience the identifiers have to be explicitly designed for the specific system and do not generalize. - Also the 'unique-in-space-and-time' statement is another point of disagreement. - To guarantee uniqueness there are two approaches: 1) arbitrator; or 2) agreed upon algorithm (with some unique input) used by everyone. - Agreed that something closer to (2) is expected but it is more of a best-effort. So the space-time guarantee is not expected to hold literally. - Noone on the call doubts the usefulness of WS-Names in helping build quality implementations. The disagreement is whether they must be mandated in the interface signatures of basic specs. - The open issue is what is the statement OGSA should make to the fellow WGs. Is this issue something that can be left to each individual WG or should it be tackled by the Architecture (so within OGSA-WG). - Hiro proposed to write up a document defining the problem, what are the requirements and proposed approach; and circulate to interested parties. Continue discussion on the list and possibly re-review next week with Ian Foster and Frank Siebenlist. - Tom volunteered to start a draft * UML Tool discussion - postponed
A more accurate answer would be that "Globus has lots of different selectors", in that given the building blocks provided by MDS and GRAM, one can build many different strategies (as was our intention), and indeed people do just that . Thus, we see e.g.: * Policy-driven schedulers that use published information about site policies to determine where to send work (e.g., Dumitrescu). * Performance-driven schedulers that use information about past performance at different sites to determine where to send work. * Load-driven schedulers that use information about current load at different sites to determine where to send work. * Reservation-based schedulers that use advance reservations. (This only in research systems so far, due to lack of advance reservation support in most production deployments.) * Condor matchmaking * Alternative matchmaking strategies based on extended ClassAd-like systems * Schedulers that use information about data location to determine where to send work. * Coschedulers that use DUROC-like mechanisms to initiate computation only on sites that turn out to be available. I need to see more details on what is being proposed in terms of a selector interface, but I would be concerned if there was an attempt to incorporate this as a primitive. As the brief and far-from-comprehensive list above shows, there are many different resource selection strategies that may be applied, and these strategies may be applied in many different ways and in different places. Ian. At 06:37 PM 11/4/2005 +0900, Andreas Savva wrote:
- Other existing work on preferences or selection policy? - EGEE uses Condor classads (with EGEE extensions) - Globus (no, since no there is no scheduler)
_______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org
Thanks Ian for your feedback, This is "a" sequence diagram not "the" diagram. I think in this diagram, EMS expects to have one of scheduler (from your list) acts as EPS. We have not yet discussed how to find out and start talking to appropriate scheduler. EMS should have flexible framework to allow wide variety of schedulers. And also have a common interface and protocol for interoperability and plugability. Let's discuss these issues at the series of EMS calls. ---- Hiro Kishimoto Ian Foster wrote:
A more accurate answer would be that "Globus has lots of different selectors", in that given the building blocks provided by MDS and GRAM, one can build many different strategies (as was our intention), and indeed people do just that . Thus, we see e.g.:
* Policy-driven schedulers that use published information about site policies to determine where to send work (e.g., Dumitrescu).
* Performance-driven schedulers that use information about past performance at different sites to determine where to send work.
* Load-driven schedulers that use information about current load at different sites to determine where to send work.
* Reservation-based schedulers that use advance reservations. (This only in research systems so far, due to lack of advance reservation support in most production deployments.)
* Condor matchmaking
* Alternative matchmaking strategies based on extended ClassAd-like systems
* Schedulers that use information about data location to determine where to send work.
* Coschedulers that use DUROC-like mechanisms to initiate computation only on sites that turn out to be available.
I need to see more details on what is being proposed in terms of a selector interface, but I would be concerned if there was an attempt to incorporate this as a primitive. As the brief and far-from-comprehensive list above shows, there are many different resource selection strategies that may be applied, and these strategies may be applied in many different ways and in different places.
Ian.
At 06:37 PM 11/4/2005 +0900, Andreas Savva wrote:
- Other existing work on preferences or selection policy? - EGEE uses Condor classads (with EGEE extensions) - Globus (no, since no there is no scheduler)
_______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org
Original posting: http://www-unix.gridforum.org/mail_archive/ogsa-wg/2005/11/msg00006.html
A more accurate answer would be that "Globus has lots of different selectors", in that given the building blocks provided by MDS and GRAM, one can build many different strategies (as was our intention), and indeed people do just that .
No disagreement with your comments. As you note there are plenty of ways that the GRAM service can be used as part of a brokering infrastructure - but AFAIK GRAM itself does not do scheduling internally. In the context of the discussion during the call (which may not have come across in the notes) we where examining the interactions between services where the decision had already been made as to which service to use. We have other scenarios - coming soon - that will include the selection element.
I need to see more details on what is being proposed in terms of a selector interface, but I would be concerned if there was an attempt to incorporate this as a primitive. As the brief and far-from-comprehensive list above shows, there are many different resource selection strategies that may be applied, and these strategies may be applied in many different ways and in different places.
A selector interface is being defined as part of the RSS activities (https://forge.gridforum.org/projects/ogsa-rss-wg/) . They may have a comment to make on the nature of the interface they are considering - and how it could/could not encompass these services. Regards, Steven -- ---------------------------------------------------------------- Dr Steven Newhouse Tel:+44 (0)2380 598789 Deputy Director, Open Middleware Infrastructure Institute (OMII) Suite 6005, Faraday Building (B21), Highfield Campus, Southampton University, Highfield, Southampton, SO17 1BJ, UK
Steven Newhouse wrote:
A selector interface is being defined as part of the RSS activities (https://forge.gridforum.org/projects/ogsa-rss-wg/) . They may have a comment to make on the nature of the interface they are considering - and how it could/could not encompass these services.
I (as OGSA-RSS co-chair) welcome any input people have over possible strategies. At the moment, I'm thinking in terms of ordered sets where the caller supplies a ranking function[*] (or input to such a function). That gives plenty of flexibility and power. I'd love to have input from other people as to what sorts of things ought to go in the ranking function parameter; we could leave it unspecified (easy for the RSS WG!) but that's not good for working towards an interoperable solution so I'd like to seed the space with at least a minimal level of things that must be supported. Donal Fellows. [* Experience with Condor indicates that a "general acceptability" filter is also useful, but a first hack at that is a straight resource satisfaction check which ought to be performed anyway. ]
participants (5)
-
Andreas Savva -
Donal K. Fellows -
Hiro Kishimoto -
Ian Foster -
Steven Newhouse