June 2005 - ogsa-bes-wg - lists.ogf.org

Discussion on resource models in GGF14
by Fred Maciel 23 Jun '05

23 Jun '05

Hi, Apparently we won't have a teleconference tomorrow, so I'm sending this request for a joint sesson in GGF14 to the mailing list. We were discussing the about having joint sessions in GGF14 between OGSA-BES-WG and the OGSA Resource Management design team to discuss resource modeling for the container. In particular, it's time to discuss our first results on a small subset of CIM that is suitable for BES (see my previous e-mail, copied below, and the attached file). Given that the RM session on the 28th (Tuesday) clashes with the EMS-RSS-UCWG session, and I won't be in GGF on the 30th due to the WSDM F2F, we have two options on the 29th (Wednesday): - 7:30 - 9:00: AM OGSA RM design team session - 4:00 - 5:30: PM OGSA-BES session When you start planning the OGSA-BES agenda for GGF14, please schedule a joint session. Regards, Fred Maciel Hitachi America R&D --------------------------------------- Hi, We decided in the F2F in London to use JSDL as a set of properties for BES, and some people decided to look into CIM as a possibility. The initial results of this work are in the attached spreadsheet. The three columns on the left are the JSDL elements, and the following two ones are the corresponding CIM classes and properties. (BTW, there seem to be several glitches on the GGF mailing lists, and apparently Andrea's e-mails which had this spreadsheet didn't reach this mailing list). Our conclusion for now is that by using a number of CIM classes and associations, with some extensions, we can represent what is in JSDL, both for the status of the container and activities and also the requirements submitted as a JSDL document. So, this CIM subset is suitable to use as a model for BES. (Notice that I'm talking about the model semantics only -- how you represent this on Web services is a different issue, which we will tackle later). The discussions we had the two RM design team teleconferences were quite interesting, with for instance discussions on how the JSDL entities map to the container and activities. Working on the resource model will help clarifying the semantics of the BES entities. Andrew, please add a discussion on the model to the next OGSA-BES teleconference (in case we are having one). Regards, Fred Maciel Hitachi America R&D

1 0

Meeting Cancelled
by Darren Pulsipher 16 Jun '05

16 Jun '05

The meeting this morning is cancelled. Darren

1 0

BES properties -- JSDL and CIM
by Fred Maciel 15 Jun '05

15 Jun '05

Hi, We decided in the F2F in London to use JSDL as a set of properties for BES, and some people decided to look into CIM as a possibility. The initial results of this work are in the attached spreadsheet. The three columns on the left are the JSDL elements, and the following two ones are the corresponding CIM classes and properties. (BTW, there seem to be several glitches on the GGF mailing lists, and apparently Andrea's e-mails which had this spreadsheet didn't reach this mailing list). Our conclusion for now is that by using a number of CIM classes and associations, with some extensions, we can represent what is in JSDL, both for the status of the container and activities and also the requirements submitted as a JSDL document. So, this CIM subset is suitable to use as a model for BES. (Notice that I'm talking about the model semantics only -- how you represent this on Web services is a different issue, which we will tackle later). The discussions we had the two RM design team teleconferences were quite interesting, with for instance discussions on how the JSDL entities map to the container and activities. Working on the resource model will help clarifying the semantics of the BES entities. Andrew, please add a discussion on the model to the next OGSA-BES teleconference (in case we are having one). Regards, Fred Maciel Hitachi America R&D

1 0

[Fwd: [jsdl-wg] JSDL 1.0 (draft 21) submitted to GGF Editor]
by Andreas Savva 15 Jun '05

15 Jun '05

FYI. (And apologies to those also on the jsdl-wg list.) -- Andreas Savva Fujitsu Laboratories Ltd

1 0

RM teleconference reminder -- container properties
by Fred Maciel 14 Jun '05

14 Jun '05

Hi, On the next teleconference we will continue discussing the container's properties, as we did last week. I and Andrea are working on a strawman mapping JSDL and CIM (the attached file is what we have as of now). Examples of things to discuss: - Review the mapping - What in JSIM and CIM apply to the container? For instance, there are more properties in a container than just JSIM (e.g. properties for monitoring). - How do Individual* and Total* JSIM properties apply to the container? Total* is across containers? - etc. My intention is to take the results of this teleconference and present on the OGSA-BES teleconference on Thursday. Date: Tuesday, June 14th, at the following times: - 11:00-12:00 US Pacific (GMT - 7) - 12:00-13:00 US Mountain (GMT - 6) - 13:00-14:00 US Central (GMT - 5) - 14:00-15:00 US Eastern (GMT - 4) - 18:00-19:00 GMT - 19:00-20:00 UK (GMT + 1) - 20:00-21:00 Central Europe (GMT + 2) - 03:00-04:00 Japan (GMT + 9, following day) -- sorry. Dial-in: Toll Free: (888) 422-7120 Int'l Access: (608) 250-0194 Participant code: 693305 If screen sharing is needed, let's use Glance: URL: http://ogsa.glance.net Session key: 0614 Explanation: http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/06/msg00077.html See you there, Fred Maciel Hitachi America R&D

1 0

food for thought
by Christopher Smith 09 Jun '05

09 Jun '05

I just wanted to send out something for people to read in preparation for the discussion on job management interfaces tomorrow. For a nice resource modelling neutral example, I'll focus on the APIs that the SAGA research group has proposed for job management, since the operations (if not their rendering) are pretty much what I would propose. Surprise, surprise ... it looks a lot like GRAM/LSF/DRMAA/<insert your favourite job management system here>. The attached document is the full job management interface for SAGA, but I've pasted the job object itself below, since that what the focus of the discussion will be. -- Chris interface Job { getJobId (out string jobId); getJobState (out JobState state); getJobInfo (out JobInfo info); getJobDefinition (out JobDefinition jobDef); getJobExitStatus (out JobExitStatus exitStatus); suspend (); resume (); hold (); release (); checkpoint (); migrate (in JobDefinition jobDef); terminate (); signal (in int signum); }

1 0

comments on scoping... (london f2f minutes)
by Karl Czajkowski 08 Jun '05

08 Jun '05

Dear OGSA-BES-WG at large: After reading the minutes from the May London f2f, I have some concerns about scoping and terminology. It seems that discussions went all over the map compared to what I gathered from the last telecon's quick summary... First off, there are (I think) three conceptual tiers or planes related to the whole execution management problem as it relates to "simple" targets like BES: 1. The allocable resource. This is the pool of capabilities which are consumed by executing processes and which are to different extents managed/scheduled/etc. For example, the pool of compute nodes w/ their respective CPUs, RAM, and interconnection hardware. 2. The "container-level" resource management service. This is the logical entity which manages (1) by accepting requests with embedded parameters such as the JSDL job description and enacting the described tasks. For example, GRAM is a Globus Toolkit container-level service that is implemented by mapping to one of several local resource managers. Many of these local managers have service interfaces in their own right. 3. The "application" or "activity". This is the domain-specific process that comes to life as a result of the execution, e.g. it is _hosted_ by (2) and consumes (1). For example, an HPC linear algebra code, or a service like NetSOLVE, or a web server or any other service that consists of a program running for some length of time. I am belaboring this point because I think the minutes show a confusion between these different tiers. There is not really a factory/child relationship between the layers, but rather each layer can be conceptually decomposed into groups and instances which have mappings between the layers: Resource Layer Container Layer Application Layer Pool <--manages--- Manager <--requests-- (Application Manager) A A A | partOf | hostedBy? | controlledBy? | | | Allocation <-uses- Job/Activity -XYZ---> App. Service XYZ=instantiates/hosts/something like that... It seems to me that most BES discussions are calling the container-level manager above "the container", but not really naming the container-level job/activity representation. This second item is what GRAM calls the Job, but it is in fact sort of a container for the single job while the manager has a set of such containers that he has created as virtualizations of the allocations that are carved out of the resource pool. For completeness, this container instance would be represented by an Agreement if BES used WS-Agreement combined with JSDL. Often, the discussions in BES seem to confusingly jump to "the activity" equally "the application", e.g. saying that the interfaces of the activity are domain-specific. I think this is wrong. The application service that is realized by the activity, e.g. by running the executable referred to in a JSDL POSIXApplication, certainly has domain-specific interfaces. However, there is a generic container-level activity which always has the same generic container interface, e.g. "POSIX activity". BES must define the interface for the activity, whether or not they are rendered as separate WSRF resources. It seems to me that BES should be rigorously observing these distinctions and seeing itself as a service for provisioning and management of application services. In other words, BES should stick to a container-level abstraction for all its stateful semantics and only reflect on the resource or application layers in its advertisement and introspection interface: A. It should have the metamodels necessary to reflect on the resource pool and allocations as they relate to discovery, selection, and monitoring of BES service instances. i. Acceptable job configurations/resource requests, e.g. basic resource pool info PLUS policies. ii. Availability/load, e.g. dynamic restrictions on (A.i) iii. Allocation plans reflecting the current/future assignment of resources to activities, e.g. the complement of (A.ii). iii. Detailed "usage records" reflecting the consumption of resource layer capabilities by current/past activities, e.g. specialization of (A.iii) w/ monitor/accounting info. B. It should define the protocol for provisioning and managing application services. i. The createActivity pattern. ii. In-scope state-changing operations on activities, e.g. signals. Specialization of BES may add things like suspend/resume and checkpoint-migration controls. iii. The metamodel for introspecting on existing activities, e.g. the "job states". iv. The cancel/destroy activity pattern(s). C. It might want to have a metamodel or metadata path for reflecting on application-specific status i. Return codes passed from application to container on exit. ii. Heartbeat or other optional reporting channels. iii. (Domain-specific) rendezvous/contact info optionally registered by application. D. It should NOT define a management interface for managing the BES service instance itself. i. NOT handling deploy/start/stop of BES. ii. This could come from WSDM or related similar specs. iii. Or recursive application of a "static" BES service to manage the provisioning of dynamic BES instances! A big can of worms here though, in separately provisioning the service logic and its own resource pool... sounds like a job for WS-Agreement to me. :-) Is this a useful map for defining the scope of BES? Am I wrong in feeling that the BES discussions are still getting lost in what these entities are, or where the boundaries lie? karl -- Karl Czajkowski karlcz(a)univa.com

1 0

Pointers on resource management specs
by Fred Maciel 07 Jun '05

07 Jun '05

Hi, Here is the list of pointers on resource management specs that I had promised. It's updated from what I posted in may to the OGSA-WG mailing list. - JSIM: the spec (GFD-I.028) is at: https://forge.gridforum.org/projects/ggf-editor/document/GFD.28/en/1 There is also a UML diagram: https://forge.gridforum.org/projects/ogsa-wg/document/JSIM_PDF/en/1 - CIM: the schemas are under: http://www.dmtf.org/standards/cim JSIM is in 2.9.1 preliminary. Try 2.9.0 to get also UML files. - JSDL: the latest snapshot is at: https://forge.gridforum.org/projects/jsdl-wg/document/draft-ggf-jsdl-spec/e… There's a nice introduction at: https://forge.gridforum.org/projects/jsdl-wg/document/JSDL-Introduction/en/ - GLUE 1.1: the schemas are in http://www.cnaf.infn.it/~sergio/datatag/glue/index.htm There's a good presentation on GLUE 1.1: http://www.dma.unina.it/~murli/SummerSchool/presentations/InformationModeli… - GLUE 1.2: the development can be followed in the mailing list archives -- see the "Mailing list (archive)" link in the URL for the GLUE 1.1 schemas above. The latest draft can be found under: http://www.hicb.org/pipermail/glue-schema/2005/frm00057.html There is a very good presentation at: http://infnforge.cnaf.infn.it/cdsagenda//askArchive.php?base=agenda&categ=a… Regards, Fred Maciel

4 4

RM teleconference on the 7th: sync with OGSA-BES
by Fred Maciel 03 Jun '05

03 Jun '05

Hi, And now for something slightly different. OGSA-BES-WG decided to use CIM to model the container, and we have to work on the semantics (choosing what to include from CIM) and the rendering (how we show that). We will use the Resource Management (RM) teleconference as a way to join the CIM experts and the related OGSA-BES participants to discuss the details. Examples of things to discuss: - Do we choose the container properties by CIM classes or by CIM property? What do do if a property in a class has nothing to do with the container? - How to render these classes and properties so that they can be accessed? - What are the properties of a container? Jobs? Operating system? Host information such as CPU architecture? - How this relates to the short-term milestone of showing JSIM attributes? This teleconference will be an important synchronization point between OGSA-BES and the RM design team, and a golden opportunity to get lots of questions on CIM answered. On the other hand, we won't have time to discuss the CIM-GLUE-JSDL comparision in this teleconference. Date: Tuesday, June 7th, at the following times: - 11:00-12:00 US Pacific (GMT - 7) - 12:00-13:00 US Mountain (GMT - 6) - 13:00-14:00 US Central (GMT - 5) - 14:00-15:00 US Eastern (GMT - 4) - 18:00-19:00 GMT - 19:00-20:00 UK (GMT + 1) - 20:00-21:00 Central Europe (GMT + 2) - 03:00-04:00 Japan (GMT + 9, following day) -- sorry. Dial-in: Toll Free: (888) 422-7120 Int'l Access: (608) 250-0194 Participant code: 693305 If screen sharing is needed, let's use Glance: URL: http://ogsa.glance.net Session key: 0607 Explanation: http://www-unix.gridforum.org/mail_archive/ogsa-wg/2004/06/msg00077.html See you there, Fred Maciel Hitachi America R&D

1 0