Notes: GLUE WG session at OGF 40
Notes from the Wednesday January 15, 2014, OGF 40 GLUE WG session. Describing Clouds ----------------- We revisited the debate on whether Cloud infrastructure should be described using the existing HPC/Grid Compute entities, using new custom Cloud entities that follow the conceptual model as closely as possible, of with a hybrid. Although we recognize there are many similarities, the consensus was that Clouds will benefit from the freedom to describe what they need independently of HPC/Grid at least for the next few years as cloud continue to rapidly evolve. If at some future point it became clear that HPC/Grid and Cloud were very similar and could be merged, that could happen in the context of GLUE 3, whenever that happens. We recognized that naming in the HPC/Grid data models would have to be further generalized if merging happens. Some differences noted between HPC/Grid and Cloud are that: HPC/Grid ApplicationHandle and ApplicationEnvironment have no equivalent in Cloud, and CloudComputeImage has no equivalent hp HPC/Grid. ACTION: The group agreed to request that the GLUE Specification v 2.1 submitted with Cloud extensions be RE-SUBMITTED by the authors with change tracking enabled so that all the changes with respect to the current Specification v2.0 are clearly visible. All other editing by anyone in the group should be change tracked until the group approves, submits, and officially releases v 2.1. Enumerations ------------ We discussed our current enumerations, ServiceType and InterfaceName, and the proposed change process circulated just prior to the meeting. We anticipate the need to use the same process to maintain all other Enumerations relevant to GLUE. We noted that the group has agreed to publish approved Enumerations lists is CSV format so that they are software consumable. We agreed to discuss and possibly approve enumerations and the enumeration process at our next call currently scheduled for January 28. LDAP rendering -------------- Difference between Stephen and Florido have now been resolved. The group agreed that the final editing for comprehensibility should be done by JP and Shiraz: 1. Make sure the purpose of the DIT Appendix is clear and who the target audience (people writing implementation and/or users) 2. Make it clear that the main purpose of the document is to define the LDAP rendering of the schema. Reference to the schema should be included, perhaps in an Appendix 3. JP and Shiraz should report back to working group with a change tracked version for final review BES/JSDL update --------------- Andrew shared a preview of the aspects of GLUE that are being adopted by BES/JSDL. Andrew was going to present that material in separate BES/JSDL OGF 40 sessions. ACTION: Andrew will submit 5 Enumeration requests to the GLUE list. Finally, if anyone else that was present remembers other significant discussions please reply and share them with the group. Regards, JP
Hi all, My apologies for not being able to make the meeting and my thanks to JP for taking these notes and making them available. On 20/01/14 21:38, Navarro, John-Paul F. wrote:
[..] Although we recognize there are many similarities, the consensus was that Clouds will benefit from the freedom to describe what they need independently of HPC/Grid at least for the next few years as cloud continue to rapidly evolve.
I may have missed this, but has anyone attempted to describe the Cloud instances using GLUE 2.0 and the built-in extension points? This would (potentially) be useful for Cloud users as they can use GLUE 2.0 as-is and not have to wait for rolling out a schema update. It would be useful for us because we designed extension points to support exactly this use-case. If it turns out that Cloud services cannot be described in GLUE 2.0 then our extension system is broken and should be updated with GLUE 2.1 So, is anyone attempting to map the Cloud use-cases to GLUE 2.0 (using extension points) ? Cheers, Paul.
Paul Millar [mailto:paul.millar@desy.de] said:
I may have missed this, but has anyone attempted to describe the Cloud instances using GLUE 2.0 and the built-in extension points?
To some extent; have a look at this, which is a commercial cloud provider which has recently joined EGI: ldapsearch -x -h site-bdii.100percentit.com -p 2170 -b GLUE2GroupID=cloud,o=glue That information isn't yet integrated into the top BDII, but the plan is to have that fairly soon.
It would be useful for us because we designed extension points to support exactly this use-case. If it turns out that Cloud services cannot be described in GLUE 2.0 then our extension system is broken and should be updated with GLUE 2.1
The extensions are designed to let you add new attributes to existing objects, they were never intended to let you build a whole new set of objects. Stephen -- Scanned by iCritical.
Hi Stephen, On 21/01/14 16:40, stephen.burke@stfc.ac.uk wrote:
Paul Millar [mailto:paul.millar@desy.de] said:
I may have missed this, but has anyone attempted to describe the Cloud instances using GLUE 2.0 and the built-in extension points?
To some extent; have a look at this, which is a commercial cloud provider which has recently joined EGI:
ldapsearch -x -h site-bdii.100percentit.com -p 2170 -b GLUE2GroupID=cloud,o=glue
That information isn't yet integrated into the top BDII, but the plan is to have that fairly soon.
Nice.
It would be useful for us because we designed extension points to support exactly this use-case. If it turns out that Cloud services cannot be described in GLUE 2.0 then our extension system is broken and should be updated with GLUE 2.1
The extensions are designed to let you add new attributes to existing objects, they were never intended to let you build a whole new set of objects.
I wasn't aware that we had excluded that possibility, but perhaps it comes down to how one defines "new set of objects". From the spec. it is legal to publish Entity (at least, nothing says you can't) and many of the top-level classes (Service, Share, etc). Publishing Domain is expressly forbidden, however. Looking at a (possibly old) copy of the LDAP schema, I see that GLUE2Entity is declared ABSTRACT and GLUE2Service and GLUE2Share are declared STRUCTURAL, so publishing GLUE2Service is allowed (and actively used) but publishing GLUE2Entity isn't allowed. If the "new set of objects" may be described in terms of these top-level objects (Service, Share, etc.) then new kinds of object may be published without schema update. Since Entity has no strong semantics, if Entity was STRUCTURAL instead of ABSTRACT then we could allow almost arbitrary objects to be published, and still provide the same GLUE2 linking and querying abilities. So, we might want to revisit making Entity ABSTRACT in the LDAP schema. (IIRC, there were good reasons for making it ABSTRACT, but I don't remember them :-) Cheers, Paul.
Paul Millar [mailto:paul.millar@desy.de] said:
From the spec. it is legal to publish Entity (at least, nothing says you can't) and many of the top-level classes (Service, Share, etc). Publishing Domain is expressly forbidden, however.
I'm not sure why you think that - many of the class definitions forbid it, e.g. "The Share class is an abstract entity that MUST NOT be instantiated". In fact in LDAP most of them are concrete and you could create prototypes, but not Entity itself so you can't have generic objects. In theory I guess we could define a generic prototype object with an attribute to carry a new type name. However, while that would let you prototype attributes for a new class type I don't think there's any way in the abstract model to prototype relations - a relation is bidirectional so it intrinsically changes the related class definition too. In LDAP you could do it to some extent because relations are just represented by attributes, but it might well be rather clumsy and wouldn't in general be translatable to another rendering technology. You would also reopen the can of worms about the DIT ...
So, we might want to revisit making Entity ABSTRACT in the LDAP schema.
It pretty much isn't possible, for example we put the ID attribute in the derived objects rather than Entity itself. Stephen -- Scanned by iCritical.
Hi all, sorry for being not present all this time but my time for GLUE2 is getting smaller and smaller. On 2014-01-21 18:38, stephen.burke@stfc.ac.uk wrote:
Paul Millar [mailto:paul.millar@desy.de] said:
From the spec. it is legal to publish Entity (at least, nothing says you can't) and many of the top-level classes (Service, Share, etc). Publishing Domain is expressly forbidden, however.
Paul is right, I think this should be fixed in the schema. Domain should be abstract. I missed that. Was there a reason to make it STRUCTURAL? I kept it like that during my revision but it might be changed into ABSTRACT.
I'm not sure why you think that - many of the class definitions forbid it, e.g. "The Share class is an abstract entity that MUST NOT be instantiated". In fact in LDAP most of them are concrete and you could create prototypes, but not Entity itself so you can't have generic objects. In theory I guess we could define a generic prototype object with an attribute to carry a new type name. However, while that would let you prototype attributes for a new class type I don't think there's any way in the abstract model to prototype relations - a relation is bidirectional so it intrinsically changes the related class definition too. In LDAP you could do it to some extent because relations are just represented by attributes, but it might well be rather clumsy and wouldn't in general be translatable to another rendering technology. You would also reopen the can of worms about the DIT ...
So, we might want to revisit making Entity ABSTRACT in the LDAP schema.
It pretty much isn't possible, for example we put the ID attribute in the derived objects rather than Entity itself.
I agree with Stephen, and moreover I don't see the benefit of it... in the end isn't it Service the main entity we always want to model? On Extensions: these are a bad way of extending. We had the same problem while designing EMI-ES. The reason is they increase complexity on the client side, that needs to retrieve those localIDs. A nice way of extending the schema (at least for XML) would be better, and I think we achieved that with the XML rendering doc. For LDAP, it doesn't really matter, due to the nature of LDAP objects that can be composed out of other objects. In my opinion, the Extension method described in GFD.147 it's theoretically nice (can be applied regardless of the technology) but practically cumbersome (every technology has his own good way of extending a schema, and Extensions add unnecessary/unwanted complexity in both LDAP and XML). -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Location: Fysikum, Hus B, Rum B313 Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
Hi All, Sorry for being unresponsive for all this time. A few questions/answers inline. On 2014-01-20 21:38, Navarro, John-Paul F. wrote:
Notes from the Wednesday January 15, 2014, OGF 40 GLUE WG session.
[...] Enumerations ------------ We discussed our current enumerations, ServiceType and InterfaceName, and the proposed change process circulated just prior to the meeting. We anticipate the need to use the same process to maintain all other Enumerations relevant to GLUE. We noted that the group has agreed to publish approved Enumerations lists is CSV format so that they are software consumable. We agreed to discuss and possibly approve enumerations and the enumeration process at our next call currently scheduled for January 28.
Any follow up from this? I could not follow this one too.
LDAP rendering -------------- Difference between Stephen and Florido have now been resolved.
The group agreed that the final editing for comprehensibility should be done by JP and Shiraz: 1. Make sure the purpose of the DIT Appendix is clear and who the target audience (people writing implementation and/or users) 2. Make it clear that the main purpose of the document is to define the LDAP rendering of the schema. Reference to the schema should be included, perhaps in an Appendix 3. JP and Shiraz should report back to working group with a change tracked version for final review
Nice. But don't forget that accepting the document will trigger changes in the LDAP schema that need to be done, committed to git and blessed as official. Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Location: Fysikum, Hus B, Rum B313 Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
On Feb 10, 2014, at 4:06 AM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
On 2014-01-20 21:38, Navarro, John-Paul F. wrote:
Notes from the Wednesday January 15, 2014, OGF 40 GLUE WG session.
[...] Enumerations ------------ We discussed our current enumerations, ServiceType and InterfaceName, and the proposed change process circulated just prior to the meeting. We anticipate the need to use the same process to maintain all other Enumerations relevant to GLUE. We noted that the group has agreed to publish approved Enumerations lists is CSV format so that they are software consumable. We agreed to discuss and possibly approve enumerations and the enumeration process at our next call currently scheduled for January 28.
Any follow up from this? I could not follow this one too.
We discussed the enumeration review process suggestions from Stephen below. Florido, since you've done some work to load initial enumerations into Git, would you be willing to take Stephen's suggestions, modify as you think appropriate, and turn them into a brief set of suggested enumeration maintenance steps that we can discuss at our next meeting. If you don't have time Shiraz or I can do it. At our next meeting we could discuss the proposed process and approve it. JP
From: <stephen.burke@stfc.ac.uk> Subject: Re: [glue-wg] New Endpoint and Service types Date: January 14, 2014 at 6:45:46 AM CST To: <florido.paganelli@hep.lu.se>, <glue-wg@ogf.org>
glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On
Behalf Of Florido Paganelli said: Adding these values for me would be easy. The problem is that the group didn't decide a procedure to _approve_ them to make them _official_, mostly because I promised I would propose one but I never did.
This should be a lightweight process - the main point is to record the values which are in use. In general there's no problem with adding new types on request; the only issues to consider are:
1) Namespace - if it uses a DNS-style name does the requestor reasonably justify relating it to that domain, i.e. do they speak for the relevant project or is it clearly related to a standard service provided by the project.
2) Checking that we don't already have something suitable in the existing list.
3) Sanity checking that the value is naming something that matches the conceptual purpose of the enumeration.
I don't see any reason that can't be done quickly with an email vote. Anyway the worst case is that you end up with an unsuitable value which has to be changed later, and that will generally only cause problems for whoever requested it.
In this particular case I don't see any problem with the QCG-specific names, QCG is a well-defined project so we can assume they know what they're doing, or will sort out their own problems if they don't. org.oasis.notification is more general so it would be useful if someone could do a quick check that it makes sense - personally I know nothing about it.
Stephen
Sorry to be so late to respond, but I disagree with the cloud approach below. As someone involved in the operation of both clouds and clusters, I don't think they are all that different. I think we can/should represent them both using the same schema - in fact, I've already used GLUE2 to represent a couple different IaaS clouds well enough for my needs. I also think that the basic IaaS cloud infrastructure concepts are pretty stable so we don't have a need to have a specific representation for them. Furthermore, the clouds and clusters I'm involved with are part of the same infrastructure (e.g. FutureGrid, the ones we have at TACC, and the ones that will be coming in to XSEDE). I want to use the same schema to represent all of these resources. In my opinion, we should be looking at generalizing glue2 where needed and perhaps adding more specific entities (e.g. instance, cluster job). I think this would benefit cluster environments, too. Warren
-----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of Navarro, John-Paul F. Sent: Monday, January 20, 2014 2:39 PM To: OGF GLUE Working Group Subject: [glue-wg] Notes: GLUE WG session at OGF 40
Notes from the Wednesday January 15, 2014, OGF 40 GLUE WG session.
Describing Clouds ----------------- We revisited the debate on whether Cloud infrastructure should be described using the existing HPC/Grid Compute entities, using new custom Cloud entities that follow the conceptual model as closely as possible, of with a hybrid. Although we recognize there are many similarities, the consensus was that Clouds will benefit from the freedom to describe what they need independently of HPC/Grid at least for the next few years as cloud continue to rapidly evolve. If at some future point it became clear that HPC/Grid and Cloud were very similar and could be merged, that could happen in the context of GLUE 3, whenever that happens. We recognized that naming in the HPC/Grid data models would have to be further generalized if merging happens. Some differences noted between HPC/Grid and Cloud are that: HPC/Grid ApplicationHandle and ApplicationEnvironment have no equivalent in Cloud, and CloudComputeImage has no equivalent hp HPC/Grid.
ACTION: The group agreed to request that the GLUE Specification v 2.1 submitted with Cloud extensions be RE-SUBMITTED by the authors with change tracking enabled so that all the changes with respect to the current Specification v2.0 are clearly visible. All other editing by anyone in the group should be change tracked until the group approves, submits, and officially releases v 2.1.
Enumerations ------------ We discussed our current enumerations, ServiceType and InterfaceName, and the proposed change process circulated just prior to the meeting. We anticipate the need to use the same process to maintain all other Enumerations relevant to GLUE. We noted that the group has agreed to publish approved Enumerations lists is CSV format so that they are software consumable. We agreed to discuss and possibly approve enumerations and the enumeration process at our next call currently scheduled for January 28.
LDAP rendering -------------- Difference between Stephen and Florido have now been resolved.
The group agreed that the final editing for comprehensibility should be done by JP and Shiraz: 1. Make sure the purpose of the DIT Appendix is clear and who the target audience (people writing implementation and/or users) 2. Make it clear that the main purpose of the document is to define the LDAP rendering of the schema. Reference to the schema should be included, perhaps in an Appendix 3. JP and Shiraz should report back to working group with a change tracked version for final review
BES/JSDL update --------------- Andrew shared a preview of the aspects of GLUE that are being adopted by BES/JSDL. Andrew was going to present that material in separate BES/JSDL OGF 40 sessions.
ACTION: Andrew will submit 5 Enumeration requests to the GLUE list.
Finally, if anyone else that was present remembers other significant discussions please reply and share them with the group.
Regards,
JP
_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
participants (5)
-
Florido Paganelli -
Navarro, John-Paul F. -
Paul Millar -
stephen.burke＠stfc.ac.uk -
Warren Smith