Hi all, I was thinking a little more about Florido's request to nest 'grouping' elements as children of the 'Entities' element bag (the doc root element), i.e. wrap all related elements that have the same 'xsd:substitutionGroup' using grouping elements: '<Services>' for wrapping all service implementations, '<Endpoints>' for grouping all endpoint implementations etc. I think these group elements could be useful for simplifying XPath queries and for visualizing large documents with tools that can collapse/expand elements (see attached image, also uploaded at the following link). Therefore, please find v2 of the flat schema with examples at the following link (Documents/Drafts/Renderings/Flat Schema/Flat_v2): http://redmine.ogf.org/dmsf/glue-wg?folder_id=32 Note, this is only a small modification to the flat schema located in the parent directory - the XSD is still essentially 'flat' and uses element ID references to represent entity relationships (as opposed to strict hierarchical nesting). Cheers, David

-----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of david.meredith@stfc.ac.uk Sent: 30 August 2012 17:11 To: florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: Re: [glue-wg] flat xml rendering

Hi Florido, Comments interleaved below. Cheers, David

...

* grouping elements commented out. is it possible to have them? any issue with that? I'd also like to have a <Services> grouping. Balazs said it was taken out at a certain moment, but I'd like it back.

I am not really sure why you would need more grouping elements? The 'Entities' element is the document root element. This element wraps the other glue entity elements in an predefined order. Therefore, the different types of entity elements are always grouped together in established locations within 'Entities.' For example, on line 91 the 'AbstractService' element positions the different types of service element (Service, ComputingService, StorageService) after <Domain> elements (AdminDomain, UserDomain) but before <AbstractEndpoint> elements (Endpoint, ComputingEndpoint, StorageEndpoint).

...

* Naive question: in glue2_2SampleDoc1.xml, the <glue:...> prefix is added to every element. Is that mandatory? A consumer should search within the namespaces in the header, isn't it like that?

You just need to add the default namespace declaration for the glue2 namespace to the Entities element and you can drop all the '<glue:...>' prefixes. Sample below (note the addition of the default namespace declaration within Entities: 'xmlns='http://schemas.ogf.org/glue/2009/03/spec_2.0_r1'):

<?xml version="1.0" encoding="UTF-8"?> <Entities xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://schemas.ogf.org/glue/2009/03/spec_2.0_r1' xmlns:glue='http://schemas.ogf.org/glue/2009/03/spec_2.0_r1' xsi:schemaLocation='http://schemas.ogf.org/glue/2009/03/spec_2.0_r1 glue2_2.xsd'> <Location> <ID>location1</ID> <Associations> <ServiceID>service1</ServiceID> </Associations> </Location> <Contact> <ID>contact1</ID> <Detail>http://some.uri/embedding/contact/info</Detail> <Type>general</Type> <Associations> <ServiceID>service1</ServiceID> </Associations> </Contact> </Entities>

...

I already know my XML document will not validate due to the absent groupings. I will comment those out and check if it works.

I doubt that your documents will validate as this updated schema is flat, not hierarchical and a number of other changes have been made too.

...

Cheers, -- Florido Paganelli Lund University - Particle Physics ARC Middleware EMI Project

On 2012-06-29 14:36, david.meredith@stfc.ac.uk wrote:

...

Hi all,

Following OGF 35 I have produced an updated draft of the XSD: http://tools.ngs.ac.uk/ngstools/glue2proposal/flat.zip

This rendering has a ‘flat’ style and includes the global <Entities> element bag with abstract elements/substitution groups. There are a few choices that still need to be addressed which are highlighted with TODO annotations (please do a search on ‘TODO’ in glue2_2.xsd). A sample instance document is also provided (glue2_2SampleDoc1.xml).

For the purposes of demonstration, there is also a sample XSD (sampleGlue2_2Extension.xsd) that imports the glue2_2.xsd to define new service and endpoint element types. This shows how future profiles could extend the glue XSD in order to define new sub-type specialisations (also see the sample instance doc, sampleGlue2_2ExtensionDoc.xml).

I guess the next step is to do another telephone conf to agree/disagree. If possible, i would prefer to do this before spending too much effort on updating the Word doc.

Cheers,

David

==========================

David Meredith Scientific Computing Technology eScience Daresbury Laboratory Tel: +44 (0) 1925 603762 Fax: +44 (0) 1925 603100 (Site) Email: david.meredith@stfc.ac.uk

Skype name: davidismeredith

-- Scanned by iCritical.

_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg

_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg

My general opinion is not to have any -ies elements (e.g. ComputingActivities) to group elements. To me, this grouping of elements (e.g. ComputingActivity) isn't needed in XML. It doesn't seem to make it any easier to process the XML using SAX or DOM. It seems easy to find all -y elements in a document without a -ies element as a parent for visualizing. Similarly, it also seems easy to find -y elements in an XML document if you want to translate it to LDAP or SQL or whatever. I haven't really done much xPath... Can someone explain how grouping elements would help? It seems like either way you'd include 'ComputingActivity' in the expression. That is the expression would start with "/Entities/ComputingActivities/ComputingActivity..." or "/Entities/ComputingActivity...". Warren -----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of stephen.burke@stfc.ac.uk Sent: Friday, August 31, 2012 5:20 AM To: david.meredith@stfc.ac.uk; david.meredith@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: Re: [glue-wg] flat xml rendering glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On

Behalf Of david.meredith@stfc.ac.uk said: I think these group elements could be useful for simplifying XPath queries and for visualizing large documents with tools that can collapse/expand elements (see attached image, also uploaded at the following link).

Bear in mind that in the production Grid there are about 4000 endpoints, so grouping may not help a lot with visualisation unless you do it more finely. Stephen _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg

Ok, I see what you are saying - I'm just now looking at the schema and I had some assumptions about it that weren't correct. However, after a little reading about XPath, it looks like there are a few ways that you can return all activities without requiring an Activities element as a parent. /Entities/*[contains(local-name(),'Activity')] - returns all elements under Entities that have 'Activity' in them /Entities/*[substring(name(), string-length(name()) - 7) = 'Activity'] - returns all elements under Entities that end in 'Activity' These XPath expressions aren't as nice as '/Entities/Activities/*', but they aren't that bad and I don't know how often this case will come up. In XPath 2.0, it looks like there is an 'ends-with()' function that allows a much simpler version of the 2nd expression above. I personally wouldn't use the additional complexity of these XPath statements as a reason to add *s parent elements. Sorry that I had to miss the last telecon since I was on vacation, so I apologize if some of my comments about the schema were discussed: * My preference is to do away with all ...s grouping elements. I still don't see them as being needed (or desired). * If the decision is to keep the ...s grouping elements (I hope not! :-P ), the schema should probably use them consistently (e.g. wherever the element has maxOccurs="unbounded"). They aren't being used right now in quite a few spots: Location, Contact, the elements at the end of ExtensibleEntities_t, and number of other complexType definitions. * Why is there a BaseType="..." attribute on many of the elements? It seems like redundant information and looking at the schema, I can't figure out why they are needed... * The Extension representation is slightly different from how I defined them in the TeraGrid schema. This schema has elements for LocalID and Key (as well as Value). For the TeraGrid schema, I had an Extension element with Key as an attribute (I forgot to include a LocalID attribute), and the text of the Extension element was the value. This schema mostly avoids attributes, so using elements probably makes more sense... That's all I notice at this point. Warren -----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Friday, August 31, 2012 10:00 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering Hi Warren, In the glue2_2.xsd at http://redmine.ogf.org/dmsf/glue-wg?folder_id=32, there is no '<ComputingActivites>,' only '<Activities>' which serves to group the elements that are substitutable for the 'AbstractActivity' substitutionGroup (so currently this includes both 'Activity' and 'ComputingActivity' - and of course any activities that may be profiled in the future that may also define the same substitutionGroup). The relevant XSD fragment is as follows (line 139) - note that it simply serves as a wrapper around AbstractActivity: <element name="Activities" minOccurs="0" maxOccurs="1"> <complexType> <sequence> <element ref="glue:AbstractActivity" minOccurs="0" maxOccurs="unbounded"/> </sequence> </complexType> </element> Therefore, given the following instance doc: <Entities... > ... <glue:Activities> <glue:Activity BaseType="Activity"> <glue:ID>http://activity1.ac.uk/1</glue:ID> <glue:Associations/> </glue:Activity> <glue:ComputingActivity BaseType="Activity"> <glue:ID>computingActivity1</glue:ID> <glue:State>some state</glue:State> <glue:Owner>someone</glue:Owner> <glue:Associations/> </glue:ComputingActivity> ...here nest more elements that can substitute for the AbstractActivity substitutionGroup... </glue:Activities> ... </Entities> To select all of the different flavours of Activity in a single query, you could use the following XPath: '/Entities/Activities/*'. Similarly, for all services; '/Entities/Services/*' and for all Endpoints; '/Entities/Endpoints/*' etc. I guess it depends if you think this is useful. Without the 'Activities' element, to select all the different types of activity, you would currently need to issue two XPath queries and concat the two docs: '/Entities/Activity' and '/Entities/ComputingActivity'. Personally, I think it is useful to be able to select/collapse all of the Services, Endpoints, Activities etc with one button click or query, especially when considering large documents that may have 1000s of endpoints listed. Either way, it's only a small change from the version that does not define these grouping elements (http://redmine.ogf.org/dmsf/glue-wg?folder_id=31) David

-----Original Message----- From: Warren Smith [mailto:wsmith@tacc.utexas.edu] Sent: 31 August 2012 14:48 To: Burke, Stephen (STFC,RAL,PPD); Meredith, David (STFC,DL,SC); Meredith, David (STFC,DL,SC); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

My general opinion is not to have any -ies elements (e.g. ComputingActivities) to group elements. To me, this grouping of elements (e.g. ComputingActivity) isn't needed in XML. It doesn't seem to make it any easier to process the XML using SAX or DOM. It seems easy to find all -y elements in a document without a -ies element as a parent for visualizing. Similarly, it also seems easy to find -y elements in an XML document if you want to translate it to LDAP or SQL or whatever.

I haven't really done much xPath... Can someone explain how grouping elements would help? It seems like either way you'd include 'ComputingActivity' in the expression. That is the expression would start with "/Entities/ComputingActivities/ComputingActivity..." or "/Entities/ComputingActivity...".

Warren

-----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of stephen.burke@stfc.ac.uk Sent: Friday, August 31, 2012 5:20 AM To: david.meredith@stfc.ac.uk; david.meredith@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: Re: [glue-wg] flat xml rendering

glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On

...

Behalf Of david.meredith@stfc.ac.uk said: I think these group elements could be useful for simplifying XPath queries and for visualizing large documents with tools that can collapse/expand elements (see attached image, also uploaded at the following link).

Bear in mind that in the production Grid there are about 4000 endpoints, so grouping may not help a lot with visualisation unless you do it more finely.

Stephen

_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg -- Scanned by iCritical.

I still disagree for a few reasons. The grouping elements do add a tiny bit of complexity, particularly if you feel (like I do) that they are unnecessary. A "flat" argument along these lines is that for FutureGrid (and perhaps for XSEDE in the near future), I produce GLUE 2 documents that only contain a single ComputingActivity. These documents describe updates to jobs that I'm pulling from scheduler log files. Using an Activities element in this document is really unnecessary. Regarding new elements, I don't think speculation about adding new elements in the future is a strong argument for having the XML schema a certain way. I'm generally wary of "just in case" designing and we already seem to have a naming convention that includes the base type in the element name in almost all cases. In the very few current cases where an element name doesn't indicate what substitution group it is in, I also don't mind consulting the schema to determine this - that's one of the reasons why we have a schema. Another thought on the grouping elements is which ones to use? If we use them, I don't necessarily agree that using the substitution groups or base complexTypes is the best approach. It may be logical for us as the schema creators, but it may not be logical to users. For example, as a user, in a single document, I'd like to have compute-related information in one area and storage-related information in another for easier viewing. A different way to put it is that just because two objects are in the same substitution group, that doesn't mean that that is the strongest relationship and they should be grouped together. The above argument can also be applied to the ordering of the elements in the schema - I just noticed that this schema orders elements by substitution group. In the TeraGrid GLUE 2 XML schema, I ordered them in the order they are presented in GFD.147: base/abstract first, then compute, then storage. I think I personally prefer this ordering (because it matches GFD.147 and because it fits the way I look at things), but it doesn't seem critical. Heh, I honestly find the BaseType attribute more objectionable than the grouping entities. It just seems redundant, like something that would rarely, if ever, be used, and it just looks odd when viewing the XML document. Warren -----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Monday, September 17, 2012 3:43 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering hi Warren, all, (in response to Warren's response, cc'd below): I think that having the ...s grouping elements does not add any complexity and at the same time provides a slightly more structured document; it provides a fixed location/bag for those elements that implement a particular glue2 xsd:substitutionGroup (this is why the grouping elements are not defined universally for all the elements in <Entities> - these other elements are concrete and cannot be substituted for different element implementations).  Now, consider new elements that may come along in future glue profiles that also define one of the glue2 xsd:substitutionGroups; looking at an instance document, it is not obvious that a new element (e.g. new activty, new service, new endpoint etc) is a substitute for a particular substitutionGroup without consulting the schema. I think this is also the original reason for the BaseType="..." attribute (which I carried over from the original nested schema); it also serves to clearly indicate the elements base type without having to consult the schema or make any inferences from the elements name (a per your modified XPath below). It is also easy to build XPath to say "select all entities that have BaseType=X" (which is similar to the xpath for selecting all elements of a particular type, e.g. select all Activities '/Entities/Activities/*').  Now, I do admit that there is a simple elegance in having all the glue entities as equal siblings within <Entities> (no ...s grouping elements), and there may be a good technical reason to keep all entities as siblings with the same level of nesting.... Therefore, at minimum I think it necessary to keep the 'BaseType=...' attributes, but its not really a big deal (at least to me) to keep or drop the '...s' grouping elements, even though I do find it v.useful to be able to collapse/expand the ...s grouping elements when browsing through an instance doc (if all the elements are equal siblings, i can't do this).  I'll be giving a talk at the EGI TF on GLUE2 XML renderings and can provide some examples, hopefully we will gain some more insights.  cheers, david -------------------------------------------------------------- David Meredith STFC eScience Centre Daresbury Laboratory (A32) Warrington, Cheshire WA4 4AD email: david.meredith@stfc.ac.uk tel: +44 (0)1925 603762 ________________________________________ From: Warren Smith [wsmith@tacc.utexas.edu] Sent: 06 September 2012 23:50 To: Meredith, David (STFC,DL,SC); Burke, Stephen (STFC,RAL,PPD); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering Ok, I see what you are saying - I'm just now looking at the schema and I had some assumptions about it that weren't correct. However, after a little reading about XPath, it looks like there are a few ways that you can return all activities without requiring an Activities element as a parent. /Entities/*[contains(local-name(),'Activity')] - returns all elements under Entities that have 'Activity' in them /Entities/*[substring(name(), string-length(name()) - 7) = 'Activity'] - returns all elements under Entities that end in 'Activity' These XPath expressions aren't as nice as '/Entities/Activities/*', but they aren't that bad and I don't know how often this case will come up. In XPath 2.0, it looks like there is an 'ends-with()' function that allows a much simpler version of the 2nd expression above. I personally wouldn't use the additional complexity of these XPath statements as a reason to add *s parent elements. Sorry that I had to miss the last telecon since I was on vacation, so I apologize if some of my comments about the schema were discussed: * My preference is to do away with all ...s grouping elements. I still don't see them as being needed (or desired). * If the decision is to keep the ...s grouping elements (I hope not! :-P ), the schema should probably use them consistently (e.g. wherever the element has maxOccurs="unbounded"). They aren't being used right now in quite a few spots: Location, Contact, the elements at the end of ExtensibleEntities_t, and number of other complexType definitions. * Why is there a BaseType="..." attribute on many of the elements? It seems like redundant information and looking at the schema, I can't figure out why they are needed... * The Extension representation is slightly different from how I defined them in the TeraGrid schema. This schema has elements for LocalID and Key (as well as Value). For the TeraGrid schema, I had an Extension element with Key as an attribute (I forgot to include a LocalID attribute), and the text of the Extension element was the value. This schema mostly avoids attributes, so using elements probably makes more sense... That's all I notice at this point. Warren -----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Friday, August 31, 2012 10:00 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering Hi Warren, In the glue2_2.xsd at http://redmine.ogf.org/dmsf/glue-wg?folder_id=32, there is no '<ComputingActivites>,' only '<Activities>' which serves to group the elements that are substitutable for the 'AbstractActivity' substitutionGroup (so currently this includes both 'Activity' and 'ComputingActivity' - and of course any activities that may be profiled in the future that may also define the same substitutionGroup). The relevant XSD fragment is as follows (line 139) - note that it simply serves as a wrapper around AbstractActivity: <element name="Activities" minOccurs="0" maxOccurs="1"> <complexType> <sequence> <element ref="glue:AbstractActivity" minOccurs="0" maxOccurs="unbounded"/> </sequence> </complexType> </element> Therefore, given the following instance doc: <Entities... > ... <glue:Activities> <glue:Activity BaseType="Activity"> <glue:ID>http://activity1.ac.uk/1</glue:ID> <glue:Associations/> </glue:Activity> <glue:ComputingActivity BaseType="Activity"> <glue:ID>computingActivity1</glue:ID> <glue:State>some state</glue:State> <glue:Owner>someone</glue:Owner> <glue:Associations/> </glue:ComputingActivity> ...here nest more elements that can substitute for the AbstractActivity substitutionGroup... </glue:Activities> ... </Entities> To select all of the different flavours of Activity in a single query, you could use the following XPath: '/Entities/Activities/*'. Similarly, for all services; '/Entities/Services/*' and for all Endpoints; '/Entities/Endpoints/*' etc. I guess it depends if you think this is useful. Without the 'Activities' element, to select all the different types of activity, you would currently need to issue two XPath queries and concat the two docs: '/Entities/Activity' and '/Entities/ComputingActivity'. Personally, I think it is useful to be able to select/collapse all of the Services, Endpoints, Activities etc with one button click or query, especially when considering large documents that may have 1000s of endpoints listed. Either way, it's only a small change from the version that does not define these grouping elements (http://redmine.ogf.org/dmsf/glue-wg?folder_id=31) David

-----Original Message----- From: Warren Smith [mailto:wsmith@tacc.utexas.edu] Sent: 31 August 2012 14:48 To: Burke, Stephen (STFC,RAL,PPD); Meredith, David (STFC,DL,SC); Meredith, David (STFC,DL,SC); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

My general opinion is not to have any -ies elements (e.g. ComputingActivities) to group elements. To me, this grouping of elements (e.g. ComputingActivity) isn't needed in XML. It doesn't seem to make it any easier to process the XML using SAX or DOM. It seems easy to find all -y elements in a document without a -ies element as a parent for visualizing. Similarly, it also seems easy to find -y elements in an XML document if you want to translate it to LDAP or SQL or whatever.

I haven't really done much xPath... Can someone explain how grouping elements would help? It seems like either way you'd include 'ComputingActivity' in the expression. That is the expression would start with "/Entities/ComputingActivities/ComputingActivity..." or "/Entities/ComputingActivity...".

Warren

-----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of stephen.burke@stfc.ac.uk Sent: Friday, August 31, 2012 5:20 AM To: david.meredith@stfc.ac.uk; david.meredith@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: Re: [glue-wg] flat xml rendering

glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On

...

Behalf Of david.meredith@stfc.ac.uk said: I think these group elements could be useful for simplifying XPath queries and for visualizing large documents with tools that can collapse/expand elements (see attached image, also uploaded at the following link).

Bear in mind that in the production Grid there are about 4000 endpoints, so grouping may not help a lot with visualisation unless you do it more finely.

Stephen

_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg -- Scanned by iCritical. -- Scanned by iCritical.

I don't think I was arguing against the substitution groups, I think I was arguing against the reasoning that since we may add new GLUE2 entities in the future, we should have the -ies elements (e.g. Activities) at the top level to group them so that a user knows what elements are Activities without having to look at the schema. So yeah, it seems like we haven't convinced each other, so maybe what we should do is describe each of the choices, get everyone who is interested to vote (or whatever), make a choice and move forward. Warren -----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Tuesday, October 02, 2012 5:45 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering Hi Warren, all, Comments interleaved below. While I disagree with some of your comments below, I still don't think any of these points are significant blockers. Cheers, David

-----Original Message----- From: Warren Smith [mailto:wsmith@tacc.utexas.edu] Sent: 20 September 2012 13:51 To: Meredith, David (STFC,DL,SC); Burke, Stephen (STFC,RAL,PPD); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

I still disagree for a few reasons.

The grouping elements do add a tiny bit of complexity, particularly if you feel (like I do) that they are unnecessary. A "flat" argument along these lines is that for FutureGrid (and perhaps for XSEDE in the near future), I produce GLUE 2 documents that only contain a single ComputingActivity. These documents describe updates to jobs that I'm pulling from scheduler log files. Using an Activities element in this document is really unnecessary.

I agree that it adds a 'tiny bit of extra complexity'. I guess we leave this to the WG to decide.

Regarding new elements, I don't think speculation about adding new elements in the future is a strong argument for having the XML schema a certain way. I'm generally wary of "just in case" designing and we already seem to have a naming convention that includes the base type in the element name in almost all cases. In the very few current cases where an element name doesn't indicate what substitution group it is in, I also don't mind consulting the schema to determine this - that's one of the reasons why we have a schema.

I disagree. Substitution groups follow the good coding principle of 'Open for extension, but closed for modification'. In addition, substitution groups realize the conceptual spec more closely which explicitly states that "Grid services requiring a richer set of attributes for the [Endpoint|Service], specific models MAY be derived by specializing from the [Endpoint|Service] class and adding new properties or relationships." Without substitution groups, the XSD doesn't support this.

Another thought on the grouping elements is which ones to use? If we use them, I don't necessarily agree that using the substitution groups or base complexTypes is the best approach. It may be logical for us as the schema creators, but it may not be logical to users. For example, as a user, in a single document, I'd like to have compute-related information in one area and storage-related information in another for easier viewing. A different way to put it is that just because two objects are in the same substitution group, that doesn't mean that that is the strongest relationship and they should be grouped together.

The above argument can also be applied to the ordering of the elements in the schema - I just noticed that this schema orders elements by substitution group. In the TeraGrid GLUE 2 XML schema, I ordered them in the order they are presented in GFD.147: base/abstract first, then compute, then storage. I think I personally prefer this ordering (because it matches GFD.147 and because it fits the way I look at things), but it doesn't seem critical.

I disagree. For me, the benefits provided using substitution groups far outweigh the disadvantages of less strict ordering when ordering by substitution group.

Heh, I honestly find the BaseType attribute more objectionable than the grouping entities. It just seems redundant, like something that would rarely, if ever, be used, and it just looks odd when viewing the XML document.

Regardless of BaseType attribute or grouping entity, I still think it is useful to explicitly indicate what is the base type of an element within the XML instance doc without having to explicitly refer to the XSD; both approaches enable simple XPath style queries on a doc e.g. 'select all activities' or 'select all endpoints'. I think this is valuable.

Warren

-----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Monday, September 17, 2012 3:43 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue- wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

hi Warren, all, (in response to Warren's response, cc'd below):

I think that having the ...s grouping elements does not add any  complexity and at the same time provides a slightly more structured document; it provides a fixed location/bag for those elements that implement a particular glue2 xsd:substitutionGroup (this is why the grouping elements are not defined universally for all the elements in <Entities> - these other elements are concrete and cannot be substituted for different element implementations).

Now, consider new elements that may come along in future glue profiles that also define one of the glue2 xsd:substitutionGroups; looking at an instance document, it is not obvious that a new element (e.g. new activty, new service, new endpoint etc) is a substitute for a particular substitutionGroup without consulting the schema. I think this is also the original reason for the BaseType="..." attribute (which I carried over from the original nested schema); it also serves to clearly indicate the elements base type without having to consult the schema or make any inferences from the elements name (a per your modified XPath below). It is also easy to build XPath to say "select all entities that have BaseType=X" (which is similar to the xpath for selecting all elements of a particular type, e.g. select all Activities '/Entities/Activities/*').

Now, I do admit that there is a simple elegance in having all the glue  entities as equal siblings within <Entities> (no ...s grouping elements), and there may be a good technical reason to keep all entities as siblings with the same level of nesting....

Therefore, at minimum I think it necessary to keep the 'BaseType=...' attributes, but its not really a big deal (at least to me) to keep or drop the '...s' grouping elements, even though I do find it v.useful to be able to collapse/expand the ...s grouping elements when browsing through an instance doc (if all the elements are equal siblings, i can't do this).

I'll be giving a talk at the EGI TF on GLUE2 XML renderings and can provide some examples, hopefully we will gain some more insights.

cheers, david

-------------------------------------------------------------- David Meredith STFC eScience Centre Daresbury Laboratory (A32) Warrington, Cheshire WA4 4AD

email: david.meredith@stfc.ac.uk tel: +44 (0)1925 603762

________________________________________ From: Warren Smith [wsmith@tacc.utexas.edu] Sent: 06 September 2012 23:50 To: Meredith, David (STFC,DL,SC); Burke, Stephen (STFC,RAL,PPD); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

Ok, I see what you are saying - I'm just now looking at the schema and I had some assumptions about it that weren't correct. However, after a little reading about XPath, it looks like there are a few ways that you can return all activities without requiring an Activities element as a parent.

/Entities/*[contains(local-name(),'Activity')] - returns all elements under Entities that have 'Activity' in them

/Entities/*[substring(name(), string-length(name()) - 7) = 'Activity'] - returns all elements under Entities that end in 'Activity'

These XPath expressions aren't as nice as '/Entities/Activities/*', but they aren't that bad and I don't know how often this case will come up. In XPath 2.0, it looks like there is an 'ends-with()' function that allows a much simpler version of the 2nd expression above. I personally wouldn't use the additional complexity of these XPath statements as a reason to add *s parent elements.

Sorry that I had to miss the last telecon since I was on vacation, so I apologize if some of my comments about the schema were discussed:

* My preference is to do away with all ...s grouping elements. I still don't see them as being needed (or desired).

* If the decision is to keep the ...s grouping elements (I hope not! :-P ), the schema should probably use them consistently (e.g. wherever the element has maxOccurs="unbounded"). They aren't being used right now in quite a few spots: Location, Contact, the elements at the end of ExtensibleEntities_t, and number of other complexType definitions.

* Why is there a BaseType="..." attribute on many of the elements? It seems like redundant information and looking at the schema, I can't figure out why they are needed...

* The Extension representation is slightly different from how I defined them in the TeraGrid schema. This schema has elements for LocalID and Key (as well as Value). For the TeraGrid schema, I had an Extension element with Key as an attribute (I forgot to include a LocalID attribute), and the text of the Extension element was the value. This schema mostly avoids attributes, so using elements probably makes more sense...

That's all I notice at this point.

Warren

-----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Friday, August 31, 2012 10:00 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue- wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

Hi Warren, In the glue2_2.xsd at http://redmine.ogf.org/dmsf/glue-wg?folder_id=32, there is no '<ComputingActivites>,' only '<Activities>' which serves to group the elements that are substitutable for the 'AbstractActivity' substitutionGroup (so currently this includes both 'Activity' and 'ComputingActivity' - and of course any activities that may be profiled in the future that may also define the same substitutionGroup). The relevant XSD fragment is as follows (line 139) - note that it simply serves as a wrapper around AbstractActivity:

<element name="Activities" minOccurs="0" maxOccurs="1"> <complexType> <sequence> <element ref="glue:AbstractActivity" minOccurs="0" maxOccurs="unbounded"/> </sequence> </complexType> </element>

Therefore, given the following instance doc:

<Entities... > ... <glue:Activities> <glue:Activity BaseType="Activity"> <glue:ID>http://activity1.ac.uk/1</glue:ID> <glue:Associations/> </glue:Activity>

<glue:ComputingActivity BaseType="Activity"> <glue:ID>computingActivity1</glue:ID> <glue:State>some state</glue:State> <glue:Owner>someone</glue:Owner> <glue:Associations/> </glue:ComputingActivity>

...here nest more elements that can substitute for the AbstractActivity substitutionGroup... </glue:Activities> ... </Entities>

To select all of the different flavours of Activity in a single query, you could use the following XPath: '/Entities/Activities/*'. Similarly, for all services; '/Entities/Services/*' and for all Endpoints; '/Entities/Endpoints/*' etc. I guess it depends if you think this is useful. Without the 'Activities' element, to select all the different types of activity, you would currently need to issue two XPath queries and concat the two docs: '/Entities/Activity' and '/Entities/ComputingActivity'.

Personally, I think it is useful to be able to select/collapse all of the Services, Endpoints, Activities etc with one button click or query, especially when considering large documents that may have 1000s of endpoints listed. Either way, it's only a small change from the version that does not define these grouping elements (http://redmine.ogf.org/dmsf/glue-wg?folder_id=31)

David

...

-----Original Message----- From: Warren Smith [mailto:wsmith@tacc.utexas.edu] Sent: 31 August 2012 14:48 To: Burke, Stephen (STFC,RAL,PPD); Meredith, David (STFC,DL,SC); Meredith, David (STFC,DL,SC); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

My general opinion is not to have any -ies elements (e.g. ComputingActivities) to group elements. To me, this grouping of elements (e.g. ComputingActivity) isn't needed in XML. It doesn't seem to make it any easier to process the XML using SAX or DOM. It seems easy to find all -y elements in a document without a -ies element as a parent for visualizing. Similarly, it also seems easy to find -y elements in an XML document if you want to translate it to LDAP or SQL or whatever.

I haven't really done much xPath... Can someone explain how grouping elements would help? It seems like either way you'd include 'ComputingActivity' in the expression. That is the expression would start with "/Entities/ComputingActivities/ComputingActivity..." or "/Entities/ComputingActivity...".

Warren

-----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of stephen.burke@stfc.ac.uk Sent: Friday, August 31, 2012 5:20 AM To: david.meredith@stfc.ac.uk; david.meredith@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: Re: [glue-wg] flat xml rendering

glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On

...

Behalf Of david.meredith@stfc.ac.uk said: I think these group elements could be useful for simplifying XPath queries and for visualizing large documents with tools that can collapse/expand elements (see attached image, also uploaded at the following link).

Bear in mind that in the production Grid there are about 4000 endpoints, so grouping may not help a lot with visualisation unless you do it more finely.

Stephen

_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg -- Scanned by iCritical. -- Scanned by iCritical. -- Scanned by iCritical.

Hi all, I agree with most of David's comments. As for the arguments about the ordering of the items, I don't really care. I actually hate order, and my poor knowledge of XML does not allow me to understand its benefit. If any of you has a strong argument about it, please shed some light. Those I read up to now didn't convince me of any benefit so far. Warren's following the GFD147 for me is not an argument, I think is just a style issue, rather academic somewhat. David's ordering by substitution group is just an easy way to read through the XSD code for substitution groups. Frankly, I don't think we shouldn't spend too much time discussing it... On 10/02/2012 12:45 PM, david.meredith@stfc.ac.uk wrote:

Hi Warren, all, Comments interleaved below. While I disagree with some of your comments below, I still don't think any of these points are significant blockers.

Cheers, David

...

-----Original Message----- From: Warren Smith [mailto:wsmith@tacc.utexas.edu] Sent: 20 September 2012 13:51 To: Meredith, David (STFC,DL,SC); Burke, Stephen (STFC,RAL,PPD); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

I still disagree for a few reasons.

The grouping elements do add a tiny bit of complexity, particularly if you feel (like I do) that they are unnecessary. A "flat" argument along these lines is that for FutureGrid (and perhaps for XSEDE in the near future), I produce GLUE 2 documents that only contain a single ComputingActivity. These documents describe updates to jobs that I'm pulling from scheduler log files. Using an Activities element in this document is really unnecessary.

I agree that it adds a 'tiny bit of extra complexity'. I guess we leave this to the WG to decide.

...

Regarding new elements, I don't think speculation about adding new elements in the future is a strong argument for having the XML schema a certain way. I'm generally wary of "just in case" designing and we already seem to have a naming convention that includes the base type in the element name in almost all cases. In the very few current cases where an element name doesn't indicate what substitution group it is in, I also don't mind consulting the schema to determine this - that's one of the reasons why we have a schema.

I disagree. Substitution groups follow the good coding principle of 'Open for extension, but closed for modification'. In addition, substitution groups realize the conceptual spec more closely which explicitly states that "Grid services requiring a richer set of attributes for the [Endpoint|Service], specific models MAY be derived by specializing from the [Endpoint|Service] class and adding new properties or relationships." Without substitution groups, the XSD doesn't support this.

...

Another thought on the grouping elements is which ones to use? If we use them, I don't necessarily agree that using the substitution groups or base complexTypes is the best approach. It may be logical for us as the schema creators, but it may not be logical to users. For example, as a user, in a single document, I'd like to have compute-related information in one area and storage-related information in another for easier viewing. A different way to put it is that just because two objects are in the same substitution group, that doesn't mean that that is the strongest relationship and they should be grouped together.

The above argument can also be applied to the ordering of the elements in the schema - I just noticed that this schema orders elements by substitution group. In the TeraGrid GLUE 2 XML schema, I ordered them in the order they are presented in GFD.147: base/abstract first, then compute, then storage. I think I personally prefer this ordering (because it matches GFD.147 and because it fits the way I look at things), but it doesn't seem critical.

I disagree. For me, the benefits provided using substitution groups far outweigh the disadvantages of less strict ordering when ordering by substitution group.

...

Heh, I honestly find the BaseType attribute more objectionable than the grouping entities. It just seems redundant, like something that would rarely, if ever, be used, and it just looks odd when viewing the XML document.

Regardless of BaseType attribute or grouping entity, I still think it is useful to explicitly indicate what is the base type of an element within the XML instance doc without having to explicitly refer to the XSD; both approaches enable simple XPath style queries on a doc e.g. 'select all activities' or 'select all endpoints'. I think this is valuable.

...

Warren

-----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Monday, September 17, 2012 3:43 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue- wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

hi Warren, all, (in response to Warren's response, cc'd below):

I think that having the ...s grouping elements does not add any complexity and at the same time provides a slightly more structured document; it provides a fixed location/bag for those elements that implement a particular glue2 xsd:substitutionGroup (this is why the grouping elements are not defined universally for all the elements in <Entities> - these other elements are concrete and cannot be substituted for different element implementations).

Now, consider new elements that may come along in future glue profiles that also define one of the glue2 xsd:substitutionGroups; looking at an instance document, it is not obvious that a new element (e.g. new activty, new service, new endpoint etc) is a substitute for a particular substitutionGroup without consulting the schema. I think this is also the original reason for the BaseType="..." attribute (which I carried over from the original nested schema); it also serves to clearly indicate the elements base type without having to consult the schema or make any inferences from the elements name (a per your modified XPath below). It is also easy to build XPath to say "select all entities that have BaseType=X" (which is similar to the xpath for selecting all elements of a particular type, e.g. select all Activities '/Entities/Activities/*').

Now, I do admit that there is a simple elegance in having all the glue entities as equal siblings within <Entities> (no ...s grouping elements), and there may be a good technical reason to keep all entities as siblings with the same level of nesting....

Therefore, at minimum I think it necessary to keep the 'BaseType=...' attributes, but its not really a big deal (at least to me) to keep or drop the '...s' grouping elements, even though I do find it v.useful to be able to collapse/expand the ...s grouping elements when browsing through an instance doc (if all the elements are equal siblings, i can't do this).

I'll be giving a talk at the EGI TF on GLUE2 XML renderings and can provide some examples, hopefully we will gain some more insights.

cheers, david

-------------------------------------------------------------- David Meredith STFC eScience Centre Daresbury Laboratory (A32) Warrington, Cheshire WA4 4AD

email: david.meredith@stfc.ac.uk tel: +44 (0)1925 603762

________________________________________ From: Warren Smith [wsmith@tacc.utexas.edu] Sent: 06 September 2012 23:50 To: Meredith, David (STFC,DL,SC); Burke, Stephen (STFC,RAL,PPD); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

Ok, I see what you are saying - I'm just now looking at the schema and I had some assumptions about it that weren't correct. However, after a little reading about XPath, it looks like there are a few ways that you can return all activities without requiring an Activities element as a parent.

/Entities/*[contains(local-name(),'Activity')] - returns all elements under Entities that have 'Activity' in them

/Entities/*[substring(name(), string-length(name()) - 7) = 'Activity'] - returns all elements under Entities that end in 'Activity'

These XPath expressions aren't as nice as '/Entities/Activities/*', but they aren't that bad and I don't know how often this case will come up. In XPath 2.0, it looks like there is an 'ends-with()' function that allows a much simpler version of the 2nd expression above. I personally wouldn't use the additional complexity of these XPath statements as a reason to add *s parent elements.

Sorry that I had to miss the last telecon since I was on vacation, so I apologize if some of my comments about the schema were discussed:

* My preference is to do away with all ...s grouping elements. I still don't see them as being needed (or desired).

* If the decision is to keep the ...s grouping elements (I hope not! :-P ), the schema should probably use them consistently (e.g. wherever the element has maxOccurs="unbounded"). They aren't being used right now in quite a few spots: Location, Contact, the elements at the end of ExtensibleEntities_t, and number of other complexType definitions.

* Why is there a BaseType="..." attribute on many of the elements? It seems like redundant information and looking at the schema, I can't figure out why they are needed...

* The Extension representation is slightly different from how I defined them in the TeraGrid schema. This schema has elements for LocalID and Key (as well as Value). For the TeraGrid schema, I had an Extension element with Key as an attribute (I forgot to include a LocalID attribute), and the text of the Extension element was the value. This schema mostly avoids attributes, so using elements probably makes more sense...

That's all I notice at this point.

Warren

-----Original Message----- From: david.meredith@stfc.ac.uk [mailto:david.meredith@stfc.ac.uk] Sent: Friday, August 31, 2012 10:00 AM To: Warren Smith; stephen.burke@stfc.ac.uk; florido.paganelli@hep.lu.se; glue- wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

Hi Warren, In the glue2_2.xsd at http://redmine.ogf.org/dmsf/glue-wg?folder_id=32, there is no '<ComputingActivites>,' only '<Activities>' which serves to group the elements that are substitutable for the 'AbstractActivity' substitutionGroup (so currently this includes both 'Activity' and 'ComputingActivity' - and of course any activities that may be profiled in the future that may also define the same substitutionGroup). The relevant XSD fragment is as follows (line 139) - note that it simply serves as a wrapper around AbstractActivity:

<element name="Activities" minOccurs="0" maxOccurs="1"> <complexType> <sequence> <element ref="glue:AbstractActivity" minOccurs="0" maxOccurs="unbounded"/> </sequence> </complexType> </element>

Therefore, given the following instance doc:

<Entities... > ... <glue:Activities> <glue:Activity BaseType="Activity"> <glue:ID>http://activity1.ac.uk/1</glue:ID> <glue:Associations/> </glue:Activity>

<glue:ComputingActivity BaseType="Activity"> <glue:ID>computingActivity1</glue:ID> <glue:State>some state</glue:State> <glue:Owner>someone</glue:Owner> <glue:Associations/> </glue:ComputingActivity>

...here nest more elements that can substitute for the AbstractActivity substitutionGroup... </glue:Activities> ... </Entities>

To select all of the different flavours of Activity in a single query, you could use the following XPath: '/Entities/Activities/*'. Similarly, for all services; '/Entities/Services/*' and for all Endpoints; '/Entities/Endpoints/*' etc. I guess it depends if you think this is useful. Without the 'Activities' element, to select all the different types of activity, you would currently need to issue two XPath queries and concat the two docs: '/Entities/Activity' and '/Entities/ComputingActivity'.

Personally, I think it is useful to be able to select/collapse all of the Services, Endpoints, Activities etc with one button click or query, especially when considering large documents that may have 1000s of endpoints listed. Either way, it's only a small change from the version that does not define these grouping elements (http://redmine.ogf.org/dmsf/glue-wg?folder_id=31)

David

...

-----Original Message----- From: Warren Smith [mailto:wsmith@tacc.utexas.edu] Sent: 31 August 2012 14:48 To: Burke, Stephen (STFC,RAL,PPD); Meredith, David (STFC,DL,SC); Meredith, David (STFC,DL,SC); florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: RE: [glue-wg] flat xml rendering

My general opinion is not to have any -ies elements (e.g. ComputingActivities) to group elements. To me, this grouping of elements (e.g. ComputingActivity) isn't needed in XML. It doesn't seem to make it any easier to process the XML using SAX or DOM. It seems easy to find all -y elements in a document without a -ies element as a parent for visualizing. Similarly, it also seems easy to find -y elements in an XML document if you want to translate it to LDAP or SQL or whatever.

I haven't really done much xPath... Can someone explain how grouping elements would help? It seems like either way you'd include 'ComputingActivity' in the expression. That is the expression would start with "/Entities/ComputingActivities/ComputingActivity..." or "/Entities/ComputingActivity...".

Warren

-----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of stephen.burke@stfc.ac.uk Sent: Friday, August 31, 2012 5:20 AM To: david.meredith@stfc.ac.uk; david.meredith@stfc.ac.uk; florido.paganelli@hep.lu.se; glue-wg@ogf.org Subject: Re: [glue-wg] flat xml rendering

glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On

...

Behalf Of david.meredith@stfc.ac.uk said: I think these group elements could be useful for simplifying XPath queries and for visualizing large documents with tools that can collapse/expand elements (see attached image, also uploaded at the following link).

Bear in mind that in the production Grid there are about 4000 endpoints, so grouping may not help a lot with visualisation unless you do it more finely.

Stephen

_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg -- Scanned by iCritical. -- Scanned by iCritical.

-- Florido Paganelli Lund University - Particle Physics ARC Middleware EMI Project