- glue-wg - lists.ogf.org

Model Questions
by Donal K. Fellows 14 Dec '07

14 Dec '07

Hi everyone I've been looking through the current draft and I've got a few questions... :-) 1) Should the Policy class have Scheme and Rule attributes? They seem to be common across a number of subclasses. 2) What's the relationship between Mapping Policy and Share Policy? 3) Should the Downtime class relate to a Service instead of an Endpoint? 4) Are Shares inherent features of all Services in the GLUE model, or can a particular subclass of Service not use them? (Might they only be features of Computing Services? Might they only be features of particular important subclasses of Computing Services, i.e. those with queues?) 5) Why is Share State separate from Share? Does it relate to Service state? 6) In the Information Model level, I'd be strongly tempted to just say that integers are integers and not give the width of the integers. (That's more appropriate for the data model.) 7) There are unit mistakes in Computing Share Policy. (I also strongly recommend only using SI base units (plus bytes) for units.) 8) What is the relationship of the Execution and Application Environment classes to the core GLUE model? 9) I've had systems in the past where the job acquired a Usage Record as a property when it completed. That worked quite well. 10) You probably should talk to the OGSA-DMI-WG folks about the Storage Endpoint class's attributes. 11) There needs to be a relationship between Compute and Storage sides (essential questions are probably "where can I get the output of the Job from?" "where do I ship my input data to in order to run on that system?" and "where can I run my job so that I don't need to ship the data about unnecessarily?"). That's a very important relationship! :-) 12) Do you plan to construct a dictionary of Application names? If not, it's going to make finding Application Environment instances tricky in practice. This will also need to tie in with JSDL, HPC Profile and ACS groups. Maybe also OGSA Naming, though I believe they name something else. If you do, it probably should be in a separate document. (I'd suggest some sort of structured name so that you don't need to name everything!) It's also a shame that CIM hasn't done anything in this area that I can see... Donal.

2 2

GridAustralia (previously know as the APACGrid) requirements for GLUE 2.0
by Gerson Galang 14 Dec '07

14 Dec '07

Hi Sergio, I've been pulled away to work on another project so I haven't been able to actively look at the specifications you guys have been writing. Anyway, I've got some free time now so I was able to look at what you guys have done with the schema. So my first question is, how you guys are planning to address the requirements (http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/GridAustrali… ) we've given you in May 2007? I've looked at our requirements and only four of them (stories 1, 2, 3, and 6 in the link above) have been addressed by GLUE 2.0 spec (draft). All of these four requirements have already been addressed by GLUE 1.3. So from what I see, there hasn't really been anything new added to GLUE 2.0 in terms of the requirements we've asked you to add in the new schema. Isn't the GLUE 2.0 spec already going for public comments after the next OGF? How are you planning to satisfy our requirements? GridAustralia will be very happy to contribute some of its resources just so the current draft can modified to satisfy our schema. I don't think our proposed schema is only GridAustralia specific so what we add into the specification shouldn't be an issue with anyone on this list. I also just want to comment on story 6 of our requirements. We know that you've satisfied that requirement by adding the ModuleName attribute to the ApplicationEnvironment entity but some Grids (e.g. TeraGrid) don't use modules and JP Navarro has suggested a more generic approach (and a better way of representing this type of information) and that's by using HandleType and HandleKey attributes. See below.. <ApplicationEnvironment> <ID>123</ID> <Name>apache-ant</Name> <Version>1.6.5</Version> <HandleType>softenv</HandleType>  <HandleKey>+apache-ant-1.6.5</HandleKey>  </ApplicationEnvironment> We're happy that you've got ModuleName in there but since this is a schema that will be used by a number of grid middleware/apps around the world, we just want to do it properly and avoid tying this version of the spec to a particular environment loading package implementation like "modules". It may be that you might not want to implement stories 9, 10, and 11 because you might think that it's just too hard to implement or you just think that there won't be any users/applications going to query for that information. If so, it would be good to get some feedback on the issues with our requirements. Story 13 is not related to software application so we want to know how you are planning to address that in GLUE 2.0? We also would like to hear your comments about the things we've listed under the "Glue 1.3 issues/comments" section of the GridAustraliaRequirements. I know those are not only applicable to GLUE 1.3 because I don't see the current draft of GLUE 2.0 addressing those issues yet. Look forward to hearing from you. Regards, Gerson On Dec 2, 2007 1:22 AM, Sergio Andreozzi <sergio.andreozzi(a)cnaf.infn.it> wrote: > Dear all, > > at the following page you can find the minutes of yesterday call: > http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting… > > > the new GLUE Spec draft (no.18) is available here: > http://forge.ogf.org/sf/docman/do/viewDocument/projects.glue-wg/docman.root… > > > the new XML Schema rendering, XML example and updated mapping rules can > be reached from this page: > > http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/GLUE2XMLSche… > > during the next week I will be in USA (West Coast) because I'm attending > a conference ( http://www.mandevcon.com/) where I will be presenting the > work that we (INFN) are doing in the framework of OMII-Europe to > implement GLUE 2.0 based on WBEM technologies. > > For next calls, I talked to Balazs and seems that the first intersection > is in the week 17-21/12. > I would propose 18 and 20 Dec as the last calls for this year: > http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeetings > > > Please let me know if you can make it. > > Meanwhile, we can make progress on the mailing lists. I will send > specific e-mails to ask for feedback contributions on open tasks: > > http://forge.ogf.org/sf/tracker/do/listArtifacts/projects.glue-wg/tracker.g… > > > Cheers, Sergio > > -- > Sergio Andreozzi > INFN-CNAF, Tel: +39 051 609 2860 > Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 > 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi<http://www.cnaf.infn.it/%7Eandreozzi> > > _______________________________________________ > glue-wg mailing list > glue-wg(a)ogf.org > http://www.ogf.org/mailman/listinfo/glue-wg >

2 1

Concrete example for alternative primary source [option P.O]
by Paul Millar 11 Dec '07

11 Dec '07

Hi, Here's a concrete example, a primary source (a Service) publishes XML with a top-level element of <Service> Variant #1 (with attributes) <Service id="Service-URI" parent="AdminDomain-URI" name="Fred">  </Service> Variant #2 (with no attributes) <Service> <ID>Service-URI</ID> <Parent>AdminDomain-URI</Parent> <Name>Fred</Name>  </Service> When aggregation happens, the services are collected under their "Parent" (some AdminDomain). This allows the site information to using the simple One-to-Many mapping to parent/child elements concept. Here is a worked example. Consider three services: a CE & SE service (under AdminDomain-1) and an FTS service (under AdminDomain-2). File: service-1.xml <?xml version="1.0"?> <Service> <ID>http://example.org/Service-1</ID> <Parent>http://example.org/AdminDomain-1</Parent> <Name>MyCE</Name> </Service> File: service-2.xml <?xml version="1.0"?> <Service> <ID>http://example.org/Service-2</ID> <Parent>http://example.org/AdminDomain-1</Parent> <Name>MySE</Name> </Service> File: service-3.xml <?xml version="1.0"?> <Service> <ID>http://example.org/Service-3</ID> <Parent>http://example.org/AdminDomain-2</Parent> <Name>FTS</Name> </Service> Here the site is defined through the site.xml file. This toy example just listing the service URIs and the list of AdminDomain we wish to publish. The AdminDomains here are void of any additional information. That information could be included in site.xml, or imported from elsewhere. To keep things simple, Service elements href attributes use relative URIs to access each service's GLUE/XML info; but the URIs could be absolute and the information collected directly (e.g. via HTTP). File: site.xml <?xml version="1.0"?> <Site> <ServiceList> <Service href="service1.xml"/> <Service href="service2.xml"/> <Service href="service3.xml"/>  </ServiceList> <AdminDomainList> <AdminDomain id="http://example.org/AdminDomain-1"/> <AdminDomain id="http://example.org/AdminDomain-2"/> <AdminDomain id="http://example.org/AdminDomain-3"/> </AdminDomainList> </Site> Then, just transform this into the GLUE/XML for aggregation: File: process.xsl <?xml version="1.0"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:template match="/"> <Grid> <Site> <xsl:apply-templates/> </Site> </Grid> </xsl:template> <xsl:template match="AdminDomain"> <AdminDomain> <ID><xsl:value-of select="@id"/></ID> <xsl:apply-templates select="/Site/ServiceList/Service" mode="import-admin-service"> <xsl:with-param name="adminservice-uri" select="@id"/> </xsl:apply-templates> </AdminDomain> </xsl:template> <xsl:template match="ServiceList/Service[@href]" mode="import-admin-service"> <xsl:param name="adminservice-uri"/> <xsl:copy-of select="document(@href)/Service[Parent=$adminservice-uri]"/> </xsl:template> </xsl:stylesheet> Transform the site.xml into GLUE/XML aggregated format using XSLT: xsltproc process.xsl site.xml > /tmp/out.xml && xmllint -format /tmp/out.xml Gives the following output: <?xml version="1.0"?> <Grid> <Site> <AdminDomain> <ID>http://example.org/AdminDomain-1</ID> <Service> <ID>http://example.org/Service-1</ID> <Parent>http://example.org/AdminDomain-1</Parent> <Name>MyCE</Name> </Service> <Service> <ID>http://example.org/Service-2</ID> <Parent>http://example.org/AdminDomain-1</Parent> <Name>MySE</Name> </Service> </AdminDomain> <AdminDomain> <ID>http://example.org/AdminDomain-2</ID> <Service> <ID>http://example.org/Service-3</ID> <Parent>http://example.org/AdminDomain-2</Parent> <Name>FTS</Name> </Service> </AdminDomain> <AdminDomain> <ID>http://example.org/AdminDomain-3</ID> </AdminDomain> </Site> </Grid> [For aesthetics, one could filter out the Parent children from Service elements.] Is this about right? Cheers, Paul.

1 0

Values with additional sematic meaning.
by Paul Millar 06 Dec '07

06 Dec '07

Hi all, One of the complaints I've heard from people is that GLUE schema doesn't have a mechanism for knowing that certain results are invalid. In my "spare time", I put together some material that might be of use here. Of course, this is just a first draft; comments gratefully appreciated. Cheers, Paul. PS. my thanks to Graeme Steward for his comments and suggestions. ------------- GLUE proposal on values with additional semantic meaning. ---- Introduction --- GLUE Schema provides a common method of providing information about a Grid such that specific knowledge of that Grid is not needed to appreciate its current state. The current GLUE schema assumes that all information is available and that any static information has been correctly configured. This is not always true: for whatever reason, information may be temporary or more permanently unavailable or some static information may be left unconfigured. These imply that some additional (semantic) meaning of values is needed. This is most naturally accommodated by returning a special (or "illegal") value: a value that should not occur during normal operation. This is already in practise; however it is not specified within GLUE, so different implementations can choose different values. Knowledge of whether a particular value is valid then becomes information-provider specific, going against the spirit of the GLUE Schema. This document aims to rationalises and standardises values that have some additional semantic meaning. It is split into two further sections: use-cases and proposed values. Use-cases: --- Two scenarios are considered: Scenario 1. a static value has no (good) default value and has not been configured. Some static values have no good default value. This may be because the configuration requires some knowledge of a site's configuration or for some other reason. Whilst this value should be altered to reflect the site's configuration this might not happen[1], so exposing the application's default configuration. Given there is no (good) default value, what should this value be? [1] We note the available of tools such as YAIM to assist in configuration of EGEE sites, which will reduce the likelihood of this scenario. However, 1. not all sites are configured with YAIM, 2. not all Grids have a tool like YAIM, 3. Upgrading components may result in changes in configuration, so increasing the likelihood of this problem. Scenario 2. information provider is unable to obtain a dynamic value. A dynamic value is provided by an information provider by querying the Grid resource. This query will use a number of ancillary resources (e.g., DNS, network hardware) that might fail; the service itself might fail. Given a lack of information[2], what value should the information provider return? [2] If caching of previous results is available, temporary failures may be mitigated. However, it is an open question for how long any such cached information should be permitted. Proposed illegal values: --- This section describes a number of values that can be represented within a given address space (e.g., UTF-8, Integers, FQDNs, IPv4 address space). With Scenario 1, configuration SHOULD use these default values wherever possible; likewise, dynamic information provides SHOULD use these default values whenever they wish to indicate a problem gathering information. The semantic meaning SHOULD BE limited to simply that the value is invalid and is not to be relied upon. A client using this information SHOULD NOT draw any conclusions as to why the information is invalid from the information presented here. 1. All GLUE-specific enumerated types: SHOULD use the designated "unknown" value. Rational: If a value is unknown, it should be specified as such. Either this or the GLUE schema clearly state that, should some values be unknown that the whole entry should not be reported. 2. Simple strings (UTF-8): SHOULD use "ILLEGALVALUE". Rational: Upper-case letters make it easier to spot and a single word avoids white-space issues. This is a default option for strings that have no widely-known structure. If a value is from a more restrictive sub-type (e.g. FQDNs), then the rules for more restrictive form SHOULD be used. 3. Fully qualified domain names: SHOULD use a hostname ending "invalid." for scenario 2. and either "example.org." or "example.com." for scenario 1. Rational: RFC 2606 reserves the ".invalid" TLD as always invalid and clearly so. For dynamic information a value of "unknown.invalid." MAY be used. If an alternative is used, it SHOULD use the TLD ".invalid". RFC 2606 also defines the ".example" TLD and two second-level domains: "example.org" and "example.com". These domains have the advantage of ending with a recognisable TLD, so looking like a DNS name. Default configuration SHOULD use DNS names that end ".example.com." or ".example.org." The final dot at the end of the DNS name SHOULD be included as it prevents local DNS expansion. 4. IPv4 addr: SHOULD use 192.0.2.250 Rational: There are several portions of IPv4 addresses that should not appear on a network, but none that are documented as being illegal. Using an arbitrary address leads to the risk of side-effects. The best option is an IP address from the 192.0.2.0/24 subnet. This subnet is defined in RFC 3330 as "TEST-NET" for use in documentation and example code. Although any IPv4 address from 192.0.2.0/24 MAY be used, the above address SHOULD use for consistency. 5. IPv6 addr: SHOULD use 2001:DB8::FFFF Rational: There is no documented illegal IPv6 address. RFC 3849 reserves the address prefix 2001:DB8::/32 for documentation. For consistency, the address SHOULD BE the one noted above. 6. Counting/Natural numbers: SHOULD use 0 Rational: Counting- (also known as Natural-) numbers exclude the number zero, so information provider SHOULD use this value to indicate an illegal value. 7. Integers: SHOULD use MAXINT (maximum value representable in the domain). For int32 this is 2,147,483,647 " uint32 this is 4,294,967,295 " int64 this is 9,223,372,036,854,775,807 " uint64 this is 18,446,744,073,709,551,615 Rational: For non-negative integers, all numbers expressible within the encoding (int32/uint32/etc...) are valid so there is no safe choice. Although any value may be chosen, the value least likely to be encountered in any given domain is that domain's maximum value. If an unsigned integer is encoded as a signed integer, it is possible to use negative numbers safely. However, these numbers will be unrepresentable if the number is stored as an unsigned integer. For values that might be a positive, zero or a negative integer, all numbers expressible within the encoding (unsigned int) are valid so there is no safe choice. For consistency, the MAXINT value SHOULD be used. Information providers SHOULD NOT attempt to conveying further semantic distinction by using more than one illegal number. 8. Filenames: SHOULD use "ILLEGALPATH". Rational: As with the simple string, a single upper-case word is recommended. 9. Uniform Resource Identifier (URI): schema-specific Rational: RFC 3986 defines URIs as a "federated and extensible naming system." All URIs start with a schema-name part and no schema-name has been reserved for illegal or example values. For any given URI schema, it may be possible to define an illegal value within that name-space. If a value has only one valid schema, the illegal value should be taken from that schema. If several schemata are possible, one should be chosen. For schemata that include a reference to an Internet host (e.g. http, httpg, mailto), an illegal value SHOULD be derived by using an illegal FQDN (see above). For other schemata, some element should indicate that the value is illegal. This is subject to further work. 10. X509 Distinguished Names: SHOULD include a RDN of CN=ILLEGALUSER Rational: X509 uses a X500 namespace, represented as several RDNs concatinated by commas. The final RDN is usually a single common name (CN). It is possible for more than one CN to be present, allowing inclusion of additional sematic meaning. This is outwith the scope of the document. Definition of words: --- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are meant as described in RFC 2119. A brief summary is given here. 1. MUST (or "REQUIRED" or "SHALL") means that no deviation is allowed from conforming software. 2. MUST NOT (or "SHALL NOT") means complete prohibition of this behaviour with conforming software. 3. SHOULD (or "RECOMMENDED") means that there may be reasons why conforming software does not to adopt this behaviour, but all the effects of an alternative behaviour must be understood and considered before choosing a different course. 4. SHOULD NOT (or "NOT RECOMMENDED") means that there may be reasons why conforming software adopts this behaviour, but all the effects of an alternative behaviour must be understood and considered before choosing a different course. 5. MAY (or "OPTIONAL") means an item is completely optional.

2 1

Re: [glue-wg] [ogsa-wg] Nov 29 - Info Model call minutes
by Sergio Andreozzi 03 Dec '07

03 Dec '07

Donal K. Fellows ha scritto: > Sergio Andreozzi wrote: >> Ellen Stokes ha scritto: >>> One observation on your modified diagram....what do you mean by >>> 'Resource Model'? >>> >>> In OGSA, we use the terms information model (abstract) and data >>> model (concrete)....we do not use or define resource model since it >>> is an ambiguous term. >> >> yes, Information Model is a better term in that case. GLUE is a >> possible instance. I attach the modified version. > > I think you need to be careful with GLUE since that's very close to > defining both an information model and a data model, making it easy to > get confused. For example of how they differ, an information model might > say that a particular class supports an extensible list of properties > (other than the standard set it defines) and the data model would say > what format that list of properties must take (e.g. XML document, Java > properties format, comma-separated list, etc.) It's the information > model that it is important to standardize at the architectural level, > since it is fairly easy to code up data model transformations (though > those matter much more for specific protocols). GLUE is an information model of Grid resources. This is the main goal and nature. The aspect of extensibility is secondary to this and is present because is a requirement from many parties. The fact that the abstract GLUE Information Model (represented using the object-oriented constructs of UML Class diagrams) is rendered on different concrete data models (XML Schema, relational and LDAP) makes it difficult to delegate at the concrete data model level the extensibility aspects. "Hooks" are currently present at the information model level and can be used to advertise information that is not currently captured by the specification without waiting for a new spec version. The GLUE Spec is deliberately not rigorous in defining the format of the content of the extensibility points. Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

2 1

30 Nov telecon minutes + GLUE Spec draft no.18 + new XSD mapping
by Sergio Andreozzi 02 Dec '07

02 Dec '07

Dear all, at the following page you can find the minutes of yesterday call: http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting… the new GLUE Spec draft (no.18) is available here: http://forge.ogf.org/sf/docman/do/viewDocument/projects.glue-wg/docman.root… the new XML Schema rendering, XML example and updated mapping rules can be reached from this page: http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/GLUE2XMLSche… during the next week I will be in USA (West Coast) because I'm attending a conference (http://www.mandevcon.com/) where I will be presenting the work that we (INFN) are doing in the framework of OMII-Europe to implement GLUE 2.0 based on WBEM technologies. For next calls, I talked to Balazs and seems that the first intersection is in the week 17-21/12. I would propose 18 and 20 Dec as the last calls for this year: http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeetings Please let me know if you can make it. Meanwhile, we can make progress on the mailing lists. I will send specific e-mails to ask for feedback contributions on open tasks: http://forge.ogf.org/sf/tracker/do/listArtifacts/projects.glue-wg/tracker.g… Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

2 1

[artf6088, artf6089] XML Schema rendering of GLUE 2.0
by Sergio Andreozzi 01 Dec '07

01 Dec '07

Dear all, in order to progress on the XML Schema rendering of GLUE 2.0, we need that each middleware provider interested in its adoption investigates with their own developers the following aspects: http://forge.ogf.org/sf/go/artf6088 http://forge.ogf.org/sf/go/artf6089 please, provide a feedback on your preference during next week. Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

1 0

GLUE Telecon now - number changed to INFN one
by Sergio Andreozzi 30 Nov '07

30 Nov '07

Dear all, we have a GLUE telecon starting now. Please use the new number advertised here: http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting… Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

1 0

Today telecon (29-11, 3 PM UK) moved to tomorrow (30-11, 2 PM UK)
by Sergio Andreozzi 29 Nov '07

29 Nov '07

Dear all, this is to confirm that the today GLUE telecon is moved to tomorrow at 2 PM UK Time: http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting… Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

2 1

tomorrow telecon
by Sergio Andreozzi 28 Nov '07

28 Nov '07

Dear all, tomorrow we have a scheduled telecon at 3 PM UK time: http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting… unfortunately I won't be able to attend. The options are: 1. you run as well to conference (Balazs or Laurence can lead it) 2. we postpone to Friday, 2 PM UK time would be ok for me Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

1 0