- glue-wg - lists.ogf.org

REMINDER: GLUE2 XML Rendering Session today at 9 am in Room Snowbird
by Morris Riedel 17 Jul '11

17 Jul '11

All, just a gentle reminder that we have our GLUE2 session today [1]: GLUE2 - XML Rendering Specification Progress and Plans (90 mins) Sunday, July 17 9:00 am - 10:30 am Room Snowbird Firstly, we will have a short update on the XML rendering status and gather feedback from the audience. Afterwards we discuss relationships to existing stakeholders (i.e. PGI, EMI) and identify other possible stakeholders. Finally, we briefly discuss how this fits in to our plan of the BES/JSDL/GLUE2/PGI activities from yesterday. Take care, Morris [1] GLUE2 Session http://www.gridforum.org/gf/event_schedule/index.php?id=2334 ------------------------------------------------------------ Morris Riedel Deputy Division Leader Federated Systems and Data (FSD) Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/SharedDocs/Personen/IAS/JSC/EN/staff/riedel_m.html

1 0

GLUE 2.0 XSD Rendering - progress update
by Sergio Andreozzi 02 May '11

02 May '11

Dear all, I'm happy to announce that a concrete progress was made in aligning the XSD document to GLUE 2.0 final. This is a very delicate work which needs a focused activity and, being delayed for too long, we decided to run a face-to-face meeting at the end of March to make progress. Adrian Taga, Balazs Konya, Shiraz Memon and I met in Lund (Sweden) and worked two full days with the participation of Paul Millar from remote. Thanks to this effort, we managed to update the GLUE XSD and the related descriptive document which now need the final make-up before submitting them for the last call and subsequent public comment to the GLUE WG list. In parallel, Ilya Saverchenko and his colleagues from LRZ are pushing forward the SQL rendering and document. The LDAP rendering is in production since few months already and the specification needs to be read for consistency (action being carried out by Adrian Taga). We are therefore in a good shape to move all the three rendering to last call in the coming few weeks (hopefully before the end of April) and start the public comment phase. As a side news, I'd like to announce that: - the new official source code repository is now hosted in github.com. I have created three repositories (one per rendering) which you can access following this link: https://github.com/OGF-GLUE/ - the descriptive documents accompanying the renderings are available here (with the new draft for the XSD): http://forge.ogf.org/sf/go/projects.glue-wg/docman.root.drafts - the wiki pages have been polished and updated: http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/HomePage Looking forward to writing soon about when all the three specs and documents are ready for last call. Meanwhile, if you are building on top of GLUE XSD, you are welcome to download the new version for early feedback. Kind regards, Sergio -- Sergio Andreozzi - Policy Development Manager EGI.eu - Amsterdam - The Netherlands - http://egi.eu

2 1

Problem with LDAP binding inherited foreign-keys
by Paul Millar 23 Feb '11

23 Feb '11

Hi all, While writing the dCache GLUE2 info-provider, I came across a problem with the current GLUE2 LDAP binding. The problem is that inherited associations are represented twice. The schema uses three kinds of LDAP class: ABSTRACT, AUXILIARY and STRUCTURAL. ABSTRACT is only used for the LDAP class for Entity (GLUE2Entity), all objects that are a directly derived from Entity are STRUCTURAL, everything else is AUXILIARY. So: LDAP classes for Service and Share (GLUE2Service and GLUE2Share respectively) are STRUCTURAL, LDAP classes for StorageService and StorageShare (GLUE2StorageService and GLUE2StorageShare) are AUXILIARY. Any published object must have at least one STRUCTURAL class and zero or more AUXILIARY classes. To publish a StorageService, the LDAP object must have GLUE2StorageService and GLUE2Service object-classes. To publish a StorageShare, the LDAP object must have GLUE2StorageShare and GLUE2Share object-classes. In GLUE v2.0, the Share object has a required (i.e., multiplicity of '1') association with a Service. This is represented in the LDAP Schema as a required attribute in the GLUE2Share class: the GLUE2ShareServiceForeignKey attribute. The StorageShare object inherits this required association, only it is against a StorageService now. This association is represented as a required attribute in the GLUE2StorageShare class: the GLUE2StorageShareStorageServiceForeignKey attribute. Since a StorageShare object must be published as both GLUE2StorageShare and GLUE2Share object-classes, this object must publish the same association twice: once once as GLUE2ShareServiceForeignKey and again as GLUE2StorageShareStorageServiceForeignKey. The same problem exists for very many (perhaps all) ForeignKey attributes in AUXILIARY classes but is hidden since those foreign-key attributes are optional. I've talked to Laurence about this problem and our preferred solution is to delete GLUE2StorageShareStorageServiceForeignKey and have all a StorageShare publish their association with StorageService using a GLUE2ShareServiceForeignKey. I believe this approach will work for the other "inherited" foreign-key attributes. This solution has the disadvantage that the association between a StorageService cannot be restricted to a StorageService (and not, for example, to a ComputingService object). However, since LDAP doesn't support foreign- key constraints, such consistency may only be enforced by manual checking (gstat2, glue-validation suite, etc). So, I believe removing GLUE2StorageShareStorageServiceForeignKey doesn't loose us anything. Does this make sense? Are people in favour of this approach? Cheers, Paul.

3 5

Erratum
by stephen.burke＠stfc.ac.uk 21 Feb '11

21 Feb '11

Hi, I just added this to the GLUE 2 errata page: "The TrustedCA attribute in the Endpoint (GLUE Specification v. 2.0, page 15) includes the text "Alternatively this may identify a standard bundle of accepted CAs, e.g. those accredited by the IGTF. Note that this does not imply that such certificates will be authorized to use the Endpoint.". This text is omitted in the description of the corresponding inherited attribute in ComputingEndpoint (page 25) and StorageEndpoint (page 48)." http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/GLUE20Errata Thanks to Adrian Taga for pointing it out. Stephen -- Scanned by iCritical.

1 0

Glue Service Types
by Laurence Field 11 Feb '11

11 Feb '11

Dear All, I have migrated the old service type registry http://glueschema.forge.cnaf.infn.it/V12/ServiceType to Grid Forge. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/Servic… Stephen Burke will maintin his role as defacto curator. Laurence

2 1

GLUE for describing resource requests (use case DGSI)
by Mikael Högqvist 12 Jan '11

12 Jan '11

Hi, I'm working in the DGSI project (a German D-Grid project) where our goal is to design and implement an interoperability framework between community schedulers and community bound resources. More specifically, this include protocols and services for delegation of activities as well as resources. We are primarily basing our protocols on OGF standards and recommendations and have tight collaboration with both the GRAAP (WS-Agreement/Negotiation) and DCI-Fed working groups within OGF. In the first information model we decided to use GLUE 2.0 for both resource descriptions and resource request specification. However describing resource requests with GLUE turned out to be difficult to implement. The main issue that we have is the restrictions in the GLUE 2.0 XML schema which makes it difficult to use only specific elements/group of elements in a document instance. For example, we would like to use the AccessPolicy element to describe requirements on who (e.g. DN/VO) is authorized to access a delegated resource. But to use this element, we need to create the entire hierarchy with Service/ComputingService/ComputingEndpoint and their respective required elements (e.g. ID, QualityLevel, ...) when using the GLUE 2.0 XSD. Currently, our solution is to use JSDL for specifying both activity and resource delegation requests. However, JSDL is not sufficient for all our use cases/requirements and we would like to use GLUE for specifying resource requests in our next version. Is there any work going on in the glue-wg to make the XSD rendering more lenient towards other use cases such as resource requirement specification? Or making it possible to use a subset of GLUE and combine it with other info models such as JSDL? Thank you in advance and Best Regards, Mikael Högqvist

3 3

Re: [glue-wg] GLUE for describing resource requests (use case DGSI)
by Sergio Andreozzi 19 Nov '10

19 Nov '10

Hi Alexander, yep, now I get it right. The change you require is in the pipeline. Other people already asked for the possibility to create individual fragments without need for the hierarchical dependencies. I will call for your and Mikael validation on the new version as soon as I have it. Cheers, Sergio On 18 November 2010 18:42, <alexander.papaspyrou(a)tu-dortmund.de> wrote: > Hi Sergio, > > probably, I wasn't too clear about what I wanted. > > Actually, we would like to see element-per-type approach that JSDL took. > There, if you need a specific part of the JSDL to be "torn out" and reused > somewhere else (let's say, in some other context where only the "Application > Part" is of interest), you can just do this, because every type (class in > Java-speak) in the XSD has a corresponding element (instance in Java-speak). > With GLUE which doesnt have this in the current state of the XML rendering), > you need to build a full GLUE document and fill several mandatory elements > with nonsense data, just because you need some low-level fragment to express > a certain issue. > > My only concern was that this fragment usage will be allowed in the > end-of-the-year version of the GLUE XSD, and I wanted to make sure it is :-) > > Best, > Alexander > > Am 18.11.2010 um 14:43 schrieb Sergio Andreozzi: > > > Hi Alexander, > > > > the final XSD for GLUE 2.0 is expected by the end of the year. > > > > Concerning the integration with JSDL for resource requirements matching, > this is still open to debate and falls in the area of PGI work. > > > > Nevertheless, the topic has been touched few times in the past. I can > provide my vision: > > > > the current JSDL envisions a number of terms to be used to express > requirements on resources. They are part of the JSDL specification therefore > binding the resource description to the job description. A couple of years > ago, in an OGF joint session (http://bit.ly/byqg5T) we discussed the idea > of separating them in order to easy the evolution of resource models > independently from JSDL. Requirements are usually expressed as constraints > on resource properties, therefore the idea could be to enable to express > them as XPath/XQuery expressions on the resource description. An example is > given in Section 5.3 of this OGF document: > http://ogf.org/documents/GFD.137.pdf > > > > Expressing requirements as constraints on a resource description is the > approach in place since years in the ClassAd language and its gLite JDL > derivation. XPath/XQuery can be a translation into an XML world. > > > > Cheers, Sergio > > > > > > On 18 November 2010 14:17, <alexander.papaspyrou(a)tu-dortmund.de> wrote: > > Sergio, > > > > I think this boils down to more or less the old "allow fragments of GLUE > to be used independently" debate we had in Brussels. Do we have a timeline > for a version bump of the XML/XSD rendering, and will it be possible to > render the XSD as it was done in JSDL (i.e. having an "xs:element" for each > "xs:*Type*")? > > > > Best, > > Alexander > > > > Am 18.11.2010 um 12:27 schrieb Mikael Högqvist: > > > > > Hi, > > > > > > I'm working in the DGSI project (a German D-Grid project) where our > > > goal is to design and implement an interoperability framework between > > > community schedulers and community bound resources. More specifically, > > > this include protocols and services for delegation of activities as > > > well as resources. We are primarily basing our protocols on OGF > > > standards and recommendations and have tight collaboration with both > > > the GRAAP (WS-Agreement/Negotiation) and DCI-Fed working groups within > > > OGF. > > > > > > In the first information model we decided to use GLUE 2.0 for both > > > resource descriptions and resource request specification. However > > > describing resource requests with GLUE turned out to be difficult to > > > implement. The main issue that we have is the restrictions in the GLUE > > > 2.0 XML schema which makes it difficult to use only specific > > > elements/group of elements in a document instance. > > > > > > For example, we would like to use the AccessPolicy element to describe > > > requirements on who (e.g. DN/VO) is authorized to access a delegated > > > resource. But to use this element, we need to create the entire > > > hierarchy with Service/ComputingService/ComputingEndpoint and their > > > respective required elements (e.g. ID, QualityLevel, ...) when using > > > the GLUE 2.0 XSD. > > > > > > Currently, our solution is to use JSDL for specifying both activity > > > and resource delegation requests. However, JSDL is not sufficient for > > > all our use cases/requirements and we would like to use GLUE for > > > specifying resource requests in our next version. > > > > > > Is there any work going on in the glue-wg to make the XSD rendering > > > more lenient towards other use cases such as resource requirement > > > specification? Or making it possible to use a subset of GLUE and > > > combine it with other info models such as JSDL? > > > > > > Thank you in advance and Best Regards, > > > Mikael Högqvist > > > > > > > > > > -- > > Sergio Andreozzi - Policy Development Manager > > EGI.eu - Amsterdam - The Netherlands - http://egi.eu > > -- Sergio Andreozzi - Policy Development Manager EGI.eu - Amsterdam - The Netherlands - http://egi.eu

1 0

Re: [glue-wg] GLUE for describing resource requests (use case DGSI)
by Sergio Andreozzi 18 Nov '10

18 Nov '10

Hi Alexander, the final XSD for GLUE 2.0 is expected by the end of the year. Concerning the integration with JSDL for resource requirements matching, this is still open to debate and falls in the area of PGI work. Nevertheless, the topic has been touched few times in the past. I can provide my vision: the current JSDL envisions a number of terms to be used to express requirements on resources. They are part of the JSDL specification therefore binding the resource description to the job description. A couple of years ago, in an OGF joint session (http://bit.ly/byqg5T) we discussed the idea of separating them in order to easy the evolution of resource models independently from JSDL. Requirements are usually expressed as constraints on resource properties, therefore the idea could be to enable to express them as XPath/XQuery expressions on the resource description. An example is given in Section 5.3 of this OGF document: http://ogf.org/documents/GFD.137.pdf Expressing requirements as constraints on a resource description is the approach in place since years in the ClassAd language and its gLite JDL derivation. XPath/XQuery can be a translation into an XML world. Cheers, Sergio On 18 November 2010 14:17, <alexander.papaspyrou(a)tu-dortmund.de> wrote: > Sergio, > > I think this boils down to more or less the old "allow fragments of GLUE to > be used independently" debate we had in Brussels. Do we have a timeline for > a version bump of the XML/XSD rendering, and will it be possible to render > the XSD as it was done in JSDL (i.e. having an "xs:element" for each > "xs:*Type*")? > > Best, > Alexander > > Am 18.11.2010 um 12:27 schrieb Mikael Högqvist: > > > Hi, > > > > I'm working in the DGSI project (a German D-Grid project) where our > > goal is to design and implement an interoperability framework between > > community schedulers and community bound resources. More specifically, > > this include protocols and services for delegation of activities as > > well as resources. We are primarily basing our protocols on OGF > > standards and recommendations and have tight collaboration with both > > the GRAAP (WS-Agreement/Negotiation) and DCI-Fed working groups within > > OGF. > > > > In the first information model we decided to use GLUE 2.0 for both > > resource descriptions and resource request specification. However > > describing resource requests with GLUE turned out to be difficult to > > implement. The main issue that we have is the restrictions in the GLUE > > 2.0 XML schema which makes it difficult to use only specific > > elements/group of elements in a document instance. > > > > For example, we would like to use the AccessPolicy element to describe > > requirements on who (e.g. DN/VO) is authorized to access a delegated > > resource. But to use this element, we need to create the entire > > hierarchy with Service/ComputingService/ComputingEndpoint and their > > respective required elements (e.g. ID, QualityLevel, ...) when using > > the GLUE 2.0 XSD. > > > > Currently, our solution is to use JSDL for specifying both activity > > and resource delegation requests. However, JSDL is not sufficient for > > all our use cases/requirements and we would like to use GLUE for > > specifying resource requests in our next version. > > > > Is there any work going on in the glue-wg to make the XSD rendering > > more lenient towards other use cases such as resource requirement > > specification? Or making it possible to use a subset of GLUE and > > combine it with other info models such as JSDL? > > > > Thank you in advance and Best Regards, > > Mikael Högqvist > > -- Sergio Andreozzi - Policy Development Manager EGI.eu - Amsterdam - The Netherlands - http://egi.eu

1 0

WLCG Information System - GLUE
by Etienne URBAH 11 Nov '10

11 Nov '10

Flavia, Information Services -------------------- I just finished reading your presentation 'WLCG Information Officer' at http://indico.cern.ch/getFile.py/access?contribId=2&sessionId=0&resId=0&mat… Inside the EGDI project http://edgi-project.eu/ integrating Service Grids (such as WLCG) and Desktop Grids (such as SZDG or IberCivis), I am working on standardization. In particular, I consider that an IS (Information System) permitting providers to publish information and clients to discover information is the mandatory foundation for any distributed data processing infrastructure (Grid). Moreover, standardized interface about data model and protocol for information providers and client queries is mandatory for interoperation between grids such as EGI, OSG, Desktop Grids, ... Therefore, I am member of the OGF GLUE Working Group http://forge.gridforum.org/sf/projects/glue-wg Other involved parties ---------------------- For interoperation, I suggest you to add following stakeholders to your list of 'Other involved parties' : - OGF GLUE, - the EC-funded DCI projects (EDGI, EGI-Inspire, EMI, IGE, StatusLab, Venus-C), - University of Chicago/ANL http://forge.ogf.org/sf/go/doc16100 - Naregi-Renkei, which is currently using CIM. WLCG Information Service ------------------------ The current WLCG Information Service is the BDII based on the GLUE 1.3 schema and bootstrapped with the GOCDB https://goc.gridops.org/ and the CIC https://cic.gridops.org/ Thank you very much for having gathered the opinion of the 4 LCH experiments. Although it is a pity that the 4 LCH experiments do not really trust the WLCG Information Service yet, it is very important that IS designers, developers and integrators know this distrust, in order to search for the real reasons, and to carefully fix all issues. Below are some issues which I know, and proposals for solutions : - Accuracy of published information (in particular storage capacity) : The solution is to take into account 'information provider code' as real software, which must be designed using the best practices of software engineering, must have an installation manual, and must be deployed on all sites using the best practices of grid operation. When an adequate 'information provider' exists and has an adequate installation manual, sites publishing inaccurate information should be blamed. - Relevance and lifetime of published information : Among the various items of information published inside the BDII, users need to know which items they can trust and for how long. This is clearly a requirement for a document (or wiki page) containing a table describing the most relevant items and their probable lifetime. Moreover, it is probably the role of the WLCG Information Officer (yourself) to write down and publish this table. - Rigid implementation of GLUE 1.3 (unless it has already been fixed) : As far as I know, inside a single 'Grid Site' modeled by 'GlueSite' and 'GlueCluster', it would be useful to model each set of machines having same physical characteristics as a separate 'GlueSubCluster', but this is difficult to achieve because installation software does NOT support this completely yet. The solution would be to improve the installation software. - Query language : I am an experienced programmer in various languages, but I assess that building LDAP queries to extract useful information is difficult. A (perhaps restricted) SQL interface would be much easier to use, and there are many software developers and even users who already have a little experience with SQL. - Bad terminology by GLUE 1.3 : A 'Job Queue' is modeled by a 'GlueCE', and a 'Grid CE' is modeled by a 'GlueSubCluster'. Which user would be happy with that ? The solution is GLUE 2.0, where a 'Job Queue' is modeled by a 'ComputingEndpoint', and a 'Grid CE' is modeled by a 'ComputingResource'. Do not hesitate to provide answers or ask for precisions. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 0

slides from the OGF GLUE session
by Sergio Andreozzi 29 Oct '10

29 Oct '10

Dear all, in the GLUE session page you can find the link to the presentations folder: http://www.ogf.org/gf/event_schedule/?id=2108 the session was very positive and relevant activities about GLUE 2.0 adoption from LCG and TeraGrid were presented. Up to the end of the year, we will focus our energies on finalizing all the renderings and documents to be sent for public comments. The PGI working group refined the set of requirements they will address in their activities ( http://www.ogf.org/pipermail/pgi-wg/2010-October/000940.html) and the usage of GLUE 2.0 for both advertising resource capabilities from the service side and expressing requirements on them in the client side (JSDL) plays a key role. In the rendering of the XML Schema we should also evaluate how this impacts the requirements expression. A bit of insight on this: the current JSDL envisions a number of terms on which to express requirements. They are part of the JSDL specification therefore binding the resource description to the job description. A couple of years ago, in an OGF joint session (http://bit.ly/byqg5T) we discussed the idea of separating them in order to easy the evolution of resource models independently from JSDL. Requirements are usually expressed as constraints on resource properties, therefore the idea was to enable to express them as XPath/XQuery expressions on the resource description. An example is given in Section 5.3 of this OGF document: http://ogf.org/documents/GFD.137.pdf This is a relevant topic and contributions are very welcome. Other areas of evolution for the activity (after we finalize the renderings) are: - collaborate in the definition of a common information service interface (see presentations from JP Navarro on IIS) - collaborate with the fresh new ARI WG to support them in defining an Instrument data model All the best, Sergio -- Sergio Andreozzi - Policy Development Manager EGI.eu - Amsterdam - The Netherlands - http://egi.eu

1 0