November 2010 - glue-wg - lists.ogf.org

GLUE for describing resource requests (use case DGSI)
by Mikael Högqvist 12 Jan '11

12 Jan '11

Hi, I'm working in the DGSI project (a German D-Grid project) where our goal is to design and implement an interoperability framework between community schedulers and community bound resources. More specifically, this include protocols and services for delegation of activities as well as resources. We are primarily basing our protocols on OGF standards and recommendations and have tight collaboration with both the GRAAP (WS-Agreement/Negotiation) and DCI-Fed working groups within OGF. In the first information model we decided to use GLUE 2.0 for both resource descriptions and resource request specification. However describing resource requests with GLUE turned out to be difficult to implement. The main issue that we have is the restrictions in the GLUE 2.0 XML schema which makes it difficult to use only specific elements/group of elements in a document instance. For example, we would like to use the AccessPolicy element to describe requirements on who (e.g. DN/VO) is authorized to access a delegated resource. But to use this element, we need to create the entire hierarchy with Service/ComputingService/ComputingEndpoint and their respective required elements (e.g. ID, QualityLevel, ...) when using the GLUE 2.0 XSD. Currently, our solution is to use JSDL for specifying both activity and resource delegation requests. However, JSDL is not sufficient for all our use cases/requirements and we would like to use GLUE for specifying resource requests in our next version. Is there any work going on in the glue-wg to make the XSD rendering more lenient towards other use cases such as resource requirement specification? Or making it possible to use a subset of GLUE and combine it with other info models such as JSDL? Thank you in advance and Best Regards, Mikael Högqvist

3 3

Re: [glue-wg] GLUE for describing resource requests (use case DGSI)
by Sergio Andreozzi 19 Nov '10

19 Nov '10

Hi Alexander, yep, now I get it right. The change you require is in the pipeline. Other people already asked for the possibility to create individual fragments without need for the hierarchical dependencies. I will call for your and Mikael validation on the new version as soon as I have it. Cheers, Sergio On 18 November 2010 18:42, <alexander.papaspyrou(a)tu-dortmund.de> wrote: > Hi Sergio, > > probably, I wasn't too clear about what I wanted. > > Actually, we would like to see element-per-type approach that JSDL took. > There, if you need a specific part of the JSDL to be "torn out" and reused > somewhere else (let's say, in some other context where only the "Application > Part" is of interest), you can just do this, because every type (class in > Java-speak) in the XSD has a corresponding element (instance in Java-speak). > With GLUE which doesnt have this in the current state of the XML rendering), > you need to build a full GLUE document and fill several mandatory elements > with nonsense data, just because you need some low-level fragment to express > a certain issue. > > My only concern was that this fragment usage will be allowed in the > end-of-the-year version of the GLUE XSD, and I wanted to make sure it is :-) > > Best, > Alexander > > Am 18.11.2010 um 14:43 schrieb Sergio Andreozzi: > > > Hi Alexander, > > > > the final XSD for GLUE 2.0 is expected by the end of the year. > > > > Concerning the integration with JSDL for resource requirements matching, > this is still open to debate and falls in the area of PGI work. > > > > Nevertheless, the topic has been touched few times in the past. I can > provide my vision: > > > > the current JSDL envisions a number of terms to be used to express > requirements on resources. They are part of the JSDL specification therefore > binding the resource description to the job description. A couple of years > ago, in an OGF joint session (http://bit.ly/byqg5T) we discussed the idea > of separating them in order to easy the evolution of resource models > independently from JSDL. Requirements are usually expressed as constraints > on resource properties, therefore the idea could be to enable to express > them as XPath/XQuery expressions on the resource description. An example is > given in Section 5.3 of this OGF document: > http://ogf.org/documents/GFD.137.pdf > > > > Expressing requirements as constraints on a resource description is the > approach in place since years in the ClassAd language and its gLite JDL > derivation. XPath/XQuery can be a translation into an XML world. > > > > Cheers, Sergio > > > > > > On 18 November 2010 14:17, <alexander.papaspyrou(a)tu-dortmund.de> wrote: > > Sergio, > > > > I think this boils down to more or less the old "allow fragments of GLUE > to be used independently" debate we had in Brussels. Do we have a timeline > for a version bump of the XML/XSD rendering, and will it be possible to > render the XSD as it was done in JSDL (i.e. having an "xs:element" for each > "xs:*Type*")? > > > > Best, > > Alexander > > > > Am 18.11.2010 um 12:27 schrieb Mikael Högqvist: > > > > > Hi, > > > > > > I'm working in the DGSI project (a German D-Grid project) where our > > > goal is to design and implement an interoperability framework between > > > community schedulers and community bound resources. More specifically, > > > this include protocols and services for delegation of activities as > > > well as resources. We are primarily basing our protocols on OGF > > > standards and recommendations and have tight collaboration with both > > > the GRAAP (WS-Agreement/Negotiation) and DCI-Fed working groups within > > > OGF. > > > > > > In the first information model we decided to use GLUE 2.0 for both > > > resource descriptions and resource request specification. However > > > describing resource requests with GLUE turned out to be difficult to > > > implement. The main issue that we have is the restrictions in the GLUE > > > 2.0 XML schema which makes it difficult to use only specific > > > elements/group of elements in a document instance. > > > > > > For example, we would like to use the AccessPolicy element to describe > > > requirements on who (e.g. DN/VO) is authorized to access a delegated > > > resource. But to use this element, we need to create the entire > > > hierarchy with Service/ComputingService/ComputingEndpoint and their > > > respective required elements (e.g. ID, QualityLevel, ...) when using > > > the GLUE 2.0 XSD. > > > > > > Currently, our solution is to use JSDL for specifying both activity > > > and resource delegation requests. However, JSDL is not sufficient for > > > all our use cases/requirements and we would like to use GLUE for > > > specifying resource requests in our next version. > > > > > > Is there any work going on in the glue-wg to make the XSD rendering > > > more lenient towards other use cases such as resource requirement > > > specification? Or making it possible to use a subset of GLUE and > > > combine it with other info models such as JSDL? > > > > > > Thank you in advance and Best Regards, > > > Mikael Högqvist > > > > > > > > > > -- > > Sergio Andreozzi - Policy Development Manager > > EGI.eu - Amsterdam - The Netherlands - http://egi.eu > > -- Sergio Andreozzi - Policy Development Manager EGI.eu - Amsterdam - The Netherlands - http://egi.eu

1 0

Re: [glue-wg] GLUE for describing resource requests (use case DGSI)
by Sergio Andreozzi 18 Nov '10

18 Nov '10

Hi Alexander, the final XSD for GLUE 2.0 is expected by the end of the year. Concerning the integration with JSDL for resource requirements matching, this is still open to debate and falls in the area of PGI work. Nevertheless, the topic has been touched few times in the past. I can provide my vision: the current JSDL envisions a number of terms to be used to express requirements on resources. They are part of the JSDL specification therefore binding the resource description to the job description. A couple of years ago, in an OGF joint session (http://bit.ly/byqg5T) we discussed the idea of separating them in order to easy the evolution of resource models independently from JSDL. Requirements are usually expressed as constraints on resource properties, therefore the idea could be to enable to express them as XPath/XQuery expressions on the resource description. An example is given in Section 5.3 of this OGF document: http://ogf.org/documents/GFD.137.pdf Expressing requirements as constraints on a resource description is the approach in place since years in the ClassAd language and its gLite JDL derivation. XPath/XQuery can be a translation into an XML world. Cheers, Sergio On 18 November 2010 14:17, <alexander.papaspyrou(a)tu-dortmund.de> wrote: > Sergio, > > I think this boils down to more or less the old "allow fragments of GLUE to > be used independently" debate we had in Brussels. Do we have a timeline for > a version bump of the XML/XSD rendering, and will it be possible to render > the XSD as it was done in JSDL (i.e. having an "xs:element" for each > "xs:*Type*")? > > Best, > Alexander > > Am 18.11.2010 um 12:27 schrieb Mikael Högqvist: > > > Hi, > > > > I'm working in the DGSI project (a German D-Grid project) where our > > goal is to design and implement an interoperability framework between > > community schedulers and community bound resources. More specifically, > > this include protocols and services for delegation of activities as > > well as resources. We are primarily basing our protocols on OGF > > standards and recommendations and have tight collaboration with both > > the GRAAP (WS-Agreement/Negotiation) and DCI-Fed working groups within > > OGF. > > > > In the first information model we decided to use GLUE 2.0 for both > > resource descriptions and resource request specification. However > > describing resource requests with GLUE turned out to be difficult to > > implement. The main issue that we have is the restrictions in the GLUE > > 2.0 XML schema which makes it difficult to use only specific > > elements/group of elements in a document instance. > > > > For example, we would like to use the AccessPolicy element to describe > > requirements on who (e.g. DN/VO) is authorized to access a delegated > > resource. But to use this element, we need to create the entire > > hierarchy with Service/ComputingService/ComputingEndpoint and their > > respective required elements (e.g. ID, QualityLevel, ...) when using > > the GLUE 2.0 XSD. > > > > Currently, our solution is to use JSDL for specifying both activity > > and resource delegation requests. However, JSDL is not sufficient for > > all our use cases/requirements and we would like to use GLUE for > > specifying resource requests in our next version. > > > > Is there any work going on in the glue-wg to make the XSD rendering > > more lenient towards other use cases such as resource requirement > > specification? Or making it possible to use a subset of GLUE and > > combine it with other info models such as JSDL? > > > > Thank you in advance and Best Regards, > > Mikael Högqvist > > -- Sergio Andreozzi - Policy Development Manager EGI.eu - Amsterdam - The Netherlands - http://egi.eu

1 0

WLCG Information System - GLUE
by Etienne URBAH 11 Nov '10

11 Nov '10

Flavia, Information Services -------------------- I just finished reading your presentation 'WLCG Information Officer' at http://indico.cern.ch/getFile.py/access?contribId=2&sessionId=0&resId=0&mat… Inside the EGDI project http://edgi-project.eu/ integrating Service Grids (such as WLCG) and Desktop Grids (such as SZDG or IberCivis), I am working on standardization. In particular, I consider that an IS (Information System) permitting providers to publish information and clients to discover information is the mandatory foundation for any distributed data processing infrastructure (Grid). Moreover, standardized interface about data model and protocol for information providers and client queries is mandatory for interoperation between grids such as EGI, OSG, Desktop Grids, ... Therefore, I am member of the OGF GLUE Working Group http://forge.gridforum.org/sf/projects/glue-wg Other involved parties ---------------------- For interoperation, I suggest you to add following stakeholders to your list of 'Other involved parties' : - OGF GLUE, - the EC-funded DCI projects (EDGI, EGI-Inspire, EMI, IGE, StatusLab, Venus-C), - University of Chicago/ANL http://forge.ogf.org/sf/go/doc16100 - Naregi-Renkei, which is currently using CIM. WLCG Information Service ------------------------ The current WLCG Information Service is the BDII based on the GLUE 1.3 schema and bootstrapped with the GOCDB https://goc.gridops.org/ and the CIC https://cic.gridops.org/ Thank you very much for having gathered the opinion of the 4 LCH experiments. Although it is a pity that the 4 LCH experiments do not really trust the WLCG Information Service yet, it is very important that IS designers, developers and integrators know this distrust, in order to search for the real reasons, and to carefully fix all issues. Below are some issues which I know, and proposals for solutions : - Accuracy of published information (in particular storage capacity) : The solution is to take into account 'information provider code' as real software, which must be designed using the best practices of software engineering, must have an installation manual, and must be deployed on all sites using the best practices of grid operation. When an adequate 'information provider' exists and has an adequate installation manual, sites publishing inaccurate information should be blamed. - Relevance and lifetime of published information : Among the various items of information published inside the BDII, users need to know which items they can trust and for how long. This is clearly a requirement for a document (or wiki page) containing a table describing the most relevant items and their probable lifetime. Moreover, it is probably the role of the WLCG Information Officer (yourself) to write down and publish this table. - Rigid implementation of GLUE 1.3 (unless it has already been fixed) : As far as I know, inside a single 'Grid Site' modeled by 'GlueSite' and 'GlueCluster', it would be useful to model each set of machines having same physical characteristics as a separate 'GlueSubCluster', but this is difficult to achieve because installation software does NOT support this completely yet. The solution would be to improve the installation software. - Query language : I am an experienced programmer in various languages, but I assess that building LDAP queries to extract useful information is difficult. A (perhaps restricted) SQL interface would be much easier to use, and there are many software developers and even users who already have a little experience with SQL. - Bad terminology by GLUE 1.3 : A 'Job Queue' is modeled by a 'GlueCE', and a 'Grid CE' is modeled by a 'GlueSubCluster'. Which user would be happy with that ? The solution is GLUE 2.0, where a 'Job Queue' is modeled by a 'ComputingEndpoint', and a 'Grid CE' is modeled by a 'ComputingResource'. Do not hesitate to provide answers or ask for precisions. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 0