May 2008 - glue-wg - lists.ogf.org

updates before OGF 23
by Sergio Andreozzi 30 May '08

30 May '08

Dear all, some update about GLUE-related topics: 1. the forge portal is back at work; the public comment process should officially start next week after the GFSG F2F meeting (I will send an email to confirm it) I've updated the wiki with the links to the tracker items related to the submission to the OGF Editor 2. next week, the important event: OGF 23 for those attending OGF, please mark your agenda with the following sessions: 2.1 Group Sessions for any of this sessions, mail me in advance if you want to present any GLUE-related activity (especially for the discussion and implementation sessions) Wednesday GLUE 2.0 - Overview http://ogf.org/gf/event_schedule/index.php?id=1221 Thursday GLUE 2.0 - Discussion http://ogf.org/gf/event_schedule/index.php?id=1222 GLUE 2.0 - Implementation http://ogf.org/gf/event_schedule/index.php?id=1223 2.2. Joint Group Sessions Wednesday BES/HPC, JSDL and GLUE Profiling http://ogf.org/gf/event_schedule/index.php?id=1274 this is an important session to start discussing how to profile the usage of GLUE with the other specifications related to job management 2.3 Tutorial on implementing standard for managing computing activities (organized by OGF-Europe with the collaboration of KnowArc, EGEE, UNICORE and PSNC) Thursday http://ogf.org/gf/event_schedule/index.php?id=1256 http://ogf.org/gf/event_schedule/index.php?id=1317 http://ogf.org/gf/event_schedule/index.php?id=1369 this is a unique chance to learn about core OGF specs and the current adoption activity. Implementors will present both the specifications and status of their implementations. Looking to meet several of you in Barcelona, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

1 0

GLUE 2.0 Spec and Rendering documents submitted for public comments
by Sergio Andreozzi 20 May '08

20 May '08

Dear all, I'm happy to announce that we submitted 2 documents for public comment. From the feedback of the last call, we have identified 195 autonomous comments. They are collected together with the answer in GLUE2-WGLastCall-Q&A.doc. We have resolved 135 comments, while 60 of them have been postponed for resolution during the public comment process. Among the 60 issues collected in the document GLUE2-ToDo-PublicComment.doc, several refer to editorial changes, while some requires some discussion about the model. The title of the submitted documents for starting the public comment process are: - GLUE Specification v. 2.0 (the conceptual model of GLUE 2.0) - GLUE v. 2.0 – Reference Realizations to Concrete Data Models (realization in XML Schema/LDAP Schema/SQL) You can download a compressed archive containing all the documents at the following URL: http://www.cnaf.infn.it/~andreozzi/doc/GLUE2_0_Public_Comment.zip content: - GLUE2-ToDo-PublicComment.doc/pdf - GLUE2-WGLastCall-Q&A.doc/pdf - ogf-glue-2-rendering.doc/pdf - ogf-glue-2_public_comment.doc/pdf - ogf-glue-2_public_comment_with_track_changes.doc/pdf When the forge portal will be more stable, I will upload them in the document repository as well. The public comment period is officially expected to start just before the next OGF. We need to wait for the confirmation by the OGF Editor. I wish to thank all of you for the contribution to this important milestone. Kind regards, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

1 0

More comments on the draft spec.
by Paul Millar 20 May '08

20 May '08

Hi all, Some more comments. A general comment: To answer some questions, multiple passes through the info-system may be required. For example, see "Which sites have SEs which support the gsidcap protocol?" within Stephen's page, here: http://egee-uig.web.cern.ch/egee-uig/production_pages/Advancedldapsearch.ht… With LDAP, the object's RDN is often built from its ID or LocalID. So, fields derived from an object's RDN (such as GlueChunkKey and GlueForeignKey) will be dependent on LocalID (or ID). I believe Glue currently makes no statement about how long LocalID or ID should remain constant for the same object: the persistence of the LocalID value. Although silly, I believe it would currently be acceptable to generate fresh random LocalIDs for each object every time data is published, provided all references were updated at the same time. However, if one uses the RDN (for example, by following a GlueForeignKey value in a separate query), there is a tacit assumption that the RDN of the target object will not change (or, at least, that it is unlikely to change for the period between successive queries). This is only true if the ID or LocalID used to build the RDN doesn't change. So, it seems we have a requirement for IDs or LocalIDs to be persistent over time. This should be stated somewhere, probably in section 3 (General Comments). Some more comments: *** Page 12 "activity" is misspelt in the description of the Associate End for Share.LocalID *** Page 28 When providing a diagram representing the specialisation of entities (e.g., Fig. 3) the inherited associations are not shown. Could these be added somehow (e.g., within the Main Entities section)? Also, it isn't immediately clear that the entities in "Storage Entities" are the same as those in "Storage Entities - Inheritance". Could this identification be included in the diagram? *** Page 30 It's a little unclear why we have both StorageAccessProtocol and StorageEndpoint since StorageEndpoint can represent access protocols. The StorageAccessProtocol seems to be only of use when talking about the CE-SE-bind objects. If so, perhaps the description of StorageAccessProtocol could be updated to mention this. There's still the question why Capability is a required property of StorageEndpoint: this is an echo of my previous point about this, suggesting that it is optional in Endpoint or simply removed and added to a new subclass of Endpoint. *** Page 31 I feel we should mention in the description of StorageShare that it is: A UserDomain's view of [a utilization target for a set of StorageResources ...] This may not be obvious, especially as the UserDomain--StorageShare association is currently not shown on the Storage entities diagram (Fig.3) as it's inherited. *** Page 33 StorageResource: The Latency is the maximum latency under normal operating conditions, not the maximum under any circumstance. Cheers, Paul.

3 4

Status of GLUE 2.0 Specification as regards going public comment
by Sergio Andreozzi 19 May '08

19 May '08

Dear all, this is to inform all of you that we are still in the process of digesting all the feedback that we received about the GLUE 2.0 Spec, WG last call. We are creating a document containing all the feedback and how each individual point was considered as regards the public comment version of the spec. At the moment, we processed 50% of the feedback and we expect to finalize it and send the document to the area directors by tomorrow COB, 20 May. Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

1 0

Invoking some magic: RDF, VO annotations and Grimoires
by Paul Millar 19 May '08

19 May '08

Hi all, In a round-about way, I (re)discovered a project a friend mentioned a while ago, called "Grimoires": http://twiki.grimoires.org/bin/view/Grimoires/ If you're not familiar with the project, it's basically registry that allows for service discovery, based on UDDI but allowing RDF (i.e., "semantic") annotations by multiple user communities. This should allow different end-user communities to annotate a service in almost any way supported by RDF. Re-reading the page, I was struck by what Stephen said earlier about allowing user-communities to annotate GLUE information. The interesting part comes if all GLUE entities were cast as either a UDDI-registered service or as an annotation of a service. It should be relatively easy for someone with some spare time hands (i.e., not me :) to map GLUE vocab. into RDF and allow GLUE-like publishing of information into a Grimoires server. This would allow end-users to ask some nice queries, combining their "personal" metadata with GLUE information. There's another RDF project that hook into GT4: S-MDS. IIRC, S-MDS maps the MDS information into a SPARQL end-point via some (fairly static) mapping rules. I suspect it could be persuaded to ingest GLUE (in-band via MDS or out-of-band), but 3rd-party annotations would have to be out-of-band (unless they have write-access to MDS). Whereas the Grimoires project seems to be a more integrated approach. Should we make contact with these people to see if they're interested in adopting GLUE as a case-study? It might prove useful if we provide an RDF binding for GLUE. Cheers, Paul.

2 1

v1.3 documentation, please!
by Paul Millar 19 May '08

19 May '08

Hi all, Sorry, this is a bit of rant. I'm in the process overhauling how dCache information is published into GLUE: a complete rewrite of how information is propagated. So I'm looking for definitive information about how to publish Glue through GIP. ... and there's the rub. I can't find any good references for how to provide this information. I've started a dCache wiki page[1] that holds various nuggets of information, but none of them seem complete or authoritative. [1] http://trac.dcache.org/trac.cgi/wiki/GLUE I understand that the current dCache system, is functional through Owen's sterling effort, with various trial-and-error attempts and information conveyed through emails. I'm looking to avoid this process as much as possible; having accurate information would be a good start! First off, the only GLUE v1.3 documentation I could find is from CNAF CVS [2]. The most recent version is "Draft 3---16 Jan 2007". I think Sergio was going to make this the final version, but could this process be clarified? BTW, there's a typo in the StorageArea, UsedNearlineSize description; there are other sections that look somewhat incomplete (e.g., grep for "TO BE ADDED") [2] https://forge.cnaf.infn.it/plugins/scmsvn/viewcvs.php/*checkout*/v_1_3/spec… Second, I could also find no authoritative source of information on Glue/LDAP binding for v1.3. There are some CNAF notes[3], but these mention, in bold, that "[t]his version is for early evaluation and is not meant to be deployed yet". Can someone say what is incorrect on this page? More importantly, can someone update it so the page is correct? [3] http://glueschema.forge.cnaf.infn.it/SpecV13/LDAP The CNAF notes also mention that certain attributes are "deprecated and their use should be removed from any software". Does this mean info-provides "must" (or "should", or "may", see RFC 2119) refrain from publishing deprecated attributes? For example, should no info-provider be publishing GlueChunkKey anymore? For publishing SRM spaces, I found a *proposal* for how this should be done[4]. Confusingly, this contradicts the GLUE/LDAP notes[3] as it stipulates that "mds-vo-name = local" must be used, whereas the notes state that mds-vo-name has been removed from the DIT; in practise "mds-vo-name = resource" (as a primary document, for feeding into GIP) is what seems to work. [4] https://twiki.cern.ch/twiki/bin/view/LCG/GSSDGLUEProposal Finally, (prodding Laurence) I could not find *any* statement about what format GIP supports as primary input (although I might have missed this). I'd assume this is roughly GLUE v1.3/LDAP, but apparently there's some magic happens with DNs with RDF of "mds-vo-name=resource". Is this (still) true? Is it required? What other mappings and translations are possible? Sorry, I'm sure this info is well known by others, but it doesn't appear to be documented. Can some knowledgable person take on responsibility for ensuring this is all properly documented? Cheers, Paul.

7 18

GLUE 2.0 Specification - draft 41 - Working Group Last Call
by Sergio Andreozzi 16 May '08

16 May '08

Dear all, in our plan, we have the important milestone of presenting for the coming OGF a public comment version GLUE 2.0 in order to achieve a broader exposure of our work and receive a broader review. Starting from the GLUE 2.0 specification available at the end of April (which is the result of a long series of teleconferences and email exchange by many partners) we refactored the document to be consistent and we completed all its parts. The updated renderings of the conceptual model to the XML Schema/LDAP/relational concrete data models will be issued during this week in a separated document (possibly by Wednesday). This is the official Working Group last call for GLUE 2.0 and we invite all of you to read the specification and to send your feedback by Friday, 12 AM CEST. By Monday 19 May, we will send the document + renderings to our area managers to be presented to the OGF Editor for promoting it to public comment. All the comments that we will receive by Friday 16 May and that can be easily incorporated, will be included. The remaining comments which may require discussion or major changes will be logged in the public comment tracker of the GLUE 2.0 specification as soon as it will be available and will be resolved during the 60 days of public comment period. When sending your feedback, pay attention to aspects related to the readability and consistency of the document. We would like to remark the importance of going to the OGF 23 with a coherent and complete document aligned with the renderings. This will enable the interested groups/projects to better evaluate the current work and create proof of concepts on real systems. We thank all of you for your contribution. Kind regards, Sergio, Balazs and Laurence PS: you can find attached the specification documents in both MS Word and PDF formats; the OGF forge portal is not reachable at the moment -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

5 14

Storage Sample in Glue 2.0
by Riccardo Zappi 09 May '08

09 May '08

Hi, I am spending some time to define an example of storage with the current Glue Schema draft. Unfortunately, I have some difficulties to represents the Data Store and the Storage Resource as you have defined into Glue Schema. Suppose a site with storage system organized with a SAN and cluster file system, like GPFS. Using the same SAN and cluster file system, I could define and mount different file systems upon the same infrastructure. For example, GPFS cluster can contain up to 256 mounted file systems. I can define some StorageShares using quota GPFS tools. In this case, the SS sizes are based on GPFS metadata. Therefore, I define StorageShare using file system feature. I imagine one StorageResource per each file system created on GPFS. In this case every StorageResource could have a size and quality properties. Some StorageShares could share the same StorageResource (i.e. file system) entity. When I try to describe how StorageResource is composed some doubts appears. I could lists all volumes composing the file system.. For example in the figure attached, you can see that 16 volumes compose the file system named ‘gpfs_storm’. In any case, heterogeneous disk devices could compose each volume... and then DataStore description appears incorrect. Are you sure that Storage Service administrator knows every details of underlying storage system? Moreover, if the admin knows, are you sure that he wants describe all these details? I think we have to improve DataStore-StorageResource concepts. Perhaps, we can go on to try to merge both concepts in only one entity with multi-valued quality properties. Cheers, Riccardo -- ------------------------------------------------------------------------------- Riccardo Zappi Istituto Nazionale di Fisica Nucleare - CNAF Viale Berti Pichat, 6/2, 40127 BOLOGNA - ITALY Phone: +39-051-609-2868 Fax: +39-051-609-2746 e-mail: riccardo.zappi(a)cnaf.infn.it -------------------------------------------------------------------------------

2 1

Comparison with CIM
by Paul Millar 09 May '08

09 May '08

Hi all, I don't know if anyone has been looking at this, but there's an emerging standard (through DMTF) in modelling "things" (computer-hardware and -software, network equipment, etc): CIM. There seems to be some cross-over here through OGF, so forgive me if this is already "well known". Anyway, for those who haven't heard of them, their schema makes for interesting reading. The current version (2.18) is available from: http://www.dmtf.org/standards/cim/cim_schema_v218 "Device" (from which a StorageDevice may be published) http://www.dmtf.org/standards/cim/cim_schema_v28/CIM_Device28-Final.pdf How to publish ACLs: http://www.dmtf.org/standards/cim/cim_schema_v218/CIM_Policy.pdf Enjoy! Paul.

8 24

TmpDir, ScratchDir and DataDir
by Sergio Andreozzi 09 May '08

09 May '08

Hi JP and Gerson, during this call http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMeeting… we agreed that the attributes TmpDir, ScratchDir and DataDir part of the Computing Resource (e.g., a batch system) should be removed: http://forge.ogf.org/sf/sfmain/do/go/artf6196 the values can be discovered by the job in the worker node environment. Do you confirm that this is ok with you? Cheers, Sergio -- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi

3 4