September 2014 - glue-wg

Off on holiday
by Paul Millar 30 Sep '14

30 Sep '14

Hi all, Just a quick email to say I'm currently off on holiday this week and will be spending the first half of next week away and quite busy with an "All Hands" meeting; so I'll be unlikely to have time to reply before a Thursday next week. Cheers, Paul.

1 0

Fwd: Re: New attribute for GLUE 2.1: StorageShare.ViewID
by Paul Millar 29 Sep '14

29 Sep '14

Hi all, Sorry, my email client decided to sent the reply directly to Maria, so I'm forwarding it here. -------- Forwarded Message -------- Hi Maria, I started off replying generally, rather than point-for-point, but have included some specific points where it makes sense. Yes, dCache instances would move to GLUE 2.1 info-providers, but I would hope to do that anyway. > Do you mean SRM shares could overlap with Physical Shares or do > you mean that SRM shares overlap among themselves and the same > with Physical Shares? I mean that, for dCache: a. an SRM share do not overlap with any other SRM share. b. a Physical share do not overlap with any other physical share. c. a Physical share may overlap with a SRM share. Statement b. isn't completely true currently (as expressed by the SharingID), but I aim to fix that soon. > In any case, if only Physical Shares are intended to be used > for installed capacity, this should be the ones used by > sites who may want to rely on the BDII for installed > capacities, right? Yes, that's exactly right! The problem is GLUE 2.0 has no way for me to say "these shares are different from those ones", or "these shares are the ones to use for storage accounting". The ViewID is aimed exactly at fixing this: I give all the physical shares the same ViewID ("physical" say) and all the SRM shares the same ViewID ("srm-reservations", say) and then it's easy to select the Physical shares. > What is then published in GLUE2StorageServiceCapacity? The total > of Physical Shares? Yes. > But who has asked for this? My understanding is that nobody > is interested on relying on the BDII for this. I had to smile inwardly here ... Yes, there was an effort, starting around five years ago, in precisely this direction: https://twiki.cern.ch/twiki/pub/LCG/WLCGCommonComputingReadinessChallenges/… "The goal of this document is to detail how to use the Glue Schema version 1.3 in order to publish information to provide the WLCG management with a view of the total installed capacity and resource usage by the VOs at sites. This information can also be used by VO operations and management in order to monitor the VO usage of the resource" Myself (and others) invested considerable time in trying to provide these numbers with GLUE 1.3. GLUE 2.0 was developed with this use-case in mind, albeit (from memory) not extensively reviewed. > [...] I´m not sure is a use case we would like to address. It seems at least one site wishes to make use of these numbers and to automate the process. I don't think we should give up, just because the problem isn't solved yet. > I made a comparison in February between WLCG Accounting reports > and what the BDII and REBUS were publishing in terms of installed > capacities. And I think there was no single match for any T1: > http://wlcg-mon.cern.ch/dashboard/request.py/siteview#currentView=WLCG+Acco… I believe there are many factors influencing such dependencies; however, this would suggest a need for more automation, not less. Such dependencies should be investigated and understood. Yes, more work is needed, investigations, etc; however, we have to start somewhere, so I gave a concrete proposal so people have something they can understand: a straw-man they can hack away at. Cheers, Paul.

3 8

new servcie types
by david.meredith＠stfc.ac.uk 26 Sep '14

26 Sep '14

Hi all, GOCDB has added a number of new service types that could be added to the glue2 enum list: com.schedmd.SlurmClient, eu.unity-idm.Unity, eu.unicore.UnicorePortal, eu.egi.egrant Details copied below and at: https://ggus.eu/?mode=ticket_info&ticket_id=108465 Can these be reviewed and added? Thanks David 1. name of service: com.schedmd.SlurmClient 2. high-level description of the service functionality: SLURM (Simple Linux Utility for Resource Management) is an open-source resource manager designed for Linux clusters of all sizes. Machine flagged this type is hosting software to submit jobs to SLURM. http://slurm.schedmd.com/ 3. project/community/organization maintaining the software: open-source 4. scope of deployment (number of instances of the service currently deployed and by which organizations): ICM (Poland) site is using it, possibly more in EGI infrastructure 1. name of service: eu.unity-idm.Unity 2. high-level description of the service functionality: Unity is a complete solution for identity, federation and inter-federation management. Or, looking from a different perspective, it is an extremely flexible authentication service. (http://www.unity-idm.eu) It replaces UVOS service for UNICORE. Possibly each NGI supporting UNICORE mware should deploy one instance. 3. project/community/organization maintaining the software: open-source 4. scope of deployment (number of instances of the service currently deployed and by which organizations): ICM installation for the Polish NGI with redundant installations in future. 1. name of service: eu.unicore.UnicorePortal 2. high-level description of the service functionality: UNICORE Portal service allows users to access a UNICORE Grid via the web browser. Service be used by users instead of standalone clients like UCC and URC to access resources and submit jobs. The UNICORE Portal server is available for download at UNICORE site: http://unicore.eu/download/unicore6/. It is possible in the future that each NGI supporting UNICORE middleware will deploy at least one instance. 3. project/community/organization maintaining the software: open-source (UNICORE community) 4. scope of deployment (number of instances of the service currently deployed and by which organizations): ICM installation for the Polish NGI. 1. name of service: eu.egi.egrant 2. high-level description of the service functionality: platform for Resource Allocation management in EGI Infrastructure 3. project/community/organization maintaining the software: egi.eu 4. scope of deployment (number of instances of the service currently deployed and by which organizations): https://e-grant.egi.eu, deployed for EGI by Cyfronet ========================== David Meredith Scientific Computing Dept Daresbury Laboratory Tel: +44 (0) 1925 603762 Fax: +44 (0) 1925 603100 (Site) Email: david.meredith(a)stfc.ac.uk Skype name: davidismeredith -- Scanned by iCritical.

4 3

New attribute for GLUE 2.1: StorageShare.ViewID
by Paul Millar 26 Sep '14

26 Sep '14

Hi all, After playing some more with GLUE 2.0 (and getting feedback from someone trying to use it!), there seems to be a missing concept in StorageShare, which I'll call "views". The idea is to allow publishing two (or more) groups of Storage Shares such that, within such a group, the Storage Shares can be non-overlapping but shares from different groups can overlap. Each group of non-overlapping shares provides a different "view" on the storage system. It's likely that the underlying hardware is the same (with the sum of StorageShareCapacity.Total giving the same value), but the different views (different groups of storage shares) would partition this storage differently. Here's a more concrete example: consider a storage service that allows capacity to be reserved but doesn't bind those reservations to particular hardware. This is difficult to publish currently; but, these could be published as two different views: one view shows the hardware, the other shows the reservations. I believe we can add views while remaining backwards compatible with GLUE 2.0 attributes and semantics; in particular, we can keep the semantics of SharingID unmodified for existing publishers. Here's the proposal: Attribute: ViewID Type: LocalID_t Mult: 0..1 ("optional") Unit: <none> Description: A local identifier that groups together similar Storage Shares. Two shares are part of the same view if and only if both do not publish a ViewID or both publish the same ViewID value. Two Storage Shares that are part of the same view and have either distinct SharingID values or the same "dedicated" SharingID value are non-overlapping, otherwise they may overlap. Since the attribute is optional, existing publishers will continue to work, but will refrain from publishing a ViewID attribute. Since not publishing ViewID places the Storage Share objects in the same view, the GLUE 2.0 SharingID semantics is preserved. Cheers, Paul.

3 7

New version of GLUE 2.1 draft
by Salvatore Pinto 19 Sep '14

19 Sep '14

Dear GLUErs, attached to this mail you will find the latest version of the draft for GLUE 2.1, with the inclusion of the Cloud entities. Respect to the last draft sent, I have performed the following changes (all changes/additions are highlighted in yellow in the document): - I reviewed the representation of the virtual resources available to the VM, and decided to adopt the OpenStack model+GPU, so RAM,CPU,GPU,Network (public or private),OS Disk and Ephemeral disk. This model is the more general I could find between the open source middlewares and should fit almost all of them. - I reviewed the representation of service pricing (costs). The overall pricing issue is quite complex, and a comprehensive standardization of pricing schemes is way out of our scope. Anyway, for brokers and markeplaces to consume this information and select the "best" site for the user, I have included a simple representation via price entities which are associated to the VM image disk and resource. The final price for the VM is calculated as the sum of all the associated price elements (eg. RAM, CPU, OS License, etc...). - I added SLA (service level agreements) related to the cloud service. This is important for brokers (eg. you may select the site with lower price, but only if it provides the SLA you need). I did not standardize SLA representation, but just added a string to represent the SLA, which can be an URL or a short nametag (eg. 99% Avaliability, best effort, etc...). - I added a service ToC parameter as an URL reference to the ToC per the site. When you program a cloud broker, you need your user to accept the ToC of each of the cloud providers at first use. I think GLUE is a nice way for the broker to access the ToC of all the different sites in an uniform way. - I added the possibility to specify pre-set access credentials for the VM images. This information is relevant to the broker system if cloud-init is not supported by the provider or the image. Cheers, Salvatore.

4 4

Call for minor non-distruptive updates to GLUE 2.0
by Salvatore Pinto 08 Sep '14

08 Sep '14

Dear GLUErs, since we are going to revision GLUE 2.0 to GLUE 2.1, I would like to ask the mailing list to point me to some changes which, in your opinion, should be incorporated in this revision. I will collect these changes and then organize a discussion for the GLUE WG meetings. Changes submitted shall be NOT-DISTRUPTIVE and ensure full retro-compatibilty with GLUE 2.0. Example of such changes may be fixing of typos, making description of attributes more clear (without changing the attribute meaning), updating enumerations by adding new fields, etc... >From my side, beside, of course, the addition of the cloud entities, I would like to propose: #1. Put a link to the Open Enumerations procedure. I propose to add a short text with the link in appendix B (Data types), explaining that, before defining a new enumerations, it is recommended to apply the Open Enumerations best practices. #2. Fix the broken link to VOMS website ( http://voms.forge.cnaf.infn.it/) I propose to replace it with the WikiPedia link ( http://en.wikipedia.org/wiki/VOMS) which is the closest one to a "persistent identifier" I can find, or remove it entirely. #3. Add to EndpointHealthState_t (closed enumeration) a state named "downtime", this would be much simpler to signal current downtime status to the users or cloud brokers than via the DowntimeStart/End times. #4. Add GPU resources to the Execution Environment. This can be important for the EGI-ENGAGE and other sister projects in H2020, were we are planning to integrate GPGPU computing in both Cloud and Grid. #5. Add to the StorageShare a Maximum and Minimum object (file) size allowed. THis would be of interest in the cloud, there the object stored can be for example an additional virtual disk which size can be restricted between given boundaries (eg. for most of the EGI FedCloud sites from 1GB to 100GB). #6. For the storage elements descriptions, replace the word "file" with "data object". The idea is that the storage service may support multiple data objects types (eg. files, images, documents, etc...). The kind of objects supported will be included the service capabilities. Cheers, Salvatore.

5 9

Fwd: Re: Call for minor non-distruptive updates to GLUE 2.0
by Paul Millar 08 Sep '14

08 Sep '14

Sorry, I mistakenly replied only to Alan, but I meant to reply to the list. -------- Forwarded Message -------- Hi Alan, On 08/09/14 15:25, Sill, Alan wrote: >> On Sep 8, 2014, at 2:13 PM, Paul Millar <paul.millar(a)desy.de> >> wrote: >> 1. create a registry where people can add CA bundle along with >> some canonical name, > > This type of registry, if created, would be meaningless without > secure links to the CA bundles and an accompanying description of > policy for each. This in fact is what the IGTF provides for each of > its published bundles. I'm not sure what you mean by "meaningless", but certainly the registry entry for some CA-bundle should allow a user to navigate to the list of certificates and a description of the bundle's policy. (Be aware that this is not a security-related issue: the published information only provides a hint whether or not a client's X.509 certificate is from a CA the server trusts.) > TACAR maintains a list of individual CAs along with secure links to > download their individual CA trust anchor files, but only for a > subset of academic CAs. > > I suggest generic language that refers to both as examples. My point is that we shouldn't include these kinds of examples; they're point to a problem in the GLUE document that can be fixed for GLUE 2.1 The problem is that we (the people writing GLUE 2.x documents) might know what "IGTF-SLCS" should mean but unless it's written down we risk: a. someone will use an incompatible value to refer to the same thing ("igtf slcs", "urn:ogf:glue:ca-bundles:igtf:slcs", ...) b. someone won't understand what "IGTF-SLCS" means. HTH, Paul.

1 0

VOTE by 7/31: GLUE 2.0 LDAP Rendering release for public comment
by JP Navarro 02 Sep '14

02 Sep '14

GLUE Working Group: As agreed to at OGF 41. By July 31, please review the following document and reply with a vote to APPROVE or REJECT to release for public comment. http://redmine.ogf.org/dmsf_files/13302?download= If you vote to REJECT, please summarize your concerns. If you have minor editorial corrections or improvements, please reply and attach your change tracked edits and appending your initial to the file-name. Regards, JP and Shahbaz

6 11