- glue-wg - lists.ogf.org

Handling VO specific information in Glue 2.1
by Enol Fernández 19 Oct '17

19 Oct '17

Hi *, as we start to implement v2.1 of the GlueSchema for the EGI federated cloud we have realised that we miss some VO specific information and we are unsure how/where is the best way to publish this: Users belong to several VOs and sites may support more than one of those. When users authenticate against a given endpoint, the endpoint will return a list of local projects/groups that the users is allowed to use. Each project/group supports a VO in our current implementation. While in the past this was not an issue since the user authenticated with a VOMS proxy that only contained information a single VO and therefore the endpoint would just return a single project. Now with the transition to federated identity, the endpoint will receive claims on every VO the user is member of and there is no way for the user to determine which local project/group to use. We would need a way to publish a site-defined identifier of the project/group that supports each VO at a given site, so user could just match the VO with that id and select the appropriate one during authentication. We have checked the current draft of the schema and haven't seen a clear place to publish this kind of information. Our current guess would be to include this in the AccessPolicy or MappingPolicy. Since in our implementation we are using shares as a way to express VO information probably for us the MappingPolicy is the best fit, but would like to get your input on the best way to proceed. PS: I tried to submit this email before being subscribed, so apologies if it gets sent twice. > -- Enol Fernández | Cloud Technologist | EGI Foundation enol.fernandez(a)egi.eu

4 6

new version of UML class diagrams for GLUE2.1
by Alessandro Paolini 02 Oct '17

02 Oct '17

Dear all, I've made a first version of the UML class diagrams (htc and cloud) comprising the new classes and other changes, for replacing the ones currently included in the GLue2.1 Specification draft document. Please have a look at them and let me know if you have any comment: https://docs.google.com/document/d/1x2gUt2lJNevle8evQu6LcKr9xs5NHRd6FbNzb1y… cheers, Alessandro -- Dr. Alessandro Paolini Operations Officer - EGI Foundation Science Park 140 1098 XG Amsterdam The Netherlands skype: alessandro.paolini.egi ********************************* "I believe in the power of laughter and tears" "as an antidote to hatred and terror" "A day without laughter" "is a wasted day" >>> Charlie Chaplin

1 0

location of the new attribute VirtualizationType
by Alessandro Paolini 25 Sep '17

25 Sep '17

dear all, after adding in the draft the new GLUE2 classes for describing the accelerators ( https://docs.google.com/document/d/1BpeTBQfCn8xRo4Ff38esRhBwzoUHE4bVxba2rEA… ), Paolo is wondering if the attribute VirtualizationType (The virtualization mode adopted for creating the virtual accelerator device), now located into the new class CloudComputingManagerAcceleratorInfo, should be moved instead into CloudComputingVirtualAccelerator class. Paolo, why do you think it should be better moving it in the other class? Initially I was thinking that CloudComputingVirtualAccelerator class was better, but know it seems to me that both the classes are suitable for containing this attribute, it can be either into one or another: do you think we can add it to both the classes? cheers, Alessandro -- Dr. Alessandro Paolini Operations Officer - EGI Foundation Science Park 140 1098 XG Amsterdam The Netherlands skype: alessandro.paolini.egi ********************************* "I believe in the power of laughter and tears" "as an antidote to hatred and terror" "A day without laughter" "is a wasted day" >>> Charlie Chaplin

3 3

Glue 2.1 draft new version (0.5) - little modification
by Alessandro Paolini 19 Sep '17

19 Sep '17

Dear all, when working on the implementation of the new schema for the cloud prototype, with Baptiste and Enol we agreed on a little modification regarding two attributes for better representing the current situation. In the CloudComputingEndpoint class there are the following attributes: - ContextualizationName: supported contextualization mechanism (cloudinit, bash), if any; - ContextualizationVersion: supported contextualization mechanism version In our opinion these attributes should refer to the single image rather than describing a global feature of the entire endpoint; they would have more sense if they belong to the CloudComputingImage class. Our proposal is moving them under CloudComputingImage class, so their new name will be: GLUE2CloudComputingImageContextualizationName GLUE2CloudComputingImageContextualizationVersion Please let us know what do you think about it, in case we can also discuss it next week during the meeting. For you convenience I've also produced a new version of the draft, attached to this email, reflecting this change. Best regards, Alessandro -- Dr. Alessandro Paolini Operations Officer - EGI Foundation Science Park 140 1098 XG Amsterdam The Netherlands skype: alessandro.paolini.egi ********************************* "I believe in the power of laughter and tears" "as an antidote to hatred and terror" "A day without laughter" "is a wasted day" >>> Charlie Chaplin

5 22

naming the new StorageShare attribute requested by ATLAS
by Alessandro Paolini 14 Sep '17

14 Sep '17

Hi Jens, all coming back to this topic: add a StorageShare attribute that describes the number of files stored in this StorageShare. This should be optional (mult: 0..1). Unfortunately the original email by Paul got lost, and I don't know if there was some discussion as well at that time, before you remind us this topic. I would like just to discuss the name of this new attribute: something like StorageShareFiles or StorageShareTotalFiles ? let me know, alessandro On 24 August 2017 at 16:28, Alessandro Paolini <alessandro.paolini(a)egi.eu> wrote: > Hi Jens, > > sorry for not replying before: I've just noticed that your message (and > the consequent reply by Stephen) finished in the spam folder... > > Regarding the P1, I will correct the document, thank you for spotting this > (I'm being in vacation since tomorrow till Sep 12th, so in the next two > weeks i'm not being able to participate to the discussion). > > Regarding P2, P3, and P4 it is not completely clear to me the need, maybe > because I look at the GLUESchema as operator/administrator rather then > user, and I don't know well the needs of the experiments. > > cheers, > Alessandro > > On 23 August 2017 at 16:45, <stephen.burke(a)stfc.ac.uk> wrote: > >> glue-wg [mailto:glue-wg-bounces@ogf.org] On Behalf Of Jensen, >> > Jens (STFC,RAL,SC) said: >> > One thing I did not see in this document - but had expected? - was to >> > address Paul's suggestions from November last year. See attached. >> >> We didn't discuss this, but my comments would be: >> >> P1: seems uncontroversial. >> >> P2: The semantics for multiple paths would need to be clear - in the past >> it was used as a default path on which to write files, if you have more >> than one it wouldn't be obvious. A more backward compatible approach would >> be to add a new multivalued attribute for additional paths, then it's clear >> that the existing one should be the default. >> >> P3: The reason for having a separate Capacity object was that there are >> potentially many different semantics for what the attributes can refer to. >> Putting the attributes back in the main object would need both a clear >> definition for which precise semantics are intended, and a case for why >> those particular semantics are universally the most important ones. >> Personally I don't think it's a useful idea given that the information is >> already available from the existing object. >> >> P4: Seems like a strange idea to me. StorageShareCapacity is logically a >> part of the StorageShare object, hence the name and indeed the proposal in >> P3. If a similar object is wanted for Datastore then it should be a new >> object with a new name. In principle it's possible that one Share could be >> spread across multiple Datastores so there's no simple relation between >> them. >> >> Stephen >> >> >> >> _______________________________________________ >> glue-wg mailing list >> glue-wg(a)ogf.org >> https://www.ogf.org/mailman/listinfo/glue-wg >> > > > > -- > Dr. Alessandro Paolini > Operations Officer - EGI Foundation > Science Park 140 > 1098 XG Amsterdam > The Netherlands > skype: alessandro.paolini.egi > ********************************* > "I believe in the power of laughter and tears" > "as an antidote to hatred and terror" > "A day without laughter" > "is a wasted day" >>> Charlie Chaplin > -- Dr. Alessandro Paolini Operations Officer - EGI Foundation Science Park 140 1098 XG Amsterdam The Netherlands skype: alessandro.paolini.egi ********************************* "I believe in the power of laughter and tears" "as an antidote to hatred and terror" "A day without laughter" "is a wasted day" >>> Charlie Chaplin

2 2

GLUE WG teleconference, Tuesday, August 22, 2017
by Navarro, JP 13 Sep '17

13 Sep '17

Greetings GLUE WG, Our next GLUE WG teleconference will be: : Tuesday, August 22, 2017 : 15:30-17:00 Central Europe, 8:30-10:00 AM US Central We will use Skype for Business teleconference and screen sharing: : Web: https://meet.illinois.edu/ncsa-meeting-manager/L37V93J4 : --> Cancel login dialog, click guest "Sign in here instead", enter name, click "Join the meeting" : Phone: 1-888-983-3631, International 1-217-332-6338 or 1-312-994-8410 : Conference ID: 4463261 As a fall back we also have: : Web: https://webconf.vc.dfn.de/ogf-glue-wg/ (Adobe Connect) : Login as Guest using your real name IMPORTANT: Recommend that those that need to configure and test Skype for Business please try to connect 15 minutes early. Proposed agenda: 1) Reminder that we released two official documents since our last meeting (2 minutes) LDAP rendering: http://www.ogf.org/documents/GFD.218.pdf JSON rendering: http://www.ogf.org/documents/GFD.219.pdf 2) Cloud Extensions (55+ minutes) Review and discuss Alessandro’s GLUE 2.1 Cloud Extensions Prototype Regards, JP

2 4

GLUE 2.1 prototype
by Alessandro Paolini 14 Jun '17

14 Jun '17

Dear all, we would like to announce that the work on the prototype for the GLUE2.1 Schema is completed. My colleague Baptiste worked on the cloud info-provider for OpenStack for including the cloud extensions to the current glue schema and we would like to present and discuss the result in the next Glue-WG meeting, which can be scheduled on this purpose, in order to have the official approval for releasing in production the modifications to the schema. Best regards, Alessandro -- Dr. Alessandro Paolini Operations Officer - EGI Foundation Science Park 140 1098 XG Amsterdam The Netherlands skype: alessandro.paolini.egi ********************************* "I believe in the power of laughter and tears" "as an antidote to hatred and terror" "A day without laughter" "is a wasted day" >>> Charlie Chaplin

2 1

Update from ATLAS evaluation of GLUE2
by Paul Millar 25 Jan '17

25 Jan '17

Hi all, Here's a update from ATLAS' evaluation of GLUE2. The process is ongoing and not yet finalised. ATLAS is currently considering to take aspects of GLUE2 (trying to use the same attribute names, for example) but at the moment they are considering a format that would not be GLUE2 compliant. The process has helped identify some sticking points, which I propose we should address. Here are two concrete proposals that would help ATLAS: P1. add a StorageShare attribute that describes the number of files stored in this StorageShare. This should be optional (mult: 0..1). P2. adjust multiplicity of StorageShare#Path to 0..* The other issue they identified was that StorageShare doesn't store capacity information: that requires a whole new object (with IDs and linkage). Here's a concrete proposal (so people can disagree :-) on how we can fix this: P3. add the StorageShareCapacity usage metrics (TotalSize, FreeSize, UsedSize, ReservedSize) to StorageShare as optional attributes (mult: 0..1) These attributes MAY be used in situations where is clear which media is being used. In such circumstances, the StorageShare SHOULD have at most one DataStore association. P4. Rename StorageShareCapacity to DataStoreUsage, adding a required Association to a DataStore. This better reflects that StorageShareCapacity links a StorageShare to some corresponding hardware. P1 and P2 are both rather trivial -- these should be fixed in GLUE v2.1. P3 is also (technically) rather trivial -- if we want, this could go in GLUE v2.1 P4 would not be backwards compatible, so we would have to be a little careful how we roll it out. Cheers, Paul.

2 1

new service type requests
by david.meredith＠stfc.ac.uk 16 Dec '16

16 Dec '16

Hi all, We currently have three new service type requests registered as pull requests, see: https://github.com/OGF-GLUE/Enumerations/pulls These are: - "org.onedata.onezone" - Onezone is a Onedata component responsible for connecting multiple storage providers with installed Oneprovider component into federations of storage providers - "com.aws.s3" - S3 is a storage API - "com.ceph.object-storage" - Object store, see http://ceph.com I'm happy to merge these provided there are no objections/issued identified by the group - please let me know by COB Fri 16th Dec. Thanks, David ------------------------------------- David Meredith Distributed Computing Infrastructure Group Scientific Computing Dept Daresbury Lab Warrington, WA44AD Skype: davidismeredith (residing in stoke-on-trent) Phone: +44 (0) 1925 603762

3 3

ATLAS, GLUE and JSON
by Paul Millar 24 Oct '16

24 Oct '16

Hi everyone, <background> A little while ago, the WLCG VO ATLAS started investigating how to do storage accounting in a platform neutral fashion (i.e., not depending on SRM). Their idea was that a site would "somehow" ensure that their storage system always has a file containing the necessary information. The file would be stored in a standard location. ATLAS could then read this file using whichever protocol is easiest. The plan was for this file would be encoded as JSON. They also put together a rough description of how this file should look like. I suggested that they investigate GLUE 2 and the GLUE-JSON binding, rather than coming up with their own proprietary solution. </background> So, ATLAS are asking for some information: o people with whom they can discuss how to use GLUE2 and GLUE-JSON. o tools that parse GLUE-JSON. I guess this would be mostly from XSEDE guys. So, does anyone want to help them evaluate GLUE 2? Cheers, Paul.

8 24