Thoughts on WLCG storage accounting use-cases
Hi all, So, I've been thinking about publishing storage related objects, particularly with the WLCG accounting use-cases. One of the use-cases calls for associating each portion of a storage element to one or more VOs so that these can be used for per-VO accounting. In Glue 2.0, this association most naturally follows from instantiating MappingPolicy objects that act as links between StorageShare objects and UserDomain objects. An interesting issue is how to represent a single shared (i.e., multiple VOs may use) StorageShare. This can be represented (equivalently) as either: a single MappingPolicy object pointing to multiple UserDomain policies or multiple MappingPolicy objects, each pointing to the same StorageShare object. Each MappingPolicy object points to a single UserDomain object, but each of these UserDomain objects is different. (In fact, these two options are two extremes: one could publish a mixture of these) Aside: do we *really* need this flexibility? It seems to create unnecessary complication for the clients. We can remove it by changing either the Share: MappingPolicy.ID multiplicity to "optional" (0..1) or (Mapping)Policy: UserDomain.ID multiplicity to "required" (1). The former seems the better option (IMHO). We identify which StorageShares are to be used for accounting by selecting only those StorageShares that are published with a SharingID attribute value of "dedicated". The metric values to be used for storage accounting are the attached StorageShareCapacity objects. It might be worth adding another optional attribute, to allow accounting for (temporarily?) off-line storage. For example, adding: UnavaliableSize UInt64 0..1 GB Size of storage extent that is currently unavailable. The WLCG InstalledCapacity would then be: InstalledCapacity = TotalSize + UnavailableSize Cheers, Paul.
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Paul Millar said: MappingPolicy objects that act as links between StorageShare objects and UserDomain objects.
One point I'd make is that the UserDomain objects are a bit of a red herring in all this, when thinking about authorisation it's better just to focus on the Policy objects. Policies will generally refer to VOs and the UDs can publish information about VOs, but that's pretty much irrelevant to using the Policies. The main reason for having multiple Policy objects attached to the same Share is that they could declare policies in different authz *schemes*. For a single scheme (e.g. I expect to have a "glite" scheme for the EGEE standard use with VOMS) you would generally only expect a single policy object. In the CE discussion I raised the possibility of having multiple MappingPolicies within a single scheme to give the possibility to express an ordered priority between the Rules, since Rules within an object have no ordering, but I think that was rejected. So I think the answer is that for the normal case, e.g. an SE within EGEE/LCG, you will just have a single MP with a set of authz rules (unordered) looking much as it does now, e.g. VO: cms VOMS: /atlas/Role=production ...
complication for the clients. We can remove it by changing either the Share: MappingPolicy.ID multiplicity to "optional" (0..1)
No, it needs to be * to support multiple schemes, but in general a client would only understand one scheme so it only needs to consider (at most) one of the MP objects. I think we explicitly don't define the expected behaviour if a client can understand more than one scheme. Technically you could publish multiple objects with the same scheme, but in the absence of extra rules it would be semantically identical to having a single object, and probably the scheme definitions should normally say explicitly that there should only be one object.
(Mapping)Policy: UserDomain.ID multiplicity to "required" (1).
There is no guarantee that UD objects are published at all, and in EGEE/LCG my expectation is that they won't be.
It might be worth adding another optional attribute, to allow accounting for (temporarily?) off-line storage. For example, adding:
UnavaliableSize UInt64 0..1 GB Size of storage extent that is
Yes, I think we need to add something new to allow for this distinction between "installed" and "currently available", but we should probably have a dedicated phone meeting to discuss what's needed. Stephen -- Scanned by iCritical.
Hi Stephen, On Friday 19 December 2008 12:46:56 Burke, S (Stephen) wrote:
MappingPolicy objects that act as links between StorageShare objects and UserDomain objects [...]
One point I'd make is that the UserDomain objects are a bit of a red herring in all this, when thinking about authorisation it's better just to focus on the Policy objects.
Yes, publishing a MappingPolicy object certainly a minimum requirement and the "easy bit" :-)
Policies will generally refer to VOs and the UDs can publish information about VOs, but that's pretty much irrelevant to using the Policies.
Sure, one can publish a Policy object without linking it to a UD, but that may place additional burden on clients when they attempt to do service selection. Policy objects, in general, represent some form of access control that allows end-users it. Without linking the Policy object to a UserDomain, one forces the GLUE client to understand the authorisation schema to decide whether members of the UG are allowed to access it.
The main reason for having multiple Policy objects attached to the same Share is that they could declare policies in different authz *schemes*.
Ah, that makes sense.
For a single scheme (e.g. I expect to have a "glite" scheme for the EGEE standard use with VOMS) you would generally only expect a single policy object. In the CE discussion I raised the possibility of having multiple MappingPolicies within a single scheme to give the possibility to express an ordered priority between the Rules, since Rules within an object have no ordering, but I think that was rejected. So I think the answer is that for the normal case, e.g. an SE within EGEE/LCG, you will just have a single MP with a set of authz rules (unordered) looking much as it does now, e.g.
VO: cms VOMS: /atlas/Role=production
It would be nice to have this in the document. Perhaps something like: Multiple MappingPolicy objects MAY refer to the same Share object. If so, these MappingPolicy objects SHOULD have different authorisation schemata.
complication for the clients. We can remove it by changing either the Share: MappingPolicy.ID multiplicity to "optional" (0..1)
No, it needs to be * to support multiple schemes, but in general a client would only understand one scheme so it only needs to consider (at most) one of the MP objects. I think we explicitly don't define the expected behaviour if a client can understand more than one scheme.
OK.
Technically you could publish multiple objects with the same scheme, but in the absence of extra rules it would be semantically identical to having a single object, and probably the scheme definitions should normally say explicitly that there should only be one object.
Aye, see my suggestion above.
(Mapping)Policy: UserDomain.ID multiplicity to "required" (1).
There is no guarantee that UD objects are published at all, and in EGEE/LCG my expectation is that they won't be.
Ah, OK ... but see my other email and the (open) question of doing service selection without the client understanding the authorisation schema.
It might be worth adding another optional attribute, to allow accounting for (temporarily?) off-line storage. For example, adding:
UnavaliableSize UInt64 0..1 GB Size of storage extent that is [..]
Yes, I think we need to add something new to allow for this distinction between "installed" and "currently available", but we should probably have a dedicated phone meeting to discuss what's needed.
Yup. Cheers, Paul.
Paul Millar [mailto:paul.millar@desy.de] said:
Without linking the Policy object to a UserDomain, one forces the GLUE client to understand the authorisation schema to decide whether members of the UG are allowed to access it.
It's certainly true that the client has to understand the authz scheme, but that's true regardless, it has nothing to do with the UDs, and the UDs are unlikely to offer any help to a client in interpreting authz rules.
Multiple MappingPolicy objects MAY refer to the same Share object. If so, these MappingPolicy objects SHOULD have different authorisation schemata.
I'm not sure if we can make it that strong because I have no idea what other authz schemes might look like! Basically it would be up to any grid/community devising a scheme to make sure that what it did was consistent and workable. Stephen -- Scanned by iCritical.
participants (2)
-
Burke, S (Stephen) -
Paul Millar