Hi Maria, On 25/09/14 09:04, Maria Alandes Pradillo wrote:
If we need to write GLUE 2.1 info providers this is not in my opinion a backwards compatible change.
No, please re-read my explanation: nobody needs to write a GLUE 2.1 info-provider. Some groups (like dCache) may *choose* to write a GLUE 2.1 info-provider to better describe the storage; however, existing (GLUE 2.0) info-providers will continue to work fine.
Could you please explain why suddenly this is an issue,
I *think* what has happened is this: dCache published some Storage Shares. Broadly speaking, there are two types: ones that show SRM space-reservations and ones that show physical hardware. The Storage Shares representing SRM space-reservations are correct and easily selected as they have a non-empty Tag attribute. This satisfies use-cases, like ATLAS, who do space accounting through SRM space-reservations. The Storage Shares representing physical hardware were intended for "installed capacity" calculations. I know that, for at least one Tier-1, the installed capacity numbers are provided as a spreadsheet and are calculated manually, NOT from information published by GLUE. I suspect this is true for other sites, but I could be wrong here. The problem is that, for dCache, the Storage Shares representing SRM space-reservations and those representing physical hardware are two views on the same storage which overlap. The Storage Shares in the "SRM space-reservations" view do not overlap. Similarly, the Storage Shares in the "physical hardware" view do not overlap. In GLUE-2.0, there is no way to represent this. In GLUE-2.0, pairs of Storage Shares either overlap or they are non-overlapping.
who has requested this use case,
The ultimate use-case is for calculating the installed capacity, which is a long established use-case in WLCG. I believe introducing an extra attribute is needed to provide the necessary information for a dCache instance.
how many VOs or users are affected otherwise
I believe all WLCG VOs that take part in the installed capacity calculations are affected.
and why it can't be addressed with the existing GLUE 2.0 schema?
Because GLUE 2.0 provides only the SharingID attribute, which describes whether or not two Storage Shares overlap. This is not a rich enough language to describe a dCache instance. (SharingID may have some other problems, but those we can work-around.)
Before proposing the change to the GLUE WG, I think we need further discussion also among the rest of the Storage developers.
I thought the GLUE WG was the correct place to discuss a proposed change to GLUE.
What happens with DPM or StoRM?
They need do nothing: the proposed change is backwards compatible for GLUE 2.0 info-providers.
If they don't publish this attribute, will those sites or users who have requested that use case be happy?
If they (DPM and StoRM info-providers) don't publish this attribute, sites and users will continue to be as happy as they are now.
Will they target only dCache installations?
I'm not sure what you mean by "they" and "target"; however, the proposal was carefully crafted to be backwards compatible for GLUE 2.0 info-providers. The overall effect is to unify how storage systems are represented by allowing a client to discover information about any kind of storage system.
I would like all Storage systems to be aligned.
Me too. Currently they are not. I would like to change this; hence the proposal.
I don't like the idea that we introduce an optional attribute that will be mandatory for dCache but not for the others.
Storage systems are different and behave differently. Publishing different subset of attributes is a natural consequence of this. The point here is that GLUE 2.0 is incapable of describing a dCache site in such a way that the installed capacity may be calculated from the published data. GLUE 2.0 is "broken" in the sense that it cannot describe a dCache site such that the installed capacity may be calculated. I would like to "fix" GLUE 2.0 so an info-provider can describe a dCache instance.
And why do you say that storage accounting is broken in dCache only now?
Simple: because I didn't realise, until now, that is was broken.
This is the first time I hear this, when I have spent the whole summer trying to improve storage info providers and the feedback I got from dCache is that everything was OK for GLUE 2 publishing.
Sure, dCache publishes some wonderful GLUE 2.0 information. One can do many things with that information. However, it turns out that calculating the installed-capacity isn't one of them. Like many things, it's only when there are users that the problems are found. However, please view this as a normal bug-fixing process: a problem has been discovered. We are in the process of understanding it. A solution has be proposed, which will be discussed based on its merits. Cheers, Paul.