Paul Millar wrote:

Hi all,

Hi Paul, *, [..]

I don't know how useful this is. It's just my point-of-view of things as stand now. I'm sure there are bits that are "wrong" (either I've misunderstood and/or this description breaks a use-case), but if so, helpfully people can point which bits are wrong and (perhaps) it will stimulate some discussion.

IMO it is useful. We know from past experiences that different communities interpret different things, er, differently. Sometimes that is useful - makes schema reusable and adaptable, but as a service provider publishing the same attributes with different semantics is a nightmare. I think it's an excellent attempt and I found it useful (just took me some time before I had time to parse it :-) [..]

UserDomain:

A collection of one or more end-users; a VO is an instance of a UserDomain. All end-users that interact with the physical storage are a member of a UserDomain and, in general, derive their authorisation from that membership.

If we do it like this then we should use the hierarchical feature in UserDomain: UserDomain : WLGC UserDomain : NGS UserDomain : Diamond | | | UserDomain : LHCb UserDomain : biomed UserDomain : Pr234 | | | UserDomain : prod UserDomain : NHS UserDomain : Beamline In general, I think the high level entity is more useful than a low level entity such as the VO. Within WLCG for example, most SE info is available to all VOs in the sense that they (SEs) publish information that the VOs know how to make sense of. Outside WLCG (yes, there are people not in WLCG), the same schema could be used but e.g. with different attributes published (e.g. some req'd by WLCG could be left blank for NGS). The UserDomain could be consulted by the information consumer to check whether they should even probe further.

StorageCapacity:

A StorageCapacity object describes the ability to store data within a homogeneous storage technology. This storage technology provides a common access latency.

We used to call this a StorageComponent. http://storage.esc.rl.ac.uk/GLUE-SE-1.3-input-1.03.pdf It was debated when we discussed 1.3 whether it was even useful to expose this level of detail to users via the information system, but I would suggest in the interim we have some users who do care.

All StorageCapacity objects are specified within a certain context. The context is determined by an association between the StorageCapacity object and precisely one other higher-level object. These associations are not listed here, but are described in later sections.

What if I have an

In general, a StorageCapacity object will record some context-specific information. Examples of such information include the total storage capacity of the underlying technology and how much of that total has been used.

The underlying storage technology may affect which of the context-specific attributes are available. For example, tape storage may be considered semi-infinite, so the total and free attributes have no meaning. If this is so, then it affects all StorageCapacity objects with the same underlying technology, independent of their context.

That is too prescriptive (OK, it was just an example). For another example, some customers who pay directly for media used, they will want to know how much space is available on the tapes. Better just to leave the space thingies as optional attributes. In any case, we know the concepts of "free" and "used" (etc) are so difficult to pin down that each User Domain may well have its own interpretations. Which is why the UserDomain attr could be handy.

Different contexts may also affect what context-specific attributes are recorded. This is a policy decision when implementing GLUE, as recording all possible information may be costly and provide no great benefit.

Mm hm. Which I think is why the User Domain is a good thing if we can use it for that.

[Aside: these two reasons are why many of the attributes within StorageCapacity are optional. Rather than explicitly subclassing the objects and making the values required, it is left deliberately vague which attributes are published.]

Yep, domains will define what they want and what it means.

A StorageCapacity may represent a logical aggregation of multiple underlying storage technology instances; for example, a StorageCapacity might represent many disk storage nodes, or many tapes stored within a tape silo. GLUE makes no effort to record information at this deeper level; but by not doing so, it requires that the underlying storage technology be homogeneous. Homogeneous means that the underlying storage technology is either identical or sufficiently similar that the differences don't matter.

All that is really required is homogeneity in attributes. For example, a community that does not care about AccessLatency being published (if one such exists) would see disk and tape as homogeneous.

In most cases, the homogeneity is fairly obvious (e.g., tape storage vs disk-based storage), but there may be times where this distinction becomes contentious and judgement may be required; for example, the quality of disk-base storage might indicate that one subset is useful for a higher-quality service. If this is so, then it may make sense to represent the different class of disk by different SpaceCapacities.

Yes, but the GLUE schema should not prescribe how to do this - provide attributes for communities to publish the most common capabilities but leave it to the communities to define what they mean. GLUE could include examples but they must not be normative.

StorageEnvironment:

A StorageEnvironment is a collection of one or more StorageCapacities with a set of associated (enforced) storage management policies. Examples of these policies are Type (Volatile, Durable, Permanent) and RetentionPolicy (Custodial, Output, Replica).

I completely agree with Maarten here, we need to get away from the old overloaded names (even as examples! unless we put in in big fat letters that their use is deprecated). ExpirationMode : releaseWhenExpired, warnWhenExpired, neverExpire. RetentionPolicy : Replica, Output, Custodial. Note that these come from the SRM world (it is a bug in SRM2.2 that the old volatile etc names were retained). These names may again be meaningless to other communities, who may still wish to publish values for these. It may be better to provide attributes for ExpirationMode and RetentionPolicy (etc) and explain what they're for, but not to prescribe the values.

StorageEnvironments act as a logical aggregation of StorageCapacities, so each StorageEnvironment must have at least one associated StorageCapacity. It is the associated StorageCapacities that allow a StorageEnvironment to store data with its advertised policies; for example, to act as (Permanent, Custodial) storage of data.

OK

Since a StorageEnvironment may contain multiple StorageCapacities, it may describe a heterogeneous environment. An example of this is "tape storage", which has both tape back-end and disk front-end into which users can pin files. Such a StorageEnvironment would have two associated StorageCapacities: one describing the disk storage and another describing the tape.

We've had this case before. In that case, the StorageEnvironment should publish the minimal capabilities which it can support (or leave them blank, leaving it to the client to go and query the StorageCapacities). For example, if a StorageEnvironment contains both tape and disk, its AccessLatency should be Nearline - as it is the lowest common denominator. This only makes sense if you have a partial order on capabilities.

If a StorageCapacity is associated with a StorageEnvironment, it is associated with only one. A StorageCapacity may not be shared between different StorageEnvironments.

OK

StorageCapacities associated with a StorageEnvironment must be non-overlapping with any other such StorageCapacity and the set of all such StorageCapacities must represent the complete storage available to end-users. Each physical storage device (e.g., individual disk drive or tape) that an end-user can utilise must be represented by (some part of) precisely one StorageCapacity associated with a StorageEnvironment.

OK, except we shouldn't say "it must represent the complete storage available to end users" - it's up to the information publisher. We may have "secret" storage available, via endpoints communicated by other means.

Nevertheless, the StorageCapacities associated with StorageEnvironments may be incomplete as a site may deploy physical storage devices that are not directly under end-user control; for example, disk storage used to cache incoming transfers. GLUE makes no effort to record information about such storage.

Of course.

StorageResource:

A StorageResource is an aggregation of one or more StorageEnvironments and describes the hardware that a particular software instance has under its control.

Ummm. I would avoid using the word "hardware" here, or at least I find it confusing. For example, at RAL, CASTOR does not have exclusive use of the tapestore.

A StorageResource must have at least one StorageEnvironment, otherwise there wouldn't be much point publishing information about it. [This isn't a strict requirement, but I think it makes sense to include it.]

I would be less concerned about publishing an empty StorageResource - if a top level BDII publishes the resource and gathers the Environments from lower level BDIIs, it may at some point find itself publishing an empty StorageResource, e.g. during maintenance.

All StorageEnvironments must be part of precisely one StorageResource. SoftwareEnvironments may not be shared between StorageResources. This means that all physics hardware must be published under precisely one StorageResource.

OK.

StorageShare:

A StorageShare is a logical partitioning of one or more StorageEnvironments.

OK.

Perhaps the simplest example of a StorageShare is one associated with a single StorageEnvironment with a single associated StorageCapacity, and that represents all the available storage of that StorageCapacity. An example of a storage that could be represented by this trivial StorageShare is the classic-SE.

StorageSpaces must have one or more associated StorageCapacities. These StorageCapacities provide a complete description of the different homogeneous underlying technologies that are available under the space.

OK.

In general, the number of StorageCapacities associated with a StorageShare is the sum of the number of StorageCapacities associated with each of the StorageShare's associated StorageEnvironments.

You sort of contradict yourself in this para in some of the paras below, so I have tried to summarise the conclusion: Observation [**] 1. StorageEnvironments fully partition the StorageCapacities [that is: each StorageCapacity belongs to one and only one StorageEnvironment] 2. Each StorageShare contains one or more StorageCapacities 3. A StorageShare is associated with a StorageEnvironment if and only if they contain a common StorageCapacity.

Following from this, there is an implicit association between the StorageCapacity associated with a StorageShare and the corresponding StorageCapacity associated with a StorageEnvironment. Intuitively, this association is from the fact that the two StorageCapacities share the same underlying physical storage. This implicit association is not recorded in GLUE.

Well except the StorageShare has a unique id. Also I understood there is a 1..* - 1..* association between the Share and the Environment. s/StorageSpace/StorageShare/, cf Maarten's email. (6 occurrences)

StorageShares may overlap. Specifically, given a StorageCapacity (SC_E) that is associated with some StorageEnvironment and which has totalSize TS_E, let the sum of the totalSize attributes for all StorageCapacities that are: 1. associated with a StorageShare, and 2. that are implicitly associated with SC_E be TS_S. If the StorageShares are covering then TS_S = TS_E. If the StorageShares overlap, then TS_S > TS_E. [sorry, I couldn't easily describe this with just words without it sounding awful!]

See [**] above Note 2 is still a cover - even if the StorageShares overlap they still _cover_ the set of all StorageCapacities. If $X$ is a space (e.g. topological space) then $U:=\{A_i\subseteq X\vert i\in I\}$ is said to be a cover of $X$ if $\union_{i\in I}A_i=X$.

StorageShares may be incomplete. Following the same definitions as above, this is when TS_S < TS_E. Intuitively, this happens if the site-admin has not yet assigned all available storage.

See [**] above.

End-users within a UserDomain may wish to store or retrieve files. The StorageShares provides a complete, abstract description of the underlying storage at their disposal. No member of a UserDomain may interact with the physical hardware except through a StorageShare.

In this case it makes sense to use hierarchical UserDomains.

The partitioning is persistent through file creation and deletion.

Which partitioning? The StorageShares do not partition anything in general.

The totalSize attributes (of a StorageShare's associated StorageCapacties) do not change as a result of file creation or deletion. [Does GLUE need to stipulate this, or should we leave this vague?]

This is actually not necessarily true: if I start adding files to the share, it may expand because the storage system chooses to add more Capacities to it.

A single StorageShare may allow multiple UserDomains to access storage; if so, the StorageShare is "shared" between the different UserDomains. Such a shared StorageShare is typical if a site provides storage described by the trivial StorageShare (one that covers a complete StorageEnvironment) whilst supporting multiple UserDomains.

This is getting complicated if the UserDomains themselves are hierarchical!

StorageMappingPolicy:

The StorageMappingPolicy describes how a particular UserDomain is allowed to access a particular StorageShare. No member of a UserDomain may interact with a StorageShare except as described by a StorageMappingPolicy.

This is also too prescriptive. Surely it is not up to GLUE to mandate rules for how storage systems are used. For example, I may publish a general read-only access rule for the VO, but a subset of the VO may have write access. I should not have to publish that explicitly.

The StorageMappingPolicies may contain information that is specific to that UserDomain, such as one or more associated StorageCapacities. If provided, these provide a UserDomain-specific view of their usage of the underlying physical storage technology as a result of their usage within the StorageShare.

I agree, this is consistent with how I think UserDomains should be used.

If StorageCapacities are associated with a StorageMappingPolicy, there will be the same number as are associated with the corresponding StorageShare.

This probably needs more careful checking. For example, a StorageShare can be contained in more than one StorageShare and those StorageShares can themselves be associated with different UserDomains.

StorageEndpoint:

A StorageEndpoint specifies that storage may be controlled through a particular interface. The SRM protocol is an example of such an interface and a StorageEndpoint would be advertised for each instance of SRM.

Yep.

The access policies describing which users of a UserDomain may use the StorageEndpoint are not published. On observing that a site publishes a StorageEndpoint, one may deduce only that it is valid for at least one user of one supported UserDomain.

That should be OK - as long as the endpoints themselves are interpreted the same way by all users - which seems reasonable.

StorageAccessProtocol:

A StorageAccessProtocol describes how data may be sent or received. The presence of a StorageAccessProtocol indicates that data may be fetched or stored using this interface.

Yep.

Access to the interface may be localised; that is, only available from certain computers. It may also be restricted to specified UserDomains. However, neither policy restrictions are published in GLUE. On observing a StorageAccessProcol, one may deduce only that it is valid for at least one user of one supported UserDomain.

Where did the network description go? We used to have one. The idea is that certain protocols can be used only locally, or on certain networks. For example, a single StorageElement can have a range of GridFTP data movers on the WAN, a LAN protocol internally, and an OPN link which accepts UDP-based high speed data transfer like the astronomers use. If you are a local job you can ask it "do you support gridftp" and it would say yes, but you cannot necessarily access the GridFTP data movers from the worker nodes - and it would be less efficient than the LAN protocol. I think we need to put it back, and StorageAccessProtocol seems to me the more obvious location.

StorageService: A StorageService is an aggregation of StorageEndpoints, StorageAccessProtocols and StorageResources. It is the top-level description of the ability to transfer files to and from a site, and manipulate the files once stored.

OK. Cheers --jens