Hi all, As an exercise, I've tried to jot down as precise and complete a description of each GLUE storage object as possible, also describing and how they relate to each other. I've also tried to do this without any forward-references (so, in theory, the document is readable in a single pass). In almost all cases, I've left out the attributes. I don't know how useful this is. It's just my point-of-view of things as stand now. I'm sure there are bits that are "wrong" (either I've misunderstood and/or this description breaks a use-case), but if so, helpfully people can point which bits are wrong and (perhaps) it will stimulate some discussion. BTW, I'm implicitly assuming that StorageEnvironment.RetentionPolicy can be multivalued. If this isn't true and we have the use-case of the same physical disks being part of, for example, both Custodial and Output storage, then it starts to get complicated. As always, comments appreciated. Cheers, Paul. --- UserDomain: A collection of one or more end-users; a VO is an instance of a UserDomain. All end-users that interact with the physical storage are a member of a UserDomain and, in general, derive their authorisation from that membership. StorageCapacity: A StorageCapacity object describes the ability to store data within a homogeneous storage technology. This storage technology provides a common access latency. All StorageCapacity objects are specified within a certain context. The context is determined by an association between the StorageCapacity object and precisely one other higher-level object. These associations are not listed here, but are described in later sections. In general, a StorageCapacity object will record some context-specific information. Examples of such information include the total storage capacity of the underlying technology and how much of that total has been used. The underlying storage technology may affect which of the context-specific attributes are available. For example, tape storage may be considered semi-infinite, so the total and free attributes have no meaning. If this is so, then it affects all StorageCapacity objects with the same underlying technology, independent of their context. Different contexts may also affect what context-specific attributes are recorded. This is a policy decision when implementing GLUE, as recording all possible information may be costly and provide no great benefit. [Aside: these two reasons are why many of the attributes within StorageCapacity are optional. Rather than explicitly subclassing the objects and making the values required, it is left deliberately vague which attributes are published.] A StorageCapacity may represent a logical aggregation of multiple underlying storage technology instances; for example, a StorageCapacity might represent many disk storage nodes, or many tapes stored within a tape silo. GLUE makes no effort to record information at this deeper level; but by not doing so, it requires that the underlying storage technology be homogeneous. Homogeneous means that the underlying storage technology is either identical or sufficiently similar that the differences don't matter. In most cases, the homogeneity is fairly obvious (e.g., tape storage vs disk-based storage), but there may be times where this distinction becomes contentious and judgement may be required; for example, the quality of disk-base storage might indicate that one subset is useful for a higher-quality service. If this is so, then it may make sense to represent the different class of disk by different SpaceCapacities. StorageEnvironment: A StorageEnvironment is a collection of one or more StorageCapacities with a set of associated (enforced) storage management policies. Examples of these policies are Type (Volatile, Durable, Permanent) and RetentionPolicy (Custodial, Output, Replica). StorageEnvironments act as a logical aggregation of StorageCapacities, so each StorageEnvironment must have at least one associated StorageCapacity. It is the associated StorageCapacities that allow a StorageEnvironment to store data with its advertised policies; for example, to act as (Permanent, Custodial) storage of data. Since a StorageEnvironment may contain multiple StorageCapacities, it may describe a heterogeneous environment. An example of this is "tape storage", which has both tape back-end and disk front-end into which users can pin files. Such a StorageEnvironment would have two associated StorageCapacities: one describing the disk storage and another describing the tape. If a StorageCapacity is associated with a StorageEnvironment, it is associated with only one. A StorageCapacity may not be shared between different StorageEnvironments. StorageCapacities associated with a StorageEnvironment must be non-overlapping with any other such StorageCapacity and the set of all such StorageCapacities must represent the complete storage available to end-users. Each physical storage device (e.g., individual disk drive or tape) that an end-user can utilise must be represented by (some part of) precisely one StorageCapacity associated with a StorageEnvironment. Nevertheless, the StorageCapacities associated with StorageEnvironments may be incomplete as a site may deploy physical storage devices that are not directly under end-user control; for example, disk storage used to cache incoming transfers. GLUE makes no effort to record information about such storage. StorageResource: A StorageResource is an aggregation of one or more StorageEnvironments and describes the hardware that a particular software instance has under its control. A StorageResource must have at least one StorageEnvironment, otherwise there wouldn't be much point publishing information about it. [This isn't a strict requirement, but I think it makes sense to include it.] All StorageEnvironments must be part of precisely one StorageResource. SoftwareEnvironments may not be shared between StorageResources. This means that all physics hardware must be published under precisely one StorageResource. StorageShare: A StorageShare is a logical partitioning of one or more StorageEnvironments. Perhaps the simplest example of a StorageShare is one associated with a single StorageEnvironment with a single associated StorageCapacity, and that represents all the available storage of that StorageCapacity. An example of a storage that could be represented by this trivial StorageShare is the classic-SE. StorageSpaces must have one or more associated StorageCapacities. These StorageCapacities provide a complete description of the different homogeneous underlying technologies that are available under the space. In general, the number of StorageCapacities associated with a StorageShare is the sum of the number of StorageCapacities associated with each of the StorageShare's associated StorageEnvironments. Following from this, there is an implicit association between the StorageCapacity associated with a StorageShare and the corresponding StorageCapacity associated with a StorageEnvironment. Intuitively, this association is from the fact that the two StorageCapacities share the same underlying physical storage. This implicit association is not recorded in GLUE. StorageSpaces may overlap. Specifically, given a StorageCapacity (SC_E) that is associated with some StorageEnvironment and which has totalSize TS_E, let the sum of the totalSize attributes for all StorageCapacities that are: 1. associated with a StorageSpace, and 2. that are implicitly associated with SC_E be TS_S. If the StorageSpaces are covering then TS_S = TS_E. If the StorageSpaces overlap, then TS_S > TS_E. [sorry, I couldn't easily describe this with just words without it sounding awful!] StorageSpaces may be incomplete. Following the same definitions as above, this is when TS_S < TS_E. Intuitively, this happens if the site-admin has not yet assigned all available storage. End-users within a UserDomain may wish to store or retrieve files. The StorageShares provides a complete, abstract description of the underlying storage at their disposal. No member of a UserDomain may interact with the physical hardware except through a StorageShare. The partitioning is persistent through file creation and deletion. The totalSize attributes (of a StorageSpace's associated StorageCapacties) do not change as a result of file creation or deletion. [Does GLUE need to stipulate this, or should we leave this vague?] A single StorageShare may allow multiple UserDomains to access storage; if so, the StorageShare is "shared" between the different UserDomains. Such a shared StorageShare is typical if a site provides storage described by the trivial StorageShare (one that covers a complete StorageEnvironment) whilst supporting multiple UserDomains. StorageMappingPolicy: The StorageMappingPolicy describes how a particular UserDomain is allowed to access a particular StorageShare. No member of a UserDomain may interact with a StorageShare except as described by a StorageMappingPolicy. The StorageMappingPolicies may contain information that is specific to that UserDomain, such as one or more associated StorageCapacities. If provided, these provide a UserDomain-specific view of their usage of the underlying physical storage technology as a result of their usage within the StorageShare. If StorageCapacities are associated with a StorageMappingPolicy, there will be the same number as are associated with the corresponding StorageShare. StorageEndpoint: A StorageEndpoint specifies that storage may be controlled through a particular interface. The SRM protocol is an example of such an interface and a StorageEndpoint would be advertised for each instance of SRM. The access policies describing which users of a UserDomain may use the StorageEndpoint are not published. On observing that a site publishes a StorageEndpoint, one may deduce only that it is valid for at least one user of one supported UserDomain. StorageAccessProtocol: A StorageAccessProtocol describes how data may be sent or received. The presence of a StorageAccessProtocol indicates that data may be fetched or stored using this interface. Access to the interface may be localised; that is, only available from certain computers. It may also be restricted to specified UserDomains. However, neither policy restrictions are published in GLUE. On observing a StorageAccessProcol, one may deduce only that it is valid for at least one user of one supported UserDomain. StorageService: A StorageService is an aggregation of StorageEndpoints, StorageAccessProtocols and StorageResources. It is the top-level description of the ability to transfer files to and from a site, and manipulate the files once stored.