Hi all, the SRM v2.2 docs are here: http://sdm.lbl.gov/srm-wg/ Quoting from that, with adjustments: --------------------------------------------------------------- The Retention Policy represents the quality of retention, which indicates the probability of the storage system losing a file. Replica quality has the highest probability of loss, but is appropriate for data for which a certain amount of loss can be tolerated, in particular when other copies can be accessed in a timely fashion. Output quality is an intermediate level and is appropriate for data which can be replaced by lengthy or effort-full processes. Custodial quality corresponds to low probability of loss. --------------------------------------------------------------- Thanks, Maarten
HI Maarten, thanks. Can you send also the same info for: ExpirationMode and its values AccessLatency and its values Thanks, Sergio Maarten Litmaath wrote:
Hi all, the SRM v2.2 docs are here:
Quoting from that, with adjustments:
--------------------------------------------------------------- The Retention Policy represents the quality of retention, which indicates the probability of the storage system losing a file.
Replica quality has the highest probability of loss, but is appropriate for data for which a certain amount of loss can be tolerated, in particular when other copies can be accessed in a timely fashion.
Output quality is an intermediate level and is appropriate for data which can be replaced by lengthy or effort-full processes.
Custodial quality corresponds to low probability of loss. ---------------------------------------------------------------
Thanks, Maarten _______________________________________________ glue-wg mailing list glue-wg@ogf.org http://www.ogf.org/mailman/listinfo/glue-wg
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Sergio Andreozzi said: thanks. Can you send also the same info for:
ExpirationMode and its values AccessLatency and its values
While we're at it, should we have anything to describe pinning, e.g. MaxPinDuration? Or are there any other SRM parameters that could/should be exposed? Stephen
On Mon, 14 Apr 2008, Burke, S (Stephen) wrote:
While we're at it, should we have anything to describe pinning, e.g. MaxPinDuration? Or are there any other SRM parameters that could/should be exposed?
What would be a use case for MaxPinDuration? An SRM client can ask for a particular time that may then be shortened by the server, possibly depending on who the client is: a production manager might be allowed to ask for more than an ordinary user. I did not see any other SRM parameters, but I think we need to resurrect MaxFileSize and probably MinFileSize as well. In the StorageService?
Maarten.Litmaath@cern.ch [mailto:Maarten.Litmaath@cern.ch] said:
What would be a use case for MaxPinDuration?
You may know that you will want to pin files for, say, 1 week, so you could exclude SEs which don't allow that. However, if it can depend on the user then maybe that isn't so useful. Could it still be useful to have a boolean to say that pinning is supported, or do we expect that all SEs will support it?
I did not see any other SRM parameters, but I think we need to resurrect MaxFileSize and probably MinFileSize as well. In the StorageService?
Are they set for the SRM as a whole? Stephen
On Tue, 15 Apr 2008, Burke, S (Stephen) wrote:
Maarten.Litmaath@cern.ch [mailto:Maarten.Litmaath@cern.ch] said:
What would be a use case for MaxPinDuration?
You may know that you will want to pin files for, say, 1 week, so you could exclude SEs which don't allow that. However, if it can depend on
I see that more as a negotiation between the VO and the SE admin: if the SE remains unusable, the VO will just blacklist it.
the user then maybe that isn't so useful. Could it still be useful to have a boolean to say that pinning is supported, or do we expect that all SEs will support it?
Well, the Classic SE does not support it, and it may have a part of its name space going to tape. But do we care? Furthermore, in SRM v2.2 a pin may be taken as advisory: when CASTOR needs room for other files, it will remove pinned files that are unused, while dCache currently will honor the pins instead.
I did not see any other SRM parameters, but I think we need to resurrect MaxFileSize and probably MinFileSize as well. In the StorageService?
Are they set for the SRM as a whole?
Quite likely. In principle it might be using different file systems with different maximum file sizes, but I doubt anyone will ever do that. The minimum file size might also depend on the tape library (CERN indeed has 2 instances with different minimum sizes), but that is for the SRM to sort out: the user still has a single minimum file size.
Maarten.Litmaath@cern.ch [mailto:Maarten.Litmaath@cern.ch] said:
I see that more as a negotiation between the VO and the SE admin: if the SE remains unusable, the VO will just blacklist it.
That's possibly a rather LCG-centric view of the world where VOs can dictate what sites do, it may well not be true for the majority of VOs! In general I would say that static blacklists are a symptom of a failure of the GLUE schema, if service discovery worked properly they shouldn't be needed.
Well, the Classic SE does not support it, and it may have a part of its name space going to tape. But do we care?
No, because you know a priori that classic SEs can't support pinning, but for SRMs it may be variable.
Furthermore, in SRM v2.2 a pin may be taken as advisory: when CASTOR needs room for other files, it will remove pinned files that are unused, while dCache currently will honor the pins instead.
The way I remember it is that the SRM spec does not allow pins to be advisory (and the users don't want it to), but the castor developers have so far refused to implement pinning according to the spec! Anyway, this could also be taken as something that should be advertised, unless you are just supposed to "know" that castor behaves differently. Stephen
Burke, S (Stephen) wrote:
I see that more as a negotiation between the VO and the SE admin: if the SE remains unusable, the VO will just blacklist it.
That's possibly a rather LCG-centric view of the world where VOs can dictate what sites do, it may well not be true for the majority of VOs! In general I would say that static blacklists are a symptom of a failure of the GLUE schema, if service discovery worked properly they shouldn't be needed.
We may be talking about different things. Some LCG VOs may be able to dictate what some sites have to do, but that is not true in general. My point is that any site can claim they offer a service to a VO; if that service turns out to remain unusable after negotiation, the VO will _have_ to blacklist it somehow, to avoid that it keeps getting discovered and tried!
Well, the Classic SE does not support it, and it may have a part of its name space going to tape. But do we care?
No, because you know a priori that classic SEs can't support pinning, but for SRMs it may be variable.
Furthermore, in SRM v2.2 a pin may be taken as advisory: when CASTOR needs room for other files, it will remove pinned files that are unused, while dCache currently will honor the pins instead.
The way I remember it is that the SRM spec does not allow pins to be advisory (and the users don't want it to), but the castor developers have so far refused to implement pinning according to the spec! Anyway, this could also be taken as something that should be advertised, unless you are just supposed to "know" that castor behaves differently.
Agreed, this ought to be advertized, but I do not think it is important enough for consideration at this stage, given the deadlines...
Maarten Litmaath [mailto:Maarten.Litmaath@cern.ch] said:
My point is that any site can claim they offer a service to a VO; if that service turns out to remain unusable after negotiation, the VO will _have_ to blacklist it somehow, to avoid that it keeps getting discovered and tried!
Yes, that's obviously true, but if the reason for blacklisting is a feature (or missing feature) of the service which is not being advertised then that would be a flag to me that the schema should be enhanced to publish it.
Agreed, this ought to be advertized, but I do not think it is important enough for consideration at this stage, given the deadlines...
Perhaps so, but I'm always unhappy about stopping something purely to meet some arbitrary deadline. In the past we regularly released buggy software to hit a deadline and it gave us a bad reputation. Realistically glue 2 will probably not be in use before 2010, and anything which is wrong or missing will not be fixed before maybe 2013, if ever (and anything which would be a structural change rather than just a new attribute is almost certainly "never"), so from my point of view we should try do do the best job we can now. Stephen
Ciao Sergio,
ExpirationMode and its values
---------------------------------------------------------------------- The Expiration Modes of a container indicate if it supports files with infinite and/or finite lifetimes, and what actions the storage system may take upon the expiration of a file. The value NeverExpire indicates support for files with infinite lifetime: they can only be removed by authorized clients, not by the storage system itself. The value WarnWhenExpired indicates support for files that have finite lifetimes, but on expiration cannot be removed by the storage system itself. The data content of an expired file may be deleted if it can be recovered from an archive. New store operations may fail for certain clients until (some of the) expired files have either been removed by authorized clients, or have had their lifetimes increased. The value ReleaseWhenExpired indicates support for files that have finite lifetimes and on expiration will be removed by the storage system. ----------------------------------------------------------------------
AccessLatency and its values
---------------------------------------------------------------------- The Access Latency of a container indicates the maximum latency for a file stored in that container to be made available for reading. The latency ONLINE indicates that files are always on disk, hence cannot have their latency improved. The latency NEARLINE indicates that a file may have its only copies in a "nearly online" component of the storage system, typically a fully automated tape robot, but also a remote storage system could fit this qualification. Such a facility will need an unspecified amount of time to make a copy of the file available on the disk component of the container under consideration. When a file is not in use, its disk copies may be removed. Hence the system cannot guarantee that a file will be immediately available on disk. The latency OFFLINE indicates that a file may have its only copies in an offline component of the storage system, for example a tape library that is not connected to an automated tape robot. Hence an operator intervention may be needed to make a copy of a file available that has a lower latency. ----------------------------------------------------------------------
thanks. Added in draft 37: http://forge.ogf.org/sf/go/projects.glue-wg/docman.root.drafts - Sergio Maarten.Litmaath@cern.ch wrote:
ExpirationMode and its values
---------------------------------------------------------------------- The Expiration Modes of a container indicate if it supports files with infinite and/or finite lifetimes, and what actions the storage system may take upon the expiration of a file.
The value NeverExpire indicates support for files with infinite lifetime: they can only be removed by authorized clients, not by the storage system itself.
The value WarnWhenExpired indicates support for files that have finite lifetimes, but on expiration cannot be removed by the storage system itself. The data content of an expired file may be deleted if it can be recovered from an archive. New store operations may fail for certain clients until (some of the) expired files have either been removed by authorized clients, or have had their lifetimes increased.
The value ReleaseWhenExpired indicates support for files that have finite lifetimes and on expiration will be removed by the storage system.
----------------------------------------------------------------------
AccessLatency and its values
---------------------------------------------------------------------- The Access Latency of a container indicates the maximum latency for a file stored in that container to be made available for reading.
The latency ONLINE indicates that files are always on disk, hence cannot have their latency improved.
The latency NEARLINE indicates that a file may have its only copies in a "nearly online" component of the storage system, typically a fully automated tape robot, but also a remote storage system could fit this qualification. Such a facility will need an unspecified amount of time to make a copy of the file available on the disk component of the container under consideration. When a file is not in use, its disk copies may be removed. Hence the system cannot guarantee that a file will be immediately available on disk.
The latency OFFLINE indicates that a file may have its only copies in an offline component of the storage system, for example a tape library that is not connected to an automated tape robot. Hence an operator intervention may be needed to make a copy of a file available that has a lower latency. ----------------------------------------------------------------------
-- Sergio Andreozzi INFN-CNAF, Tel: +39 051 609 2860 Viale Berti Pichat, 6/2 Fax: +39 051 609 2746 40126 Bologna (Italy) Web: http://www.cnaf.infn.it/~andreozzi
participants (4)
-
Burke, S (Stephen) -
Maarten Litmaath -
Maarten.Litmaath＠cern.ch
-
Sergio Andreozzi