Hi all, I'm sure that I've missed something.. sorry for that. By tomorrow (or by Friday) I'll try to create an over-complicated scenario for StoRM and map it into the current Glue Schema for storage. Now, I've some doubts: A) StoRM can manage, at the same time, different VOs. Any VO could use a different set of access protocol. For example, the StorageShare-1 could supports 'FILE', 'RFIO' and 'GSIFTP' protocols, while the StorageShare-2 (used by a different VO) 'FILE' protocol only. So, I think that StorageAccessProtocol should be linked to StorageShare, not with StorageService. B) DataStore entity should model, in a synthetic and simple form, the concept of the storage hardware (If I've understood well!). I imagine a cluster file system (GPFS for example) installed on a storage area network. In this context, the DataStore is the storage area network plus GPFS. How can define DataStore to include the GPFS characteristics? Using OtherInfo property? Perhaps we can add another property to define the software managing the hardware. Bye, Riccardo -- ------------------------------------------------------------------------------- Riccardo Zappi Istituto Nazionale di Fisica Nucleare - CNAF Viale Berti Pichat, 6/2, 40127 BOLOGNA - ITALY Phone: +39-051-609-2868 Fax: +39-051-609-2746 e-mail: riccardo.zappi@cnaf.infn.it -------------------------------------------------------------------------------
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Riccardo Zappi said: A) StoRM can manage, at the same time, different VOs. Any VO could use a different set of access protocol.
The problem here is that the available protocols could potentially depend on many different things - VO, share, hardware, network connections, ... and it would be too complicated to allow all possible combinations. Do you think that this kind of thing is really likely in practice? If so, how would you expect it to work? In terms of the schema the easiest thing to represent is probably an ACL (AccessPolicy) on the protocol, so if it would really be a per-VO setting I would prefer to do that rather than having a link between protocol and Share - but still only if we expect it to happen in real deployments.
B) DataStore entity should model, in a synthetic and simple form, the concept of the storage hardware (If I've understood well!). I imagine a cluster file system (GPFS for example) installed on a storage area network. In this context, the DataStore is the storage area network plus GPFS. How can define DataStore to include the GPFS characteristics? Using OtherInfo property? Perhaps we can add another property to define the software managing the hardware.
This is exactly the role of the Resource - maybe the name isn't very good but the Resource is supposed to represent the management software. So you would have a Resource for GPFS, with an associated Datastore to represent the disk servers. Stephen
Hi Stephen, thanks for your comments. Burke, S (Stephen) wrote:
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Riccardo Zappi said: A) StoRM can manage, at the same time, different VOs. Any VO could use a different set of access protocol.
The problem here is that the available protocols could potentially depend on many different things - VO, share, hardware, network connections, ... and it would be too complicated to allow all possible combinations. Do you think that this kind of thing is really likely in practice? If so, how would you expect it to work? In terms of the schema the easiest thing to represent is probably an ACL (AccessPolicy) on the protocol, so if it would really be a per-VO setting I would prefer to do that rather than having a link between protocol and Share - but still only if we expect it to happen in real deployments. Yes, I posted the above comment because we have a real problem with Glue Schema 1.3.
I try to explain the problem: LHCb don't want RFIO protocol published for its storage areas. I suppose for compatibility reason.. but now it is not important. LHCb want to use FILE protocol for access and GSIFTP for data transfer only. Other supported VOs want RFIO protocol available in addition to FILE and GSIFTP protocols. If we publish RFIO, FILE, and GSIFTP for StoRM service, then all LHCb jobs will fail because they try to use RFIO protocol. Our temporary (I hope ;) ) solution is to use an ad-hoc StoRM installation (and configuration) for LHCb only. I think that this is the only one big limitation of Glue Schema v1.3. You are right about the complicated scenario of supported protocol. I do not know if the use of the AccessPolicy on protocol works. Suppose the scenario where LHCb wants RFIO protocol available for one StorageShare, while for another one LHCb wants only FILE protocol. How can we represent this situation with ACL?
B) DataStore entity should model, in a synthetic and simple form, the concept of the storage hardware (If I've understood well!). I imagine a cluster file system (GPFS for example) installed on a storage area network. In this context, the DataStore is the storage area network plus GPFS. How can define DataStore to include the GPFS characteristics? Using OtherInfo property? Perhaps we can add another property to define the software managing the hardware.
This is exactly the role of the Resource - maybe the name isn't very good but the Resource is supposed to represent the management software. So you would have a Resource for GPFS, with an associated Datastore to represent the disk servers.
In this case, we could define the StorageShare on StorageResource.. or not?
Stephen Riccardo
-- ------------------------------------------------------------------------------- Riccardo Zappi Istituto Nazionale di Fisica Nucleare - CNAF Viale Berti Pichat, 6/2, 40127 BOLOGNA - ITALY Phone: +39-051-609-2868 Fax: +39-051-609-2746 e-mail: riccardo.zappi@cnaf.infn.it -------------------------------------------------------------------------------
Riccardo Zappi wrote:
You are right about the complicated scenario of supported protocol. I do not know if the use of the AccessPolicy on protocol works. Suppose the scenario where LHCb wants RFIO protocol available for one StorageShare, while for another one LHCb wants only FILE protocol. How can we represent this situation with ACL?
we might want to add the association from AccessProtocol to StorageShare; what do people think about this?
is is exactly the role of the Resource - maybe the name isn't very good but the Resource is supposed to represent the management software. So you would have a Resource for GPFS, with an associated Datastore to represent the disk servers.
In this case, we could define the StorageShare on StorageResource.. or not?
also here... I think Riccardo is right. The association from StorageShare to DataStore should be changed from StorageShare to StorageResource... alternatively, we can add in parallel if people really want to discover the type of data store on which a storage share is configured. This will complicate the life of sysadmin who have to configure the providers, therefore, let us consider this issue carefully. Cheers, Sergio
Sergio Andreozzi [mailto:sergio.andreozzi@cnaf.infn.it] said:
we might want to add the association from AccessProtocol to StorageShare; what do people think about this?
I would like to know what the real use-case is first. If this is really something specific to a particular VO and site then it might be better to e.g. use a ShareCapability or ShareOtherInfo.
also here... I think Riccardo is right. The association from StorageShare to DataStore should be changed from StorageShare to StorageResource... alternatively, we can add in parallel if people really want to discover the type of data store on which a storage share is configured.
I think we should keep the Share - Datastore relation as you can't deduce it in general if you only have Share - Resource. Maybe we could add a Share - Resource relation as well - but do we need it explicitly in the model if it's implied by the existing relations? Stephen
Burke, S (Stephen) wrote:
also here... I think Riccardo is right. The association from StorageShare to DataStore should be changed from StorageShare to StorageResource... alternatively, we can add in parallel if people really want to discover the type of data store on which a storage share is
configured.
I think we should keep the Share - Datastore relation as you can't deduce it in general if you only have Share - Resource. Maybe we could add a Share - Resource relation as well - but do we need it explicitly in the model if it's implied by the existing relations?
in the current modeling, it cannot be implied because storageResource is associated to a dataStore via a many-to-many relationship. It would have been possible in this case: storageResource 1--* dataStore Cheers, Sergio
Sergio Andreozzi wrote:
I think we should keep the Share - Datastore relation as you can't deduce it in general if you only have Share - Resource. Maybe we could add a Share - Resource relation as well - but do we need it explicitly in the model if it's implied by the existing relations?
in the current modeling, it cannot be implied because storageResource is associated to a dataStore via a many-to-many relationship.
It would have been possible in this case: storageResource 1--* dataStore
I think the relationship must be "1--*" indeed.
Sergio Andreozzi [mailto:sergio.andreozzi@cnaf.infn.it] said:
in the current modeling, it cannot be implied because storageResource is associated to a dataStore via a many-to-many relationship.
It would have been possible in this case: storageResource 1--* dataStore
That seems to be a mistake, Datastore to Resource should indeed be many to 1. I don't think it could make sense for one piece of hardware to be managed by multiple systems. Stephen
Hi Stephen, Burke, S (Stephen) wrote:
Sergio Andreozzi [mailto:sergio.andreozzi@cnaf.infn.it] said:
in the current modeling, it cannot be implied because storageResource is associated to a dataStore via a many-to-many relationship.
It would have been possible in this case: storageResource 1--* dataStore
That seems to be a mistake, Datastore to Resource should indeed be many to 1. I don't think it could make sense for one piece of hardware to be managed by multiple systems.
I'm not sure about this. Within a storage area networks you can find several servers sharing storage devices such as disk subsystem and tape libraries and also data stored upon them. SAN aggregate and share the storage devices via volume manager. You can deploy some different file systems upon the same SAN infrastructure. I imagine a cluster file system, like GPFS, managing the disk system and TSM managing the mass storage system. Both systems are using the same SAN. Moreover, the storage administrator could build several file systems using GPFS, for example one per supported VO. Now, if I've understood well, there are only two/three DataStores: - one to represents the "disk quality", - one to represents the "tape quality", - and, eventually, one to represents the "hierarchical-system" not manageable by user. But, at the same time, we have a set of StorageResources corresponding to File Systems created.. Is it correct above description? If yes, I cannot be able to assign Size properties to DataStores.. vice versa I can assign sizes to StorageResources. If no, I don't understand what DataStore is. We could move Size properties to StorageResources and make a relationship between StorageShare <-> StorageResource and between StorageResource <-> DataStore (many-to-many). In this way we can represents above example, I think. I don't know about the relationship between StorageShare and DataStore. Perhaps it is still needed.
Stephen
Bye, Riccardo -- ------------------------------------------------------------------------------- Riccardo Zappi Istituto Nazionale di Fisica Nucleare - CNAF Viale Berti Pichat, 6/2, 40127 BOLOGNA - ITALY Phone: +39-051-609-2868 Fax: +39-051-609-2746 e-mail: riccardo.zappi@cnaf.infn.it -------------------------------------------------------------------------------
Riccardo Zappi [mailto:riccardo.zappi@cnaf.infn.it] said:
Within a storage area networks you can find several servers sharing storage devices such as disk subsystem and tape libraries and also data stored upon them. SAN aggregate and share the storage devices via volume manager. You can deploy some different file systems upon the same SAN infrastructure.
Maybe we should go back a bit. The purpose of the DataStore is *not* to represent all the details of the hardware, or indeed any of the details! In the past we have had no hardware description at all, and probably we still don't need it. However, people tend to keep raising some questions related to hardware, like how you know if the site uses tape storage. My proposal was to have a *simple* description of the hardware that can answer simple questions only - basically just whether the hardware is tape or disk, and perhaps what the total size is. My hope was that by doing that we could stop having discussions about hardware. If we are actually going to have more discussions about all these details then I think I want to withdraw the proposal completely, we would be better off not to have it at all since we still have no use-cases for it.
Now, if I've understood well, there are only two/three DataStores: - one to represents the "disk quality", - one to represents the "tape quality", - and, eventually, one to represents the "hierarchical-system" not manageable by user.
I'm not quite sure what you mean by the third one, but yes, basically the DataStore just divides the storage for a given Resource into tape and disk and nothing more complex.
But, at the same time, we have a set of StorageResources corresponding to File Systems created..
Is it correct above description?
In general I wouldn't say that a StorageResource corresponds to a file system the way I would normally think of it, e.g. ext3, it's intended for some higher-level management software like Castor. GPFS may be a special case here as it's more than just another file system. Anyway, even if a file system is published as a Resource you won't usually have one Resource per file system; you would only need multiple Resources if they are really separarate instances, e.g. they are managed separately and could be at different versions. Otherwise you would have only one Resource for GPFS, with some version number, and one Datastore for all the disk it manages.
If yes, I cannot be able to assign Size properties to DataStores.. vice versa I can assign sizes to StorageResources.
I don't understand the point. A piece of software (castor, gpfs) doesn't have any size, the hardware it manages (a set of disks) has a size.
We could move Size properties to StorageResources and make a relationship between StorageShare <-> StorageResource and between StorageResource <-> DataStore (many-to-many). In this way we can represents above example, I think.
No, that's much too complicated. All we need to represent is that a site has a disk farm and a tape robot, and that there is some software layer between the SRM and the hardware. Stephen
Sergio Andreozzi wrote:
You are right about the complicated scenario of supported protocol. I do not know if the use of the AccessPolicy on protocol works. Suppose the scenario where LHCb wants RFIO protocol available for one StorageShare, while for another one LHCb wants only FILE protocol. How can we represent this situation with ACL?
we might want to add the association from AccessProtocol to StorageShare; what do people think about this?
I suppose it would be OK (just a few foreign keys in the Share), but I think Riccardo's use case is _not_ a good example: CNAF is just being asked to do something silly to work around an LHCb software bug!
is is exactly the role of the Resource - maybe the name isn't very good but the Resource is supposed to represent the management software. So you would have a Resource for GPFS, with an associated Datastore to represent the disk servers.
In this case, we could define the StorageShare on StorageResource.. or not?
also here... I think Riccardo is right. The association from StorageShare to DataStore should be changed from StorageShare to StorageResource...
No, one reason we invented the DataStore was to show explicitly what hardware is being used by a Share, so we need to keep that relationship.
alternatively, we can add in parallel if people really want to discover the type of data store on which a storage share is configured. This will complicate the life of sysadmin who have to configure the providers, therefore, let us consider this issue carefully.
I would argue in the other direction: the Share-Resource relation can be inferred (if desired) from the Share-DataStore relation, but not vice versa.
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Maarten Litmaath said: I suppose it would be OK (just a few foreign keys in the Share),
It wouldn't be a big change in the schema, but it would be quite a big change in the clients because they would have a more complicated query, and would potentially have to know which Share they were going to use before they could select an SE, which could be inconvenient. In fact in general it's impossible - you can't reliably reverse-engineer a SURL to know which Share it belongs to, so given a SURL you couldn't tell if you could read it with a given protocol or not. Stephen
-----Original Message----- From: glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of Burke, S (Stephen) Sent: Freitag, 18. April 2008 02:12 To: Maarten Litmaath; Sergio Andreozzi Cc: glue-wg@ogf.org Subject: Re: [glue-wg] Some doubts
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Maarten Litmaath said: I suppose it would be OK (just a few foreign keys in the Share),
It wouldn't be a big change in the schema, but it would be quite a big change in the clients because they would have a more complicated query, and would potentially have to know which Share they were going to use before they could select an SE, which could be inconvenient. In fact in general it's impossible - you can't reliably reverse-engineer a SURL to know which Share it belongs to, so given a SURL you couldn't tell if you could read it with a given protocol or not.
Stephen
Coming back to what Maarten said: Is this CNAF-LHCB issue a use case which appears also on other sites? I tend to agree here with Maarten and point to the (client-)software which should take care of this. In fact, this is an agreement between the site and the vo which the model currently can't consider- except in a otherInfo field. So, lets find out whether we have to model this anyway. If so, we'll see then how we could do it. Felix
Hi Maarten, thanks for your comments. Maarten Litmaath wrote:
Sergio Andreozzi wrote:
You are right about the complicated scenario of supported protocol. I do not know if the use of the AccessPolicy on protocol works. Suppose the scenario where LHCb wants RFIO protocol available for one StorageShare, while for another one LHCb wants only FILE protocol. How can we represent this situation with ACL?
we might want to add the association from AccessProtocol to StorageShare; what do people think about this?
I suppose it would be OK (just a few foreign keys in the Share), but I think Riccardo's use case is _not_ a good example: CNAF is just being asked to do something silly to work around an LHCb software bug!
Maarten, you are right. My example is not a good example. Let me try to explain better. I made some inaccuracies in my description and so misunderstanding was born. I do not know if there exists a LHCb software bug... maybe not. I know only that LHCb prefers the use of FILE protocol rather than RFIO protocol, or in other words, LHCb users would like prevent the use of RFIO protocol even if it is supported. I tried to explain the case of a StorageService serving two or more StorageShares belonging to different VOs with divergent "preferences" about supported protocols, and I've reported a real use case about this.
is is exactly the role of the Resource - maybe the name isn't very good but the Resource is supposed to represent the management software. So you would have a Resource for GPFS, with an associated Datastore to represent the disk servers.
In this case, we could define the StorageShare on StorageResource.. or not?
also here... I think Riccardo is right. The association from StorageShare to DataStore should be changed from StorageShare to StorageResource...
No, one reason we invented the DataStore was to show explicitly what hardware is being used by a Share, so we need to keep that relationship.
alternatively, we can add in parallel if people really want to discover the type of data store on which a storage share is configured. This will complicate the life of sysadmin who have to configure the providers, therefore, let us consider this issue carefully.
I would argue in the other direction: the Share-Resource relation can be inferred (if desired) from the Share-DataStore relation, but not vice versa.
I need to think more about DataStore. I'm still not completely aware of DataStore, StorageResource and StorageShare relationship. Bye, Riccardo -- ------------------------------------------------------------------------------- Riccardo Zappi Istituto Nazionale di Fisica Nucleare - CNAF Viale Berti Pichat, 6/2, 40127 BOLOGNA - ITALY Phone: +39-051-609-2868 Fax: +39-051-609-2746 e-mail: riccardo.zappi@cnaf.infn.it -------------------------------------------------------------------------------
Riccardo Zappi [mailto:riccardo.zappi@cnaf.infn.it] said:
I know only that LHCb prefers the use of FILE protocol rather than RFIO protocol, or in other words, LHCb users would like prevent the use of RFIO protocol even if it is supported.
I still don't see the problem. Advertising rfio just says that it's available, it doesn't force anyone to use it. If lhcb have coded their application in a way where it does force them to use it then that is a bug in their code. Stephen
Hi Stephen, Burke, S (Stephen) wrote:
Riccardo Zappi [mailto:riccardo.zappi@cnaf.infn.it] said:
I know only that LHCb prefers the use of FILE protocol rather than RFIO protocol, or in other words, LHCb users would like prevent the use of RFIO protocol even if it is supported.
I still don't see the problem. Advertising rfio just says that it's available, it doesn't force anyone to use it. If lhcb have coded their application in a way where it does force them to use it then that is a bug in their code.
Yes, I agree with you. My example doesn't report a real problem, but perhaps only a excessive end-user apprehension caused by some bad past experience or by something else. I can imagine jobs written to make use of an external libraries to interact with IS and SRM. That libraries could retrieve the _unordered_ set of supported protocols from IS. The result set contains all the protocols supported by the Storage Service. Then the library builds the SRM request using the _unordered_ protocol list returned by IS query. Therefore, in theory, RFIO could be before to FILE protocol because a specific order is not guaranteed by IS. If my speculation is correct, then the solutions could be: 1) Modify external library used by jobs adding the possibility to specify the order in protocol preferences. or/and 2) make possible to specify which protocols are available for each "StorageShare" I think that both solutions could be implemented. But, if you think solution 2) inopportune, it is the same. We can suggest strongest tests on services providing the protocols to make end users less worried. Bye, Riccardo
Stephen
-- ------------------------------------------------------------------------------- Riccardo Zappi Istituto Nazionale di Fisica Nucleare - CNAF Viale Berti Pichat, 6/2, 40127 BOLOGNA - ITALY Phone: +39-051-609-2868 Fax: +39-051-609-2746 e-mail: riccardo.zappi@cnaf.infn.it -------------------------------------------------------------------------------
Riccardo Zappi [mailto:riccardo.zappi@cnaf.infn.it] said:
Other supported VOs want RFIO protocol available in addition to FILE and GSIFTP protocols. If we publish RFIO, FILE, and GSIFTP for StoRM service, then all LHCb jobs will fail because they try to use RFIO protocol.
I don't think I understand this - if lhcb don't want to use rfio why will the jobs try to use it? The information system only advertises that a protocol is available, the jobs are then free to use what they want - in fact I think you can give a list of protocols to the SRM in order of preference, so they could try file first and then rfio. Anyway, if jobs would fail with rfio they shouldn't try it at all.
Suppose the scenario where LHCb wants RFIO protocol available for one StorageShare, while for another one LHCb wants only FILE protocol. How can we represent this situation with ACL?
This isn't really an ACL, it's VO policy - I would say that it's up to lhcb jobs to know which protocol to use (e.g. using the space token description to choose), I don't see why it should be done on the site side.
In this case, we could define the StorageShare on StorageResource.. or not?
In the current schema draft I think the proposal is that the Datastore has relations to both Share and Resource, but there is no explicit link between Share and Resource. Stephen
participants (5)
-
Burke, S (Stephen) -
Felix Nikolaus Ehm -
Maarten Litmaath -
Riccardo Zappi -
Sergio Andreozzi