Hi Peter, since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer. So, playing ping pong, what would you propose as a sensible approach? Others on the list any comments? Cheers, Andre. Quoting [Peter Kunszt] (Jul 26 2005):
hi andre
ok! about consistency, you are right, that should be discussed separately. however, i have another - related - issue with copy: security semantics.
if you issue a copy command and the source is not owned by you but you have read permission (say through ACLs). and it is a recursive copy; how do you propagate permission information to the target? do you make everything owned by the person whe issues the copy (which may be a service!) or do you copy the ACLs and the permissions along with the file (ie metadata copy)? how do you make sure that the same users exist then on the target site?
can of worms opened...
cheers peter
----- Original Message ----- From: "Andre Merzky" <andre@merzky.net> To: "Peter Kunszt" <peter.kunszt@cern.ch> Cc: "Andre Merzky" <andre@merzky.net>; "Simple API for Grid Applications WG" <saga-rg@ggf.org>; "Kalman Kovari" <Kalman.Kovari@cern.ch> Sent: Monday, July 25, 2005 2:25 PM Subject: Re: comment on SAGA strawman doc.
Quoting [Peter Kunszt] (Jul 21 2005):
hello andre
maybe the doc we were looking at was not the latest one.. where is the one that is the current reference? we got our copy from gridforge..
The latest version is at our wiki: http://wiki.cct.lsu.edu/saga/
On GridForge was a good version as well, the one we wanted to have reviews for. So the excuse of having given you an old copy does not work it seems ;-)
- The examples are sometimes more misleading than useful...
Hmm, we are not sure why that is. Are they too complex? Not focused enough?
our (certainly subjective) impression was that they are too simplistic and not enough explanations are given. specifying most possible errors would be also quite useful and a motivation what problem the example at hand is supposed to solve. maybe again it was an early document.
Point taken I think. Currently the examples are pretty much just pieces of C++ code which are supposed to illustrate the useage of parts of the API. Seems to work well for the authors, but not so well for the reader. Ok, thanks.
- Interface NSDir: what are the semantics of the copy method? of getName and getURL?
Hmm, we actually hoped that the semantics is well defined. If the copy call returns w/o error, a copy of a file will exist on the target location.
in the doc we had, it was just listed, no extra text was there stating what happens if something goes wrong...
For example for copy, the current version says (the flags are defined elsewhere):
------------------------------------------------------------- Methods for operation on namespace entities: - copy Purpose: copy the entry to another part of the namespace Format: void copy (in string source, in string target, in array<copyFlags,1> flags); Inputs: source: name to copy target: name to copy to flags: flags defining the operation modus Outputs: none Throws: BadParameter: name(s) and/or flags are invalid NoSuchFile: name(s) do(es) not exist NoSuccess: flags inhibited successful operation Notes: - if the target is a directory the source entry is copied into the directory - it is an error if the source is a directory and the 'Recursive' flag is not set. - if the target already exists, it will be overwritten if the 'Overwrite' flag is set, otherwise it is an error - default flags set is {NoOverwrite, NoRecursive} - Overwrite and NoOverwrite cannot be specified at the same time - Recursive and NoRecursive cannot be specified at the same time - similar to the 'cp' command in Unix shells, as defined by POSIX -------------------------------------------------------------
The old version should have been similar I think (have to check). It does indeed not tell you much about consistency, only refers to POSIX shell cp.
About consistency we had a lengthy discussion, with the aggreement that the consistency model is to be defined and documented by the implementation. The API spec itself does not assume any specific consistency model, as we feel that (a) POSIX consistency is not achievable within reasonable effort/performance, (b) if the user assumes the worst (no consistency), he will still be able to make good use of the API, and (c) reality will be somewhere in the middle.
Would you see problems with that approach?
Cheers, Andre.
-- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky@cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+
-- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky@cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+
All,
since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer.
if you issue a copy command and the source is not owned by you but you have read permission (say through ACLs). and it is a recursive copy; how do you propagate permission information to the target? do you make everything owned by the person whe issues the copy (which may be a service!) or do you copy the ACLs and the permissions along with the file (ie metadata copy)? how do you make sure that the same users exist then on the target site?
IMHO, there is only one sensible solution: the new owner of the copy determines access control to the newly created file. That should be a policy decision local to the target site. However, controling this from the SAGA API may be 'interesting'. So, should there be some kind of property determining access control for files and directories to be created? (I am afraid, we are stressing the "S" for simplicity if we are working towards a comprehensive solution...) Any thoughts? Thilo -- Thilo Kielmann http://www.cs.vu.nl/~kielmann/
On Jul 27, 2005, at 2:37 AM, Thilo Kielmann wrote:
All,
since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer.
if you issue a copy command and the source is not owned by you but you have read permission (say through ACLs). and it is a recursive copy; how do you propagate permission information to the target? do you make everything owned by the person whe issues the copy (which may be a service!) or do you copy the ACLs and the permissions along with the file (ie metadata copy)? how do you make sure that the same users exist then on the target site?
IMHO, there is only one sensible solution: the new owner of the copy determines access control to the newly created file. That should be a policy decision local to the target site.
However, controling this from the SAGA API may be 'interesting'. So, should there be some kind of property determining access control for files and directories to be created? (I am afraid, we are stressing the "S" for simplicity if we are working towards a comprehensive solution...)
Any thoughts?
My current thought on this is that file permissions management is a serious problem and it is quite unfortunate that it has been mostly overlooked in much of the current grid middleware. I can move files, but I can't use the same interface that I used to move the files around to manage the permissions on said files. Its something that I've complained about for years to no avail. This deficiency has led to a number of significant problems in many collaboratory projects, but I haven't seen it adequately addressed by any "completed" or "deployed" standard as of yet. Am I missing something or is there a group that is working on solving this problem as I speak? (I don't know because DOE doesn't let me go to grid meetings anymore) So getting back to SAGA, while I think that permissions management is an important and oft-neglected aspect of distributed file access middleware, I don't actually see any "standard" solutions to the problem. Since SAGA is supposed to be an API standardization rather than trying to write a "new grid" or fix any deficiencies in current middleware, the proper approach is to not attempt to address this issue until we see more middleware implementations that actually implement this feature. For the time being, its probably best to use the convention that Thilo mentions above because its pretty much what we are doing currently with grid file movers. This is kind of sad as far as solutions are concerned, but its probably good to set aside standardization of features in SAGA that are not already apparent in mainstream grid software implementations. -john
we have some rudimentary abstraction based on unix permissions in the Java CoG Kit. If the server supports it we can change permissions from the client. This is available as part of Java CoG kit 4. this is probably not the perfect solution but it could provide some input on how we have developed something that was useful to us. gregor On Jul 29, 2005, at 12:47 PM, John Shalf wrote:
On Jul 27, 2005, at 2:37 AM, Thilo Kielmann wrote:
All,
since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer.
if you issue a copy command and the source is not owned by you but you have read permission (say through ACLs). and it is a recursive copy; how do you propagate permission information to the target? do you make everything owned by the person whe issues the copy (which may be a service!) or do you copy the ACLs and the permissions along with the file (ie metadata copy)? how do you make sure that the same users exist then on the target site?
IMHO, there is only one sensible solution: the new owner of the copy determines access control to the newly created file. That should be a policy decision local to the target site.
However, controling this from the SAGA API may be 'interesting'. So, should there be some kind of property determining access control for files and directories to be created? (I am afraid, we are stressing the "S" for simplicity if we are working towards a comprehensive solution...)
Any thoughts?
My current thought on this is that file permissions management is a serious problem and it is quite unfortunate that it has been mostly overlooked in much of the current grid middleware. I can move files, but I can't use the same interface that I used to move the files around to manage the permissions on said files. Its something that I've complained about for years to no avail. This deficiency has led to a number of significant problems in many collaboratory projects, but I haven't seen it adequately addressed by any "completed" or "deployed" standard as of yet. Am I missing something or is there a group that is working on solving this problem as I speak? (I don't know because DOE doesn't let me go to grid meetings anymore)
So getting back to SAGA, while I think that permissions management is an important and oft-neglected aspect of distributed file access middleware, I don't actually see any "standard" solutions to the problem. Since SAGA is supposed to be an API standardization rather than trying to write a "new grid" or fix any deficiencies in current middleware, the proper approach is to not attempt to address this issue until we see more middleware implementations that actually implement this feature. For the time being, its probably best to use the convention that Thilo mentions above because its pretty much what we are doing currently with grid file movers. This is kind of sad as far as solutions are concerned, but its probably good to set aside standardization of features in SAGA that are not already apparent in mainstream grid software implementations.
-john
I agree with John: if the problem is basically unsolved, we (SAGA-RG) should noty try to solve it. We would do a bad job at it anyway I guess. So Gregors proposal to stick to the (well understood) scheme of Unix like owner/group/other permissions sounds pretty good to me... My $0.02, Andre. Quoting [Gregor von Laszewski] (Jul 29 2005):
we have some rudimentary abstraction based on unix permissions in the Java CoG Kit. If the server supports it we can change permissions from the client. This is available as part of Java CoG kit 4. this is probably not the perfect solution but it could provide some input on how we have developed something that was useful to us.
gregor
On Jul 29, 2005, at 12:47 PM, John Shalf wrote:
On Jul 27, 2005, at 2:37 AM, Thilo Kielmann wrote:
All,
since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer.
if you issue a copy command and the source is not owned by you but you have read permission (say through ACLs). and it is a recursive copy; how do you propagate permission information to the target? do you make everything owned by the person whe issues the copy (which may be a service!) or do you copy the ACLs and the permissions along with the file (ie metadata copy)? how do you make sure that the same users exist then on the target site?
IMHO, there is only one sensible solution: the new owner of the copy determines access control to the newly created file. That should be a policy decision local to the target site.
However, controling this from the SAGA API may be 'interesting'. So, should there be some kind of property determining access control for files and directories to be created? (I am afraid, we are stressing the "S" for simplicity if we are working towards a comprehensive solution...)
Any thoughts?
My current thought on this is that file permissions management is a serious problem and it is quite unfortunate that it has been mostly overlooked in much of the current grid middleware. I can move files, but I can't use the same interface that I used to move the files around to manage the permissions on said files. Its something that I've complained about for years to no avail. This deficiency has led to a number of significant problems in many collaboratory projects, but I haven't seen it adequately addressed by any "completed" or "deployed" standard as of yet. Am I missing something or is there a group that is working on solving this problem as I speak? (I don't know because DOE doesn't let me go to grid meetings anymore)
So getting back to SAGA, while I think that permissions management is an important and oft-neglected aspect of distributed file access middleware, I don't actually see any "standard" solutions to the problem. Since SAGA is supposed to be an API standardization rather than trying to write a "new grid" or fix any deficiencies in current middleware, the proper approach is to not attempt to address this issue until we see more middleware implementations that actually implement this feature. For the time being, its probably best to use the convention that Thilo mentions above because its pretty much what we are doing currently with grid file movers. This is kind of sad as far as solutions are concerned, but its probably good to set aside standardization of features in SAGA that are not already apparent in mainstream grid software implementations.
-john
-- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky@cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+
Yep, Gregor's proposal sounds good to me as well. Lets stick with "solved" problems (however they happen to be solved). :-) On Jul 29, 2005, at 1:27 PM, Andre Merzky wrote:
I agree with John: if the problem is basically unsolved, we (SAGA-RG) should noty try to solve it. We would do a bad job at it anyway I guess.
So Gregors proposal to stick to the (well understood) scheme of Unix like owner/group/other permissions sounds pretty good to me...
My $0.02,
Andre.
Quoting [Gregor von Laszewski] (Jul 29 2005):
we have some rudimentary abstraction based on unix permissions in the Java CoG Kit. If the server supports it we can change permissions from the client. This is available as part of Java CoG kit 4. this is probably not the perfect solution but it could provide some input on how we have developed something that was useful to us.
gregor
On Jul 29, 2005, at 12:47 PM, John Shalf wrote:
On Jul 27, 2005, at 2:37 AM, Thilo Kielmann wrote:
All,
since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer.
if you issue a copy command and the source is not owned by you but you have read permission (say through ACLs). and it is a recursive copy; how do you propagate permission information to the target? do you make everything owned by the person whe issues the copy (which may be a service!) or do you copy the ACLs and the permissions along with the file (ie metadata copy)? how do you make sure that the same users exist then on the target site?
IMHO, there is only one sensible solution: the new owner of the copy determines access control to the newly created file. That should be a policy decision local to the target site.
However, controling this from the SAGA API may be 'interesting'. So, should there be some kind of property determining access control for files and directories to be created? (I am afraid, we are stressing the "S" for simplicity if we are working towards a comprehensive solution...)
Any thoughts?
My current thought on this is that file permissions management is a serious problem and it is quite unfortunate that it has been mostly overlooked in much of the current grid middleware. I can move files, but I can't use the same interface that I used to move the files around to manage the permissions on said files. Its something that I've complained about for years to no avail. This deficiency has led to a number of significant problems in many collaboratory projects, but I haven't seen it adequately addressed by any "completed" or "deployed" standard as of yet. Am I missing something or is there a group that is working on solving this problem as I speak? (I don't know because DOE doesn't let me go to grid meetings anymore)
So getting back to SAGA, while I think that permissions management is an important and oft-neglected aspect of distributed file access middleware, I don't actually see any "standard" solutions to the problem. Since SAGA is supposed to be an API standardization rather than trying to write a "new grid" or fix any deficiencies in current middleware, the proper approach is to not attempt to address this issue until we see more middleware implementations that actually implement this feature. For the time being, its probably best to use the convention that Thilo mentions above because its pretty much what we are doing currently with grid file movers. This is kind of sad as far as solutions are concerned, but its probably good to set aside standardization of features in SAGA that are not already apparent in mainstream grid software implementations.
-john
-- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky@cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+
The one thing that you may want to include is an exception model that lets people know if the platform you run the command on supports the feature or not. This way you can decide if you like to use a resource or you can gather the minimal requirement for the resource. For Java we have chosen the following approach: Instead of doing this for each resource you can do it on an os basis for this case. So if you go to a windows filesystem. In the CoG kit we introduced the provider concept as part of the API. This means that you have a factory that returns pointers to the implementation. So when you instantiate the class you get the interface, but you also need a provider that puts "meat" in the class. this is real nice as you now can program against an interface, but the actual instantiation happens at runtime. I know this is Java and OO. Maybe something like this can be also done through dynamic classes in C. On Jul 29, 2005, at 4:24 PM, John Shalf wrote:
Yep, Gregor's proposal sounds good to me as well. Lets stick with "solved" problems (however they happen to be solved). :-)
On Jul 29, 2005, at 1:27 PM, Andre Merzky wrote:
I agree with John: if the problem is basically unsolved, we (SAGA-RG) should noty try to solve it. We would do a bad job at it anyway I guess.
So Gregors proposal to stick to the (well understood) scheme of Unix like owner/group/other permissions sounds pretty good to me...
My $0.02,
Andre.
Quoting [Gregor von Laszewski] (Jul 29 2005):
we have some rudimentary abstraction based on unix permissions in the Java CoG Kit. If the server supports it we can change permissions from the client. This is available as part of Java CoG kit 4. this is probably not the perfect solution but it could provide some input on how we have developed something that was useful to us.
gregor
On Jul 29, 2005, at 12:47 PM, John Shalf wrote:
On Jul 27, 2005, at 2:37 AM, Thilo Kielmann wrote:
All,
since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer.
> if you issue a copy command and the source is not owned by > you but > you have read permission (say through ACLs). and it is a > recursive copy; > how do you propagate permission information to the target? do > you make > everything owned by the person whe issues the copy (which may be > a service!) > or do you copy the ACLs and the permissions along with the file > (ie metadata > copy)? how do you make sure that the same users exist then on > the target > site? > >
IMHO, there is only one sensible solution: the new owner of the copy determines access control to the newly created file. That should be a policy decision local to the target site.
However, controling this from the SAGA API may be 'interesting'. So, should there be some kind of property determining access control for files and directories to be created? (I am afraid, we are stressing the "S" for simplicity if we are working towards a comprehensive solution...)
Any thoughts?
My current thought on this is that file permissions management is a serious problem and it is quite unfortunate that it has been mostly overlooked in much of the current grid middleware. I can move files, but I can't use the same interface that I used to move the files around to manage the permissions on said files. Its something that I've complained about for years to no avail. This deficiency has led to a number of significant problems in many collaboratory projects, but I haven't seen it adequately addressed by any "completed" or "deployed" standard as of yet. Am I missing something or is there a group that is working on solving this problem as I speak? (I don't know because DOE doesn't let me go to grid meetings anymore)
So getting back to SAGA, while I think that permissions management is an important and oft-neglected aspect of distributed file access middleware, I don't actually see any "standard" solutions to the problem. Since SAGA is supposed to be an API standardization rather than trying to write a "new grid" or fix any deficiencies in current middleware, the proper approach is to not attempt to address this issue until we see more middleware implementations that actually implement this feature. For the time being, its probably best to use the convention that Thilo mentions above because its pretty much what we are doing currently with grid file movers. This is kind of sad as far as solutions are concerned, but its probably good to set aside standardization of features in SAGA that are not already apparent in mainstream grid software implementations.
-john
-- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky@cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+
Also mind the "S" in saga ;-) On Jul 29, 2005, at 3:27 PM, Andre Merzky wrote:
I agree with John: if the problem is basically unsolved, we (SAGA-RG) should noty try to solve it. We would do a bad job at it anyway I guess.
So Gregors proposal to stick to the (well understood) scheme of Unix like owner/group/other permissions sounds pretty good to me...
My $0.02,
Andre.
Quoting [Gregor von Laszewski] (Jul 29 2005):
we have some rudimentary abstraction based on unix permissions in the Java CoG Kit. If the server supports it we can change permissions from the client. This is available as part of Java CoG kit 4. this is probably not the perfect solution but it could provide some input on how we have developed something that was useful to us.
gregor
On Jul 29, 2005, at 12:47 PM, John Shalf wrote:
On Jul 27, 2005, at 2:37 AM, Thilo Kielmann wrote:
All,
since we have not approached ACLs yet, and since I am not really knowladgable about security, I have no answer.
if you issue a copy command and the source is not owned by you but you have read permission (say through ACLs). and it is a recursive copy; how do you propagate permission information to the target? do you make everything owned by the person whe issues the copy (which may be a service!) or do you copy the ACLs and the permissions along with the file (ie metadata copy)? how do you make sure that the same users exist then on the target site?
IMHO, there is only one sensible solution: the new owner of the copy determines access control to the newly created file. That should be a policy decision local to the target site.
However, controling this from the SAGA API may be 'interesting'. So, should there be some kind of property determining access control for files and directories to be created? (I am afraid, we are stressing the "S" for simplicity if we are working towards a comprehensive solution...)
Any thoughts?
My current thought on this is that file permissions management is a serious problem and it is quite unfortunate that it has been mostly overlooked in much of the current grid middleware. I can move files, but I can't use the same interface that I used to move the files around to manage the permissions on said files. Its something that I've complained about for years to no avail. This deficiency has led to a number of significant problems in many collaboratory projects, but I haven't seen it adequately addressed by any "completed" or "deployed" standard as of yet. Am I missing something or is there a group that is working on solving this problem as I speak? (I don't know because DOE doesn't let me go to grid meetings anymore)
So getting back to SAGA, while I think that permissions management is an important and oft-neglected aspect of distributed file access middleware, I don't actually see any "standard" solutions to the problem. Since SAGA is supposed to be an API standardization rather than trying to write a "new grid" or fix any deficiencies in current middleware, the proper approach is to not attempt to address this issue until we see more middleware implementations that actually implement this feature. For the time being, its probably best to use the convention that Thilo mentions above because its pretty much what we are doing currently with grid file movers. This is kind of sad as far as solutions are concerned, but its probably good to set aside standardization of features in SAGA that are not already apparent in mainstream grid software implementations.
-john
-- +-----------------------------------------------------------------+ | Andre Merzky | phon: +31 - 20 - 598 - 7759 | | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 | | Dept. of Computer Science | mail: merzky@cs.vu.nl | | De Boelelaan 1083a | www: http://www.merzky.net | | 1081 HV Amsterdam, Netherlands | | +-----------------------------------------------------------------+
participants (4)
-
Andre Merzky -
Gregor von Laszewski -
John Shalf -
Thilo Kielmann