Hello all, Thank you Andrew for your comments and points of clarification. I do hope to incorporate the discussed changes in the RNS draft within the next couple of days. Please see my individuals comments in response to your post below: -------------------------
High level comments:
1) The basic directory function is to provide a mapping handle f(“string”), i.e., to map strings to some form of handle.
Thus, in section 1.1. Why do we need four different types of junctions? Should we not have one type of junction: mainly to an EPR. Thus a directory would map a string to an EPR. EPR f(“string”); Instead there are four “types” of junctions: EPR’s Virtualized reference – either contains an EPR or a URL that points to some other service Referrals – point to another RNS service. Q: could this not be modeled as an EPR? Alias – points to another “entry” within the “same” RNS service. This seems to imply a container like model – more on this later.
I suggest that we need only ONE type of junction – an EPR. That will simplify client coding – and the model significantly.
You are correct in asserting that the “basic” directory function is mapping a string to some identifier. That is only the “basic” function, a hierarchical namespace service that is capable of scaling, delegation, and federation of namespace services must have some means of junctioning namespaces. Although it is arguably unnecessary for the service consumer to be aware of referral junctions in normal query related operations, it is necessary for management operations; particularly the create operation which allows administrative grafting of namespaces by the use of referral entries. First, the “virtualized reference” really only makes sense if the abstract name resolution service is tightly coupled with the namespace service. Since we are factoring the resolver service out, I am also eliminating the “virtualized reference” junction. As for the “alias”, this type of junction is really nothing more than a “referral junction”. After discussing this with the GFS-WG, they feel strongly that “aliases” should be a basic feature of the service, however, agree that aliases (with hardlink “like” behavior) should be optional. So we are ultimately left with two “basic” junctions: EPR Junction and Referral Junction. A referral junction is necessary since the service is intended to accommodate requests that issue paths that span multiple repositories. It is important to differentiate between the treatment of a referral junction and an EPR junction; as noted in section 1.4.6 (RNSJunctionFault) when a path extends beyond the leaf node of an EPR junction the service targeted is unknown and therefore may not be “assumed” to be a delegate namespace service. For this reason, junction types are necessary to enable discriminatory treatment.
2) Implied model – Repositories There is a notion in the spec of “same service instance” that in conversations has also been called a “repository”. The basic idea is that an RNS service may “contain” a set of directories – and that the service has a root. Thus junction types distinguish, for example, between internal and external things that they point to. Thus an RNS service is a rooted tree – that may point at the leaves to other rooted trees. So, first of all – it is implied. If we’re going to have that model it should be right up front and discussed.
This strongly casted description of “rooted trees” is only realized when federating namespace services, which is described accordingly in section 2. The core focus of the specification should be the service itself.
What is a repository? What are it’s special port types if any, etc.
The term “repository” simply refers to the “container of stored namespace entries”. Incidentally, any given RNS service provider only “services” names that are contained within its corresponding repository; there is only one repository per RNS provider. You can easily consider the repository the backend datastore or database where names, mappings, and related metadata are persistently stored.
Second, I think that it is not the right way to think about the problem. Directories should be the resources – not collections of directories. If a particular implementation chooses to multiplex a large number of logically independent directories in a single container – great – we will certainly do that too. The issue is what is the model. I feel fairly strongly about this.
This is not clear to me. The statement “directories should be the resources - not collections of directories”, what are referring to with regard to RNS? RNS describes “virtual” directories as namespace entries that enable hierarchal relationship. They enable partitioning, grouping, categorizing, etc., however how they are used and what they are used for is not mandated. Virtual directories only facilitate directory structure; they do not hinder, restrict, or otherwise alter how namespaces are junctioned together (or how you can refer to another namespace). So what is the “issue” with the “model”?
Links “into” other RNS servers:1.1.2.4 “Alias Junction” is restricted to pointing to entries in the same repository. I think they should be able to point to anything – including directories in “other” repositories.
First an “alias” is intended to facilitate hardlink “like” behavior; the alias junction may need to be described further to communicate this point. Consequently, it far to impractical to allow aliases to refer to namespace entries in other repositories if they indeed behave like hardlinks; if you simply want to “refer” to any namespace entry (not just a root node) in another repository you would use a “referral junction”.
It has been claimed that using the EPR of the “repository” and a path you can get that effect.
As stated above, you would use a referral junction, not an EPR junction.
However, what if the path changes in the other container? My link would break – even if the directory itself still exists.
Correct, this is why some have expressed the desire to describe an optional “alias” type that would function like a hardlink; understanding the practical constraints of hardlinks this is only feasible within a single repository.
3) Full path names In ANY directory system lookup really takes two parameters – a “root” at which to start, and a path. Often the “root” is implied, or is at some well-known location. RNS – as written, implies that all lookups are based on full paths with an implied, unspecified root.
This is not true. The root of each RNS is specified, since the only way to communicate with an RNS service provider is to first establish an EPR to it. “Full paths” are then interpreted as the path from the root of the current operating service provider.
Assuming that full paths are to be used on all lookups, the potential for both hot spots AND single point of failure are clear.
First we need to be careful distinguishing between “full paths” and “absolute paths”, please see section 1.1.3. In the comment just above, I used the term “full path” to consistently signify the path from the root of a given repository. I may need to clarify this a bit in the draft, but RNS operations must be able to handle “absolute paths”. If the absolute path extends beyond the current repository (supposing a federation of namespace services with delegated namespaces), then RNS must be able to gracefully refer the client/application to the targeted namespace service provider (see section 2.2). If, however, the absolute path can be resolved within the current repository (which is expected to be the majority of cases), then RNS will simply respond with the appropriate answer. In practice RNS operations do effectively operate use “full paths”, but must be able to handle “absolute paths” (utilizing referral messages as the redirecting mechanism). The obvious optimization for an application/client is to simply avoid sending resolution requests to the upper-level RNS service providers if you are working in a lower-level of a federated namespace. For example, if I am resolving names in directory “/a/b/c”, and “b” is a referral to a delegate RNS service provider, then I would simply continue my interaction with the RNS service provider that is serving “b” (see section 2.2.1.1).
In conversation with Manual he mentioned that his clients cache intermediate parts of the tree in the sense that “/foo/bar/d1” as a prefix leads to a particular RNS service, and then use that info to not always traverse the tree. Besides the obvious implementation challenges of cache consistency when the tree is changing (a problem we certainly had/have in Legion) there is the modeling issue. If we expect clients to do that – then perhaps the architecture/specification should accommodate that and say that all lookups are relative path lookups with respect to some “root”. The root could be a true “root”, or interior node in a tree, which is itself a “root” of the subtree it defines.
As stated above, this is described in the draft. The implication of caching here is obscure in light of my description of the EPR that is used to communicate with the most appropriate RNS service provider. What is the modeling issue that you refer to here?
4) Resolve and file system profile. We discussed these on the last ogsa call, my understanding is that they are going out.
Correct, we have agreed to remove these two items understanding the following: 1) The Grid File System naming profile will be factored out into an independent document under the GFS-WG. 2) The resolver work will be discussed and an attempt to avoid duplicate effort will be pursued. Additionally, the OGSA-Naming WG will not simply disregard this existing work.
5) Iterators. OGSA-Data-WG has discussed iterators in a more general way, e.g., on data base query results etc., I think that whatever is done in RNS should be consistent with whatever is done in OGSA-Data (note – consistency can happen either way).
RNS should be consistent, however OGSA-Data has not had any material discussion along these lines. Since an iterator is necessary in RNS and can always be modified, if necessary, later we are going to leave it as is for this first submission.
Medium level comments:
S 1.1 “In all cases, junctions are capable of maintaining a list of references (EPRs/URLs) per entry, that is a single junction my render several available EPRs, each of which represent replicas, copies of the same resource, or operationally identical services. “
Why? Are you saying that replication issues and semantics should be dealt with in the directory structure? Or are you saying that directories are not “sets” in the sense of only one entry – but rather “multi-sets” in the sense that one string can map to multiple things. If the later – what are the implied semantics. I think it may be safer to keep them as sets.
We are not in attempting to deal with replication issues and semantics, however there is no apparent reason why we should not facilitate a “one-to-many” mapping at the entry level. I am not sure what you are trying to say regarding “sets” and their relationship to “directories”, there is no relationship between what is stated in section 1.1 and the concept of directories. Section 1.1 is attempting to describe the capability of a single junction. Each junction is represented as a name in the namespace, however maintains a mapping to “one or more” targets. The expounded statement regarding “replicas, copies of the same resource, or operationally identical services” is attempting to describe the necessary requirement of ensuring that all of the targets of a single junction refer to “the same thing”. For example, junction “foo” may map to a resource named “bar”, with one in San Jose and one in New York. The resource “bar” must be the same in all locations to ensure that “/a/b/c/foo” renders effectively the same resource regardless of which target EPR is selected. foo => http://sanjose.abc.com/bar => http://newyork.abc.com/bar Thank you for your time in review and comments. Best regards, Manuel Pereira =============================== IBM Almaden Research Center =============================== 650 Harry Road, San Jose, CA 95120 1-408-927-1935 [T/L 457] Pager: 800-946-4646 1492425 http://mvp3.almaden.ibm.com mpereira@us.ibm.com