Chris's email forwarding. ---- Thanks to everyone for useful responses, and to both Manuel and Mark for providing workable hierarchical, abstract naming schemes ("directory standards", in the context of this discussion), each with valuable features. I'm writing this note as a preface to the discussion of naming at the F2F meeting today, and as a response to some of the points already made in this thread. I'll mostly just be throwing out a few vaguely connected thoughts and pieces of information to help advance the starting point for the discussion today. As important as I feel it is for the community to quickly agree upon a definite direction/core draft standard, it's also crucial to get it right - we don't necessarily have to provide a perfect solution the first time, but we do have to provide a solution that can grow and change organically to fulfill new needs, as the community creates them. Just to update my previous e-mail regarding the options and suggestions out there, I've spoken with Osamu Tatebe, and it appears that the RNS item in my initial list of efforts and proposals is now coincident with Dave Berry's post-GGF16 note suggesting that RNS be re-factored into at least two separate standards. In this regard, Osamu says "At the last GFS-WG session in GGF16, we agreed to refactor RNS specs into (at least) three specs; RNS, iterator, logical name resolver." It's not immediately clear to me what is meant by "RNS" in this quote, but my guess would be that it refers to the directory standard/service itself. My understanding is that Manuel Pereira is currently working on this refactoring, and will offer a draft to the GFS (and likely OGSA-Naming) working groups for further discussion, once a workable draft is complete. To clarify my own ulterior motives, my goal is to get the work on naming and directory-related issues moved into the OGSA-Naming group, even if it is the same set of players doing the work, so that the GFS- WG can focus on the requirements for file systems within a "directory and logical/abstract naming system" - for example, a GFS directory or "file" entry should have properties such as size, support information about the source storage resource and access mechanisms, replication properties, the data's tier within an HSM, etc etc. I'm hoping that we can all work together to create a set of directory naming and resolution standard(s) that can support these needs, and I believe that if we do, they will also support just about any other function for which an abstract-name-based directory system is necessary. From the above paragraph, I think it should be clear that I believe that it is a requirement to be the able to associate arbitrary attributes/properties with entries in the directory structure. What I envision here is a simple set of base standards, ideally to come out of the OGSA-Naming group; one for hierarchically organized collections of human-readable abstract names with arbitrary properties attached, each of which resolves to an EPR, and one for an iterator interface, where the items being iterated over are the abstract names. Each standard utilizing hierarchical collections of human-readable names, including the GFS-WG, would then define a "profile" for use of the directory service, including requirements for the properties which should be associated with a directory entry, and any higher-level services which would ideally build upon the "directory" standard I'm hoping to see. I believe that the WS-Directory proposal, certainly in the v3 iteration Mark Morgan referred us to in an earlier e-mail, would fulfill the requirements I have for hierarchically organized collections of human-readable names, referring to EPRs and having attributes associated with them. We still need, in my opinion, an iterator interface standard to work in concert with these "directories"; hopefully Manuel's work with refactoring RNS will provide us with that. I'll stop here, since I believe the F2F discussion of naming will begin shortly. Thanks again to everyone for participating in this discussion so far, and I look forward to continuing it both in the F2F and through other channels. On Mar 28, 2006, at 11:17 AM, Allen Luniewski wrote:
Chris,
Thanks for the very thoughtful note! A few comments.
I agree with you that the current situation is, to put it mildly, not acceptable. We need to move to a position where the community agrees on a single means for creating a hierarchy of pointers to resources (a directory structure in Unix-speak). Clearly one aspect of this is that RNS and WS-Directory need to reconciled into a single proposal. Resolution sooner rather than later is clearly in the best interests of the various WGs who depend upon directories (including OGSA Data which I share responsibility for).
You mention a couple of specific technical issues. A few comments on those: 1. I go back and forth on attributes in the directory system. Since these are almost certainly cached from the resources, today I am inclined to say that attributes should not be part of the directory system. Ask me tomorrow, and I may have the other answer :-) But today, I would leave them out of the base specification for simplicity but consider an extension that included attributes if that were felt to be vital. 2. Dave Berry's suggestion to separate out the iterator component is a good one. There are going to be many places in the overall grid standards effort where an iterator-like structure will be needed. Standardizing this seems like a proper goal for GGF. If that is accepted then using it in a directory service seems quite natural.
As for moving forward, I think that we need to see how this thread plays out. My instincts, however, are that getting the interested parties in a room for a few hours would be the most effective way to drive this to an early resolution.
Allen Luniewski IBM Information Management Division San Jose, California
Christopher Jordan <ctjordan@sdsc.edu> Subject: GGF/OGSA standards for hierarchical namespaces
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
All,
Forgive the wide distribution on this e-mail, but this issue seems to be to be both extraordinarily important to the future of GGF/OGSA standards efforts and also in a state of either limbo or paralysis. The topic I'm addressing here, both in my capacity as the secretary of GFS-WG and as a generally interested participant on a few different GGF working groups, is the question of adopting a single, possibly minimal, standard for creating hierarchically organized collections of pointers (WS-Names? GSR/GSHs, to date myself?) to "resources", where the term "resource" could denote a service providing access to a collection of files, computational resources, or database records (that's a non-exclusive list), and where some items in the hierarchy could actually represent directory-like structures, i.e. containers for other collections of resources.
The way I got involved in this discussion through the Grid File Systems-WG, which at the time was bringing the RNS specification forward for final approval as a GFD. Subsequently, there have been numerous discussions outside of the GFS-WG context about the suitability of the RNS standard for more general applications, as well as the (perceived) complexity of the standard as a barrier to entry. There have also been alternative directory construction standards proposed by members of the OGSA-Naming-WG.
The following are the activities/proposals I know of:
RNS: I know the GGF editors have returned the final(?) RNS draft to GFS-WG, with the suggestion that it is too specific to filesystem needs, and the suggestion that it either be limited in scope to GFS applications only (a non-optimal solution for obvious reasons) or that the authors work with the OGSA-Naming people to help develop a universal standard for hierarchical resource namespaces. If we are to move forward with RNS, one of these options will clearly be a necessity, given the points Greg Newby made in his responses on behalf of the GFSG.
WS-Directory: This is the hierarchical namespace standard developed at UVa in response to their difficulty in implementing the complexities and ambiguities in RNS. I like the simplicity of WS- Directory, however it seems to be missing significant requirements for general use such as attributes, both attributed which should be required such as time-to-live, and the ability to add extensibility attributes such as resource type, QoS, etc. This ability to add arbitrary attributes is present in RNS but it still lacks some obviously fundamental required attributes.
Finally, Dave Berry sent an e-mail immediately after GGF16 in which he mentioned the suggestion that we separate this functionality into two logical functions, and therefore standards - a Directory Interface and an Iterator interface, in which Directory interfaces were essentially just pointers to Iterators, which would be standardized. However, there would be no restriction that a Directory point to a particular type of iterator interface. One point I wasn't clear on from the e-mail was whether an entry in an interator could be another directory, although I suspect it can.
This short list is what I've got within easy reach. As I said previously, I believe this is an important issue to resolve quickly, and I'm sending this note in the hopes of initiating the conversation among as many of the relevant parties as I can. Please feel free to forward at will, respond with agreement, anger, or even unconcealed rage.
Possible ways forward would be for us to have a conference call (GFS- WG meets rarely, and we could quite easily give up our call for a more focused discussion of these issues), an extended e-mail discussion, or a meeting at the next GGF (assuming we get a chance).
Let me know how you feel about the options presented above, or feel free to propose new ones if you like. The important thing is that we begin to gain momentum, and then keep it going forward.
Thanks.
N.B. For anyone who may have missed any of the discussions reference above, please let me know and I'll be happy to forward them to you from my archives.
- ---------------------------------------------------- Chris Jordan HPC Systems Engineer High End Computing Systems Group San Diego Supercomputer Center ctjordan@sdsc.edu 858.534.8347 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFEM/xVPCVtcXn6kg8RArY1AKCO4lAdeWchGcMgDcmDl1YAx5jEQwCgybp7 NZUrUXmTwpseVL9P0/WscUI= =AhSs -----END PGP SIGNATURE-----