Next phone meeting and foreign keys discussion
In order to prepare for the conference we have tomorrow, please take a look at the agenda: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMe... Moreover, in order to decide where to put foreign keys, I have created a table in the wiki so we can vote and discuss in the phone conference: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDAPFor... Please, vote if you have a strong opinion and know use cases where it should be better to put it in one place and not in another. Here, voting means putting your name in either the column "Votes on 1" or "Votes on 2". Regards, David -- David Horat Software Engineer – IT/GD – Grid Deployment Group CERN – European Organization for Nuclear Research » Where the web was born Address: 1211 Geneva - Switzerland, Office: 28/R-003 Phone +41 22 76 77996 Fax +41 22 76 68178 (fax to email service) Web: http://cern.ch/horat Web: http://davidhorat.com/ Profile: http://linkedin.com/in/davidhorat
Hi, David Horat wrote:
In order to prepare for the conference we have tomorrow, please take a look at the agenda: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMe...
Moreover, in order to decide where to put foreign keys, I have created a table in the wiki so we can vote and discuss in the phone conference: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDAPFor...
Please, vote if you have a strong opinion and know use cases where it should be better to put it in one place and not in another. Here, voting means putting your name in either the column "Votes on 1" or "Votes on 2".
Before browsing the gridforge i'd like to ask few questions: -have you updated the ldap rendering document? Do you plan to do that before the meeting? -was there any update in the examples (svn)? -can you please point me to the DIT figure you propose? Also a complete ldap example would be nice. -Is the DIT of yours similar to the xml structure? If not, what kind of deviations you found necessary in the LDAP rendering? thanks, Balazs
Hi Balazs, A new version of the LDAP rendering document has been loaded. The example LDIF files have been updated to reflect the outcome of the resent discussions. http://forge.ogf.org/svn/repos/glue/LDAP2/examples/ We have created the DIT as a directory structure in subversion http://forge.ogf.org/svn/repos/glue/LDAP2/DIT/ And used a directory browser to create the image for the document although I forgot to add the GLUE2 prefix. The DIT closely follows the XML structure and I am not aware of any exceptions. Speak to you later Laurence Balazs Konya wrote:
Hi,
David Horat wrote:
In order to prepare for the conference we have tomorrow, please take a look at the agenda: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMe...
Moreover, in order to decide where to put foreign keys, I have created a table in the wiki so we can vote and discuss in the phone conference: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDAPFor...
Please, vote if you have a strong opinion and know use cases where it should be better to put it in one place and not in another. Here, voting means putting your name in either the column "Votes on 1" or "Votes on 2".
Before browsing the gridforge i'd like to ask few questions:
-have you updated the ldap rendering document? Do you plan to do that before the meeting?
-was there any update in the examples (svn)?
-can you please point me to the DIT figure you propose? Also a complete ldap example would be nice.
-Is the DIT of yours similar to the xml structure? If not, what kind of deviations you found necessary in the LDAP rendering?
thanks, Balazs _______________________________________________ glue-wg mailing list glue-wg@ogf.org http://www.ogf.org/mailman/listinfo/glue-wg
hi Laurence, others, Laurence Field wrote:
A new version of the LDAP rendering document has been loaded.
The example LDIF files have been updated to reflect the outcome of the resent discussions. http://forge.ogf.org/svn/repos/glue/LDAP2/examples/ We have created the DIT as a directory structure in subversion http://forge.ogf.org/svn/repos/glue/LDAP2/DIT/ And used a directory browser to create the image for the document although I forgot to add the GLUE2 prefix. The DIT closely follows the XML structure and I am not aware of any exceptions.
thanks for the documents/info. I have started to read them and may come with comments later.
Speak to you later
unfortunately, it turned out that i can't join this afternoon's meeting :( bye, Balazs
Hi Laurence, I will also not be able to join the telecon. I briefly reviewed the new DIT. The only comment that I've is about the Resource which, in the XML, si child of the Manager. This is because, by the schema, we defined that a resource must have one and only one manager. There was a bit of discussion last telecon on this, but in general we do not have use cases of resources without managers or of resources with multiple managers. Cheers, Sergio Laurence Field ha scritto:
Hi Balazs,
A new version of the LDAP rendering document has been loaded.
The example LDIF files have been updated to reflect the outcome of the resent discussions. http://forge.ogf.org/svn/repos/glue/LDAP2/examples/ We have created the DIT as a directory structure in subversion http://forge.ogf.org/svn/repos/glue/LDAP2/DIT/ And used a directory browser to create the image for the document although I forgot to add the GLUE2 prefix. The DIT closely follows the XML structure and I am not aware of any exceptions.
Speak to you later
Laurence
Balazs Konya wrote:
Hi,
David Horat wrote:
In order to prepare for the conference we have tomorrow, please take a look at the agenda: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/PhoneMe...
Moreover, in order to decide where to put foreign keys, I have created a table in the wiki so we can vote and discuss in the phone conference: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDAPFor...
Please, vote if you have a strong opinion and know use cases where it should be better to put it in one place and not in another. Here, voting means putting your name in either the column "Votes on 1" or "Votes on 2".
Before browsing the gridforge i'd like to ask few questions:
-have you updated the ldap rendering document? Do you plan to do that before the meeting?
-was there any update in the examples (svn)?
-can you please point me to the DIT figure you propose? Also a complete ldap example would be nice.
-Is the DIT of yours similar to the xml structure? If not, what kind of deviations you found necessary in the LDAP rendering?
thanks, Balazs _______________________________________________ glue-wg mailing list glue-wg@ogf.org http://www.ogf.org/mailman/listinfo/glue-wg
_______________________________________________ glue-wg mailing list glue-wg@ogf.org http://www.ogf.org/mailman/listinfo/glue-wg
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Sergio Andreozzi said: There was a bit of discussion last telecon on this, but in general we do not have use cases of resources without managers or of resources with multiple managers.
Actually for storage systems I think we potentially have all possible cases. We definitely have the case of a manager without a resource because we nearly didn't define Datastore at all! (And for LCG we still have to see if we want to, or indeed can, publish it.) In the other direction, a classic SE (bare disk server) could have a Datastore (describing the server) but no manager because there is no management software in that case. And we also do have the case of multiple managers (e.g. castor has several different software components), but we just decided to be pragmatic there and not allow a hierarchy of managers on grounds of complexity. In principle we could say that you always have to publish at least a dummy Manager if there is a Resource, but I'm not sure if there's a good reason for it. Stephen -- Scanned by iCritical.
On Mon, 25 May 2009, David Horat wrote:
Moreover, in order to decide where to put foreign keys, I have created a table in the wiki so we can vote and discuss in the phone conference: http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDAPFor...
Please, vote if you have a strong opinion and know use cases where it should be better to put it in one place and not in another. [...]
FYI, for GLUE 1.3 this document was prepared by Flavia and others, showing the LDAP queries relevant for storage, in particular SRM v2.2: http://forge.gridforum.org/sf/go/doc14619?nav=1 It can be reached from the Documents tab: Background --> Info models & schemas For some reason the icon suggests the document is not a PDF file, but it is. We also could do with a set of LDAP queries relevant to the CE. The WMS does a kind of super query "give me anything potentially relevant", converts the result into its own image of the info system, the Information Super Market, built out of ClassAds, and lets the ClassAd library do the matchmaking, AFAIK; still, the placement of the foreign keys presumably could affect the efficiency there as well.
We also could do with a set of LDAP queries relevant to the CE.
Maybe worth looking at lcg-info/lcg-infosites ... http://jra1mw.cvs.cern.ch:8180/cgi-bin/jra1mw.cgi/lcg-info-reader/src/lc g-info?revision=1.12&view=markup http://jra1mw.cvs.cern.ch:8180/cgi-bin/jra1mw.cgi/lcg-infosites/src/lcg- infosites?revision=1.3&view=markup Stephen -- Scanned by iCritical.
glue-wg-bounces@ogf.org
[mailto:glue-wg-bounces@ogf.org] On Behalf Of Maarten.Litmaath@cern.ch said: The WMS does a kind of super query "give me anything potentially relevant",
See this bug: https://savannah.cern.ch/bugs/?func=detailitem&item_id=33103 Stephen -- Scanned by iCritical.
glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of David Horat said:
Moreover, in order to decide where to put foreign keys, I have created a table in the wiki so we can vote and discuss in the phone conference:
http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDA PForeignKeysDiscussion Did we decide how we want to do this? A few comments on my approach to this ... as I said in the meeting it seems obvious to me that the natural relation for Extension is to have the keys in Extension. Actually Extension is a special case - for one thing you can in fact only do it that way since Extension only has a LocalID! Also, I think the fact that we renamed EntityID also gets reflected here, i.e. instead of having one attribute GLUE2ExtensionEntityForeignKey we'll have several like GLUE2ExtensionServiceForeignKey, GLUE2ExtensionDomainForeignKey, ...? Anyway, some general arguments are: 1) Multiplicity - better to have one key than many, or few than many, especially if the many could be very many (dozens or more). Large numbers of keys will be harder and probably more inefficient to process when you extract them (see below). 2) Ease of writing info providers - in general it's easier to create a higher-level object first and then pass its ID in to the code which creates lower-level entities. In some cases it may be impossible to do it the other way around because the code which generates the objects (especially Domains) can't know about the lower-level objects. In some cases, e.g. the computing-storage service relations, you'll be stuck with hard-coding IDs somewhere; in that case I think the relation should point "out", e.g. for ToStorageService you point from the CS to the SS. 3) Likely queries. Bear in mind that queries can work in two ways - if you know an object ID you can query objects which reference it, or you can query the object itself for its key(s). By and large the first one is more efficient, potentially you can gather all the information you want in one query. As a concrete example, suppose I want to get the Path for all StorageShares with a given Tag in a given StorageService (i.e. I know the ServiceID). One way (leaving out the GLUE2 prefix): (&(objectclass=StorageShare)(ShareServiceForeignKey=sss)(StorageShareTag =MCDISK)) StorageSharePath Other way: (&(objectclass=StorageService)(ServiceID=sss)) ServiceShareForeignKey which returns numerous keys, so I either then do lots of queries like (&(objectclass=StorageShare)(ShareID=key1)(StorageShareTag=MCDISK)) StorageSharePath which in many cases will return nothing, or I construct one huge query like (&(objectclass=StorageShare)(|(ShareID=key1)(ShareID=key2)(ShareID=key3) ...)(StorageShareTag=MCDISK)) StorageSharePath I think the first one is clearly preferable! Of course I could find a query that goes the other way, e.g. given a ShareID find the Capability(s) of the corresponding Service, but in this case that's a much less likely thing to want to do, and anyway since a Share only belongs to one Service you have less complexity in the second query. Stephen -- Scanned by iCritical.
On Thu, May 28, 2009 at 1:05 PM, Burke, S (Stephen) < stephen.burke@stfc.ac.uk> wrote:
glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On Behalf Of David Horat said:
Moreover, in order to decide where to put foreign keys, I have created a table in the wiki so we can vote and discuss in the phone conference:
http://forge.gridforum.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDA PForeignKeysDiscussion
Did we decide how we want to do this?
A few comments on my approach to this ... as I said in the meeting it seems obvious to me that the natural relation for Extension is to have the keys in Extension. Actually Extension is a special case - for one thing you can in fact only do it that way since Extension only has a LocalID! Also, I think the fact that we renamed EntityID also gets reflected here, i.e. instead of having one attribute GLUE2ExtensionEntityForeignKey we'll have several like GLUE2ExtensionServiceForeignKey, GLUE2ExtensionDomainForeignKey, ...?
Anyway, some general arguments are:
1) Multiplicity - better to have one key than many, or few than many, especially if the many could be very many (dozens or more). Large numbers of keys will be harder and probably more inefficient to process when you extract them (see below).
I agree.
2) Ease of writing info providers - in general it's easier to create a higher-level object first and then pass its ID in to the code which creates lower-level entities. In some cases it may be impossible to do it the other way around because the code which generates the objects (especially Domains) can't know about the lower-level objects. In some cases, e.g. the computing-storage service relations, you'll be stuck with hard-coding IDs somewhere; in that case I think the relation should point "out", e.g. for ToStorageService you point from the CS to the SS.
Yes. Here the idea could also be, having a DIT, to try to put the ForeignKeys in the children, because usually you would create first the parent. I was discussing this with Laurence right now.
3) Likely queries. Bear in mind that queries can work in two ways - if you know an object ID you can query objects which reference it, or you can query the object itself for its key(s). By and large the first one is more efficient, potentially you can gather all the information you want in one query. As a concrete example, suppose I want to get the Path for all StorageShares with a given Tag in a given StorageService (i.e. I know the ServiceID). One way (leaving out the GLUE2 prefix):
(&(objectclass=StorageShare)(ShareServiceForeignKey=sss)(StorageShareTag =MCDISK)) StorageSharePath
Other way:
(&(objectclass=StorageService)(ServiceID=sss)) ServiceShareForeignKey
which returns numerous keys, so I either then do lots of queries like
(&(objectclass=StorageShare)(ShareID=key1)(StorageShareTag=MCDISK)) StorageSharePath
which in many cases will return nothing, or I construct one huge query like
(&(objectclass=StorageShare)(|(ShareID=key1)(ShareID=key2)(ShareID=key3) ...)(StorageShareTag=MCDISK)) StorageSharePath
I think the first one is clearly preferable! Of course I could find a query that goes the other way, e.g. given a ShareID find the Capability(s) of the corresponding Service, but in this case that's a much less likely thing to want to do, and anyway since a Share only belongs to one Service you have less complexity in the second query.
I agree and here is where your expertise comes from the use cases that you know and I don't. Thus, please vote! :D http://forge.ogf.org/sf/wiki/do/viewPage/projects.glue-wg/wiki/LDAPForeignKe...
Stephen
-- Scanned by iCritical.
-- David Horat Software Engineer – IT/GD – Grid Deployment Group CERN – European Organization for Nuclear Research » Where the web was born Address: 1211 Geneva - Switzerland, Office: 28/R-003 Phone +41 22 76 77996 Fax +41 22 76 68178 (fax to email service) Web: http://cern.ch/horat Web: http://davidhorat.com/ Profile: http://linkedin.com/in/davidhorat
participants (6)
-
Balazs Konya -
Burke, S (Stephen) -
David Horat -
Laurence Field -
Maarten.Litmaath＠cern.ch -
Sergio Andreozzi