Hi all, I couldn't follow the end of the meeting as suddenly I couldn't hear you, I guess it's because Shiraz dropped out of the server. I hope we will manage to find an agreement to close the LDAP document. I think me, Stephen and a mediator and whoever wants to follow the proceedings should sat down in front of the document and accept-reject changes. In this way we can also quickly show how things are done in reality and comment about it. As I said, since the integration effort was done during EMI, the implemented technology follows almost all the lines described in that the July 2012 review. And we tried to stick to the real implementation as much as possible. So I don't understand Stephen's fear that current implementations do not follow these guidelines. They actually do! I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document. I am in for some follow-up meeting to complete Stephen's slides, but mind that I sent a couple of slides with ARC's view on things. At the cost of repeating myself, Stephen slides are the the best summary I've seen about what happened in LDAP so far, as they focused discussion on real problems we had to face during integration. Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
Hi Florido, Stephen would you mind sending your slide decks (or pointers) around again? I guess they have been on this list in the past, but I didn't pay attention I guess... - apologies! Also:
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
If this turns out to be the crux of the problem, then separating that out might indeed be an easy option... My $0.02, Andre. On Tue, Sep 17, 2013 at 6:08 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi all,
I couldn't follow the end of the meeting as suddenly I couldn't hear you, I guess it's because Shiraz dropped out of the server.
I hope we will manage to find an agreement to close the LDAP document. I think me, Stephen and a mediator and whoever wants to follow the proceedings should sat down in front of the document and accept-reject changes. In this way we can also quickly show how things are done in reality and comment about it.
As I said, since the integration effort was done during EMI, the implemented technology follows almost all the lines described in that the July 2012 review. And we tried to stick to the real implementation as much as possible. So I don't understand Stephen's fear that current implementations do not follow these guidelines. They actually do!
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
I am in for some follow-up meeting to complete Stephen's slides, but mind that I sent a couple of slides with ARC's view on things. At the cost of repeating myself, Stephen slides are the the best summary I've seen about what happened in LDAP so far, as they focused discussion on real problems we had to face during integration.
Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ================================================== _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- Nothing is really difficult.
Hello Andre, The slides from yesterday's sessions are here: http://redmine.ogf.org/dmsf/glue-wg?folder_id=6558 Thanks for participating in the sessions yesterday. JP On Sep 18, 2013, at 1:03 AM, Andre Merzky <andre@merzky.net> wrote:
Hi Florido, Stephen
would you mind sending your slide decks (or pointers) around again? I guess they have been on this list in the past, but I didn't pay attention I guess... - apologies!
Also:
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
If this turns out to be the crux of the problem, then separating that out might indeed be an easy option...
My $0.02,
Andre.
On Tue, Sep 17, 2013 at 6:08 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi all,
I couldn't follow the end of the meeting as suddenly I couldn't hear you, I guess it's because Shiraz dropped out of the server.
I hope we will manage to find an agreement to close the LDAP document. I think me, Stephen and a mediator and whoever wants to follow the proceedings should sat down in front of the document and accept-reject changes. In this way we can also quickly show how things are done in reality and comment about it.
As I said, since the integration effort was done during EMI, the implemented technology follows almost all the lines described in that the July 2012 review. And we tried to stick to the real implementation as much as possible. So I don't understand Stephen's fear that current implementations do not follow these guidelines. They actually do!
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
I am in for some follow-up meeting to complete Stephen's slides, but mind that I sent a couple of slides with ARC's view on things. At the cost of repeating myself, Stephen slides are the the best summary I've seen about what happened in LDAP so far, as they focused discussion on real problems we had to face during integration.
Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ================================================== _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- Nothing is really difficult. _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
Florido, Sorry the phone connection failed. I think you, Stephen, and the working group chairs (myself and/or Shiraz) should go thru the document, as you suggest, and review all the changes, but first... So that we can produce a concise public summary of the changes, I would like for us to agree on two things: 1) are the proposed changes in Stephen's slides complete and accurate? 2) for you and Stephen to classify each change into a) good idea, b) neutral (no clear benefit and no harm), and c) not a good idea. We can do this during our next working group teleconference after Stephen presents the rest of his slides. Before that next teleconference, I'd like to request that you reply to this working group with the following information about the proposed changes that you know are contentious: DIT/insertion points and perhaps GLUE2GroupID. - What role needs or benefits from this information in LDAP rendering specification: the user, the information provider, the BDII/ldap administrators? Florido and Stephen, Are you available the next two GLUE WG teleconference dates September 24 or October 6 at 4 PM CET (9 AM CST) so that Stephen can present the rest of his slides? Thanks, JP On Sep 17, 2013, at 6:08 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi all,
I couldn't follow the end of the meeting as suddenly I couldn't hear you, I guess it's because Shiraz dropped out of the server.
I hope we will manage to find an agreement to close the LDAP document. I think me, Stephen and a mediator and whoever wants to follow the proceedings should sat down in front of the document and accept-reject changes. In this way we can also quickly show how things are done in reality and comment about it.
As I said, since the integration effort was done during EMI, the implemented technology follows almost all the lines described in that the July 2012 review. And we tried to stick to the real implementation as much as possible. So I don't understand Stephen's fear that current implementations do not follow these guidelines. They actually do!
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
I am in for some follow-up meeting to complete Stephen's slides, but mind that I sent a couple of slides with ARC's view on things. At the cost of repeating myself, Stephen slides are the the best summary I've seen about what happened in LDAP so far, as they focused discussion on real problems we had to face during integration.
Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ================================================== _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
Dear all,
for us to agree on two things: 1) are the proposed changes in Stephen's slides complete and accurate? 2) for you and Stephen to classify each change into a) good idea, b) neutral (no clear benefit and no harm), and c) not a good idea.
I would just like to mention here that even if for me all these changes are c), we already discussed all of them with Florido and Balasz one year ago in the context of the EMI project. Then we agreed to implement the changes in the BDII to integrate ARC resources. So currently Florido´s requests are available in the BDII and they coexist with the current GLUE 2 specification without problems: - Both GLUE2GroupID=resource and GLUE2GroupID=services can be used - Both GLUE2GroupID and GLUE2GroupName can be used This seems to be working fine in production (it´s probably not available in all BDIIs yet, since this is only available from version 5.2.17 or higher and some sites still run older versions). I also want to mention that getting rid of GLUE2GroupID=resource in EGI/WLCG implies a lot of effort and has a very big impact in many different things: information providers, clients, applications interacting with the BDII, BDII internal code... So it´s not an option. I´m afraid we need to be backwards compatible here. And in any case, changing this has such a big impact that I think none of us is entitled to take this decision, we would need to escalate this to EGI and WLCG management.
that you know are contentious: DIT/insertion points and perhaps GLUE2GroupID. - What role needs or benefits from this information in LDAP rendering specification: the user, the information provider, the BDII/ldap administrators?
The changes are not needed or represent any benefits to any middleware service running a resource BDII. As I mentioned before, the motivation to implement these changes was to integrate ARC resources in the BDII. I understand that ARC now wants to push these changes also in the GLUE 2 specification, but we have to make sure we are not breaking anything in the current production infrastructure. I wonder whether we can agree on a solution that maintains the current specification and maybe extends it to include Florido´s changes? Regards, Maria
On Sep 18, 2013, at 3:02 PM, Maria Alandes Pradillo <Maria.Alandes.Pradillo@cern.ch> wrote:
I wonder whether we can agree on a solution that maintains the current specification and maybe extends it to include Florido´s changes?
Maintaining compatibility with the current implementation and specification is an important goal. Even if we can do that, there is higher level debate about what should be in the specification, for example: There are BDII/LDAP deployment and configuration designs that are *necessary* for infrastructure publishers and operators, but *not*visible* to the users of one or multiple interoperable information systems. So the question is "who needs to know what the DIT is" and "who needs to know what the insertion points are"? If the answer is that *only* infrastructure publishers and operators, and NOT users, then the DIT doesn't belong in the LDAP rendering specification, but in a profile or community practices document. I believe this is the key question regarding DITs. JP
Hi All, JP On 2013-09-18 17:31, JP Navarro wrote:
On Sep 18, 2013, at 3:02 PM, Maria Alandes Pradillo <Maria.Alandes.Pradillo@cern.ch> wrote:
I wonder whether we can agree on a solution that maintains the current specification and maybe extends it to include Florido´s changes?
I hope so, this is why we are proposing to move the DIT section to some appendix/example of community implementation.
Maintaining compatibility with the current implementation and specification is an important goal.
Even if we can do that, there is higher level debate about what should be in the specification, for example:
There are BDII/LDAP deployment and configuration designs that are *necessary* for infrastructure publishers and operators, but *not*visible* to the users of one or multiple interoperable information systems.
So the question is "who needs to know what the DIT is" and "who needs to know what the insertion points are"?
If the answer is that *only* infrastructure publishers and operators, and NOT users, then the DIT doesn't belong in the LDAP rendering specification, but in a profile or community practices document.
I am not sure what you mean by users here. I consider users both developers and consumers of information. I didn't know we write OGF standards only for people who wants to consume information and not to produce it, this suprises me a bit. I thought OGF was something like IEEE actually... if defines standards.
I believe this is the key question regarding DITs.
I already said on top of this email and in another email that we can put these "community practices" as a list of examples in the appendix. We don't need to suggest or mandate a tree in the document, so we get closer to a final draft. Anyhow, since you asked who-benefits-of-what, I just want to argument a bit about this tree structure again. Coming back to my "file in a directory" example, knowing the absolute path of a file means one can list it directly, doesn't have to run a find for it. Knowing a *part* of the absolute path also means that one start your find from that path and not on the entire filesystem, which also gives you a performance benefit. knowing the DIT means knowing EXACTLY where an object is in the tree, like absolute paths. Knowing part of the dn is equivalent to knowing part on an absolute path, so it helps reducing search complexity. This said, usually one runs generic queries on LDAP and doesn't care about the dn. Also, Stephen showed in previous slide that the performance is already very much acceptable for common queries. However, it is clear that if a consumer (or user) *knows* the exact dn of an object, she can retrieve that object in O(1). If not, she has to run a search, and the search performance depends on the complexity of the query. This is basically why infrastructure publishers and operators based their implementation on minimal DIT knowledge. It just makes things easier and faster to code. This is the same reason why we were advocating some kind of structure in XML. As I said thousand times, ldap is a tree-like database, knowing the structure of the tree is *not needed* but *if you do* then you can query faster than if you don't. Moreover, you will have a set of databases structured more or less in the same way, that might even foster harmonization, interoperability and integration between information providers/publishers/generators, elevating GLUE2 to some kind of common datastructure. It's really as simple as this. But I don't want to start this topic again. I just wanted to answer to the who-benefits-of-what question. Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
On Thu, Sep 19, 2013 at 4:26 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
On 2013-09-18 17:31, JP Navarro wrote:
So the question is "who needs to know what the DIT is" and "who needs to know what the insertion points are"?
If the answer is that *only* infrastructure publishers and operators, and NOT users, then the DIT doesn't belong in the LDAP rendering specification, but in a profile or community practices document.
I am not sure what you mean by users here. I consider users both developers and consumers of information. I didn't know we write OGF standards only for people who wants to consume information and not to produce it, this surprises me a bit. I thought OGF was something like IEEE actually... if defines standards.
I can answer this part I think. Obviously, OGF defines standard specifications (or, more precisely, provides a process to interested communities to define standards). More specifically though, OGF supports the definition of standards with support the interoperation of distributes infrastructures. Toward that end, most standards at OGF focus on implementors of software, but in many cases, this is actually not sufficient to provide interoperation -- operational objectives, software configuration, usage policies and security policies are often needed to go from a compliant implementation to practical, usable interoperation. OGF usually encourages to separate those issues -- specifically, we don't expect that operational issues are part of the software (interface, protocol, language etc) specifications. Rather, we encourage to keep specifications focused on a specific interoperability problem, and to allow multiple specs and profiles to be combined for actual interoperable infrastructures. It seems to me that the GLUE discussions are somewhat related to the above -- there seems to be consensus on the GLUE-2 model itself, and on most of the LDAP rendering, but there seems to be contradictory opinions on what parts of the LDAP rendering are actually operational, or implied by usage policies. Sorry for the long blurb -- but what JP tries to attempt is to help to separate those issues -- and that seems indeed be necessary, in order to yield a spec which is applicable to the different existing implementations.
I already said on top of this email and in another email that we can put these "community practices" as a list of examples in the appendix. We don't need to suggest or mandate a tree in the document, so we get closer to a final draft.
This seems sensible, IMHO. If you want to go beyond the 'example' status of the DIT, it might very well fit into a community practice document, or a profile spec (I do not know enough about LDAP to see if the latter makes sense, technically).
Anyhow, since you asked who-benefits-of-what, I just want to argument a bit about this tree structure again.
Coming back to my "file in a directory" example, knowing the absolute path of a file means one can list it directly, doesn't have to run a find for it. Knowing a *part* of the absolute path also means that one start your find from that path and not on the entire filesystem, which also gives you a performance benefit.
knowing the DIT means knowing EXACTLY where an object is in the tree, like absolute paths. Knowing part of the dn is equivalent to knowing part on an absolute path, so it helps reducing search complexity.
This said, usually one runs generic queries on LDAP and doesn't care about the dn. Also, Stephen showed in previous slide that the performance is already very much acceptable for common queries.
However, it is clear that if a consumer (or user) *knows* the exact dn of an object, she can retrieve that object in O(1). If not, she has to run a search, and the search performance depends on the complexity of the query.
This is basically why infrastructure publishers and operators based their implementation on minimal DIT knowledge. It just makes things easier and faster to code. This is the same reason why we were advocating some kind of structure in XML.
As I said thousand times, ldap is a tree-like database, knowing the structure of the tree is *not needed* but *if you do* then you can query faster than if you don't. Moreover, you will have a set of databases structured more or less in the same way, that might even foster harmonization, interoperability and integration between information providers/publishers/generators, elevating GLUE2 to some kind of common datastructure. It's really as simple as this. But I don't want to start this topic again. I just wanted to answer to the who-benefits-of-what question.
Thanks for the clarification! That indeed sounds terribly useful, and I agree on the harmonization, interop and integration part. But also, it sounds like an optimization, and an operational property. Both important of course, no doubt, but, IMHO, very well separable into its own spec document? My $0.02, Andre.
Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ================================================== _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- Nothing is really difficult.
Hi Florido, Just to respond to your argument in light of the comments from Andre, I think that this discussion is irrelevant. When starting the LDAP rendering, it was decided early on that the DIT would not be considered. For WLCG, it was decided that we would not make use of the DIT. The ARC middleware it seems, does make use of a DIT for the reasons that you mention. These are currently implementation choices and creating an interoperability problem (or not according to Maria). In my view (as a former co-chair of the group who started this process), the LDAP rendering should proceede without the DIT and a separate document created on the DIT if needed for interoperability of different implementations. Defining a single DIT is only relvant for those who require this, which does not include WLCG. If there are no other implementations that require agreement on the DIT, I would suggest that you make this an ARC specific specification. Best Regards, Laurence On 19/09/13 16:26, Florido Paganelli wrote:
Hi All, JP
On 2013-09-18 17:31, JP Navarro wrote:
On Sep 18, 2013, at 3:02 PM, Maria Alandes Pradillo <Maria.Alandes.Pradillo@cern.ch> wrote:
I wonder whether we can agree on a solution that maintains the current specification and maybe extends it to include Florido´s changes? I hope so, this is why we are proposing to move the DIT section to some appendix/example of community implementation.
Maintaining compatibility with the current implementation and specification is an important goal.
Even if we can do that, there is higher level debate about what should be in the specification, for example:
There are BDII/LDAP deployment and configuration designs that are *necessary* for infrastructure publishers and operators, but *not*visible* to the users of one or multiple interoperable information systems.
So the question is "who needs to know what the DIT is" and "who needs to know what the insertion points are"?
If the answer is that *only* infrastructure publishers and operators, and NOT users, then the DIT doesn't belong in the LDAP rendering specification, but in a profile or community practices document.
I am not sure what you mean by users here. I consider users both developers and consumers of information. I didn't know we write OGF standards only for people who wants to consume information and not to produce it, this suprises me a bit. I thought OGF was something like IEEE actually... if defines standards.
I believe this is the key question regarding DITs.
I already said on top of this email and in another email that we can put these "community practices" as a list of examples in the appendix. We don't need to suggest or mandate a tree in the document, so we get closer to a final draft.
Anyhow, since you asked who-benefits-of-what, I just want to argument a bit about this tree structure again.
Coming back to my "file in a directory" example, knowing the absolute path of a file means one can list it directly, doesn't have to run a find for it. Knowing a *part* of the absolute path also means that one start your find from that path and not on the entire filesystem, which also gives you a performance benefit.
knowing the DIT means knowing EXACTLY where an object is in the tree, like absolute paths. Knowing part of the dn is equivalent to knowing part on an absolute path, so it helps reducing search complexity.
This said, usually one runs generic queries on LDAP and doesn't care about the dn. Also, Stephen showed in previous slide that the performance is already very much acceptable for common queries.
However, it is clear that if a consumer (or user) *knows* the exact dn of an object, she can retrieve that object in O(1). If not, she has to run a search, and the search performance depends on the complexity of the query.
This is basically why infrastructure publishers and operators based their implementation on minimal DIT knowledge. It just makes things easier and faster to code. This is the same reason why we were advocating some kind of structure in XML.
As I said thousand times, ldap is a tree-like database, knowing the structure of the tree is *not needed* but *if you do* then you can query faster than if you don't. Moreover, you will have a set of databases structured more or less in the same way, that might even foster harmonization, interoperability and integration between information providers/publishers/generators, elevating GLUE2 to some kind of common datastructure. It's really as simple as this. But I don't want to start this topic again. I just wanted to answer to the who-benefits-of-what question.
Cheers, Florido
Hi Laurence, Thanks for your contribution, as I didn't know the "story so far" On 2013-09-20 16:11, Laurence wrote:> Hi Florido,
Just to respond to your argument in light of the comments from Andre, I think that this discussion is irrelevant.
When starting the LDAP rendering, it was decided early on that the DIT would not be considered.
For WLCG, it was decided that we would not make use of the DIT.
The ARC middleware it seems, does make use of a DIT for the reasons that you mention.
These are currently implementation choices and creating an interoperability problem (or not according to Maria).
I see. Differences in implementations due to fuzzy specification/specification not targeted for developers lead to an "interoperability problem". I think we wanted to solve such interoperability issues with GLUE2, but it's clear I misunderstood all of it. My bad.
In my view (as a former co-chair of the group who started this process), the LDAP rendering should proceede without the DIT and a separate document created on the DIT if needed for interoperability of different implementations.
Defining a single DIT is only relvant for those who require this, which does not include WLCG. If there are no other implementations that require agreement on the DIT, I would suggest that you make this an ARC specific specification.
As I said many times, I fail to understand the above for the sake of interoperability and performance, but if this is the feeling the group shares, I have no other choice than to accept it. Regards, Florido
Best Regards,
Laurence
On 19/09/13 16:26, Florido Paganelli wrote:
Hi All, JP
On 2013-09-18 17:31, JP Navarro wrote:
On Sep 18, 2013, at 3:02 PM, Maria Alandes Pradillo <Maria.Alandes.Pradillo@cern.ch> wrote:
I wonder whether we can agree on a solution that maintains the current specification and maybe extends it to include Florido´s changes? I hope so, this is why we are proposing to move the DIT section to some appendix/example of community implementation.
Maintaining compatibility with the current implementation and specification is an important goal.
Even if we can do that, there is higher level debate about what should be in the specification, for example:
There are BDII/LDAP deployment and configuration designs that are *necessary* for infrastructure publishers and operators, but *not*visible* to the users of one or multiple interoperable information systems.
So the question is "who needs to know what the DIT is" and "who needs to know what the insertion points are"?
If the answer is that *only* infrastructure publishers and operators, and NOT users, then the DIT doesn't belong in the LDAP rendering specification, but in a profile or community practices document.
I am not sure what you mean by users here. I consider users both developers and consumers of information. I didn't know we write OGF standards only for people who wants to consume information and not to produce it, this suprises me a bit. I thought OGF was something like IEEE actually... if defines standards.
I believe this is the key question regarding DITs.
I already said on top of this email and in another email that we can put these "community practices" as a list of examples in the appendix. We don't need to suggest or mandate a tree in the document, so we get closer to a final draft.
Anyhow, since you asked who-benefits-of-what, I just want to argument a bit about this tree structure again.
Coming back to my "file in a directory" example, knowing the absolute path of a file means one can list it directly, doesn't have to run a find for it. Knowing a *part* of the absolute path also means that one start your find from that path and not on the entire filesystem, which also gives you a performance benefit.
knowing the DIT means knowing EXACTLY where an object is in the tree, like absolute paths. Knowing part of the dn is equivalent to knowing part on an absolute path, so it helps reducing search complexity.
This said, usually one runs generic queries on LDAP and doesn't care about the dn. Also, Stephen showed in previous slide that the performance is already very much acceptable for common queries.
However, it is clear that if a consumer (or user) *knows* the exact dn of an object, she can retrieve that object in O(1). If not, she has to run a search, and the search performance depends on the complexity of the query.
This is basically why infrastructure publishers and operators based their implementation on minimal DIT knowledge. It just makes things easier and faster to code. This is the same reason why we were advocating some kind of structure in XML.
As I said thousand times, ldap is a tree-like database, knowing the structure of the tree is *not needed* but *if you do* then you can query faster than if you don't. Moreover, you will have a set of databases structured more or less in the same way, that might even foster harmonization, interoperability and integration between information providers/publishers/generators, elevating GLUE2 to some kind of common datastructure. It's really as simple as this. But I don't want to start this topic again. I just wanted to answer to the who-benefits-of-what question.
Cheers, Florido
_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
Hi Florido, The question is basically flat vs nested, similar to the XML, JSON discussions. As I mentioned in the previous mail, the decision at the time was to go with flat, i.e. not to make any assumption about the DIT. I agree that this seems a little strange, as LDAP is naturally hierarchical, but if you have to interoperate with flat XML or JSON information, you would will also have an interoperability problem. So this is more a discussion about flat vs nested. If there was an abstract nested model, then the LDAP DIT would naturally follow this. I have not been following the recent developments but as far as I understand, the predominant view is to go for a flat renderings. Laurence On 23/09/13 19:50, Florido Paganelli wrote:
Hi Laurence,
Thanks for your contribution, as I didn't know the "story so far"
On 2013-09-20 16:11, Laurence wrote:> Hi Florido,
Just to respond to your argument in light of the comments from Andre, I think that this discussion is irrelevant.
When starting the LDAP rendering, it was decided early on that the DIT would not be considered.
For WLCG, it was decided that we would not make use of the DIT.
The ARC middleware it seems, does make use of a DIT for the reasons that you mention.
These are currently implementation choices and creating an interoperability problem (or not according to Maria).
I see. Differences in implementations due to fuzzy specification/specification not targeted for developers lead to an "interoperability problem".
I think we wanted to solve such interoperability issues with GLUE2, but it's clear I misunderstood all of it. My bad.
In my view (as a former co-chair of the group who started this process), the LDAP rendering should proceede without the DIT and a separate document created on the DIT if needed for interoperability of different implementations.
Defining a single DIT is only relvant for those who require this, which does not include WLCG. If there are no other implementations that require agreement on the DIT, I would suggest that you make this an ARC specific specification.
As I said many times, I fail to understand the above for the sake of interoperability and performance, but if this is the feeling the group shares, I have no other choice than to accept it.
Regards, Florido
Best Regards,
Laurence
On 19/09/13 16:26, Florido Paganelli wrote:
Hi All, JP
On 2013-09-18 17:31, JP Navarro wrote:
On Sep 18, 2013, at 3:02 PM, Maria Alandes Pradillo <Maria.Alandes.Pradillo@cern.ch> wrote:
I wonder whether we can agree on a solution that maintains the current specification and maybe extends it to include Florido´s changes? I hope so, this is why we are proposing to move the DIT section to some appendix/example of community implementation.
Maintaining compatibility with the current implementation and specification is an important goal.
Even if we can do that, there is higher level debate about what should be in the specification, for example:
There are BDII/LDAP deployment and configuration designs that are *necessary* for infrastructure publishers and operators, but *not*visible* to the users of one or multiple interoperable information systems.
So the question is "who needs to know what the DIT is" and "who needs to know what the insertion points are"?
If the answer is that *only* infrastructure publishers and operators, and NOT users, then the DIT doesn't belong in the LDAP rendering specification, but in a profile or community practices document.
I am not sure what you mean by users here. I consider users both developers and consumers of information. I didn't know we write OGF standards only for people who wants to consume information and not to produce it, this suprises me a bit. I thought OGF was something like IEEE actually... if defines standards.
I believe this is the key question regarding DITs.
I already said on top of this email and in another email that we can put these "community practices" as a list of examples in the appendix. We don't need to suggest or mandate a tree in the document, so we get closer to a final draft.
Anyhow, since you asked who-benefits-of-what, I just want to argument a bit about this tree structure again.
Coming back to my "file in a directory" example, knowing the absolute path of a file means one can list it directly, doesn't have to run a find for it. Knowing a *part* of the absolute path also means that one start your find from that path and not on the entire filesystem, which also gives you a performance benefit.
knowing the DIT means knowing EXACTLY where an object is in the tree, like absolute paths. Knowing part of the dn is equivalent to knowing part on an absolute path, so it helps reducing search complexity.
This said, usually one runs generic queries on LDAP and doesn't care about the dn. Also, Stephen showed in previous slide that the performance is already very much acceptable for common queries.
However, it is clear that if a consumer (or user) *knows* the exact dn of an object, she can retrieve that object in O(1). If not, she has to run a search, and the search performance depends on the complexity of the query.
This is basically why infrastructure publishers and operators based their implementation on minimal DIT knowledge. It just makes things easier and faster to code. This is the same reason why we were advocating some kind of structure in XML.
As I said thousand times, ldap is a tree-like database, knowing the structure of the tree is *not needed* but *if you do* then you can query faster than if you don't. Moreover, you will have a set of databases structured more or less in the same way, that might even foster harmonization, interoperability and integration between information providers/publishers/generators, elevating GLUE2 to some kind of common datastructure. It's really as simple as this. But I don't want to start this topic again. I just wanted to answer to the who-benefits-of-what question.
Cheers, Florido
glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
Hi Laurence, all, Some thoughts about Laurence claims that should help us sort out the issue. On 2013-09-24 15:49, Laurence wrote:
Hi Florido,
The question is basically flat vs nested, similar to the XML, JSON discussions. As I mentioned in the previous mail, the decision at the time was to go with flat, i.e. not to make any assumption about the DIT.
But then I really don't understand why you put a picture of the DIT in the rendering document, in the version before the one me and Balazs reviewed. If you wanted to go flat you shouldn't have suggested any DIT. The truth is that no LDAP implementation can be really flat. Here's the reasons: Even if we move the whole DIT to external documents, the draft will leave these question open both for producers and consumers of information: 1) shall we mandate a tree root or not (i.e. a DN suffix)? (the already discussed o=glue thing) [I think this is good choice] 2) shall we suggest a use of GLUE2GroupID entries? I recall these are NOT part of the GFD147, it's an invention that came with LDAP rendering. [I already motivated why I like structures. no need to add here] In spite of the above, I'd like to stress that we *CANNOT* suggest a complete FLAT tree structure, because if so, we shoudln't use groupings. Introducing a group introduces a dn structure and therefore a DIT. By defining 1) we are already mandating a minimal tree structure. All GLUE2 LDAP entries MUST be child of o=glue entry, like <glue></glue> in XML. 2) needs to be defined somewhat as it doesn't exist in the model, or removed forever. I followed the discussions about flat VS structure, but I think this has little to do with that. In this case it's only about the concept of insertion points, if these are needed or not to be explained, and how to have a GLUE2 LDAP tree live together in a LDAP database. To my view, whatever we do to finalize this document will diverge from actual implementations. This is because the document NEVER reflected any actual implementation.
I agree that this seems a little strange, as LDAP is naturally hierarchical, but if you have to interoperate with flat XML or JSON information, you would will also have an interoperability problem.
So this is more a discussion about flat vs nested. If there was an abstract nested model, then the LDAP DIT would naturally follow this.
You're right, there is no abstract nested model.
I have not been following the recent developments but as far as I understand, the predominant view is to go for a flat renderings.
True; in this case we have to narrow down 'how flat' for the LDAP technology. Cheers, Florido
Laurence
On 23/09/13 19:50, Florido Paganelli wrote:
Hi Laurence,
Thanks for your contribution, as I didn't know the "story so far"
On 2013-09-20 16:11, Laurence wrote:> Hi Florido,
Just to respond to your argument in light of the comments from Andre, I think that this discussion is irrelevant.
When starting the LDAP rendering, it was decided early on that the DIT would not be considered.
For WLCG, it was decided that we would not make use of the DIT.
The ARC middleware it seems, does make use of a DIT for the reasons that you mention.
These are currently implementation choices and creating an interoperability problem (or not according to Maria).
I see. Differences in implementations due to fuzzy specification/specification not targeted for developers lead to an "interoperability problem".
I think we wanted to solve such interoperability issues with GLUE2, but it's clear I misunderstood all of it. My bad.
In my view (as a former co-chair of the group who started this process), the LDAP rendering should proceede without the DIT and a separate document created on the DIT if needed for interoperability of different implementations.
Defining a single DIT is only relvant for those who require this, which does not include WLCG. If there are no other implementations that require agreement on the DIT, I would suggest that you make this an ARC specific specification.
As I said many times, I fail to understand the above for the sake of interoperability and performance, but if this is the feeling the group shares, I have no other choice than to accept it.
Regards, Florido
Best Regards,
Laurence
On 19/09/13 16:26, Florido Paganelli wrote:
Hi All, JP
On 2013-09-18 17:31, JP Navarro wrote:
On Sep 18, 2013, at 3:02 PM, Maria Alandes Pradillo <Maria.Alandes.Pradillo@cern.ch> wrote:
I wonder whether we can agree on a solution that maintains the current specification and maybe extends it to include Florido´s changes? I hope so, this is why we are proposing to move the DIT section to some appendix/example of community implementation.
Maintaining compatibility with the current implementation and specification is an important goal.
Even if we can do that, there is higher level debate about what should be in the specification, for example:
There are BDII/LDAP deployment and configuration designs that are *necessary* for infrastructure publishers and operators, but *not*visible* to the users of one or multiple interoperable information systems.
So the question is "who needs to know what the DIT is" and "who needs to know what the insertion points are"?
If the answer is that *only* infrastructure publishers and operators, and NOT users, then the DIT doesn't belong in the LDAP rendering specification, but in a profile or community practices document.
I am not sure what you mean by users here. I consider users both developers and consumers of information. I didn't know we write OGF standards only for people who wants to consume information and not to produce it, this suprises me a bit. I thought OGF was something like IEEE actually... if defines standards.
I believe this is the key question regarding DITs.
I already said on top of this email and in another email that we can put these "community practices" as a list of examples in the appendix. We don't need to suggest or mandate a tree in the document, so we get closer to a final draft.
Anyhow, since you asked who-benefits-of-what, I just want to argument a bit about this tree structure again.
Coming back to my "file in a directory" example, knowing the absolute path of a file means one can list it directly, doesn't have to run a find for it. Knowing a *part* of the absolute path also means that one start your find from that path and not on the entire filesystem, which also gives you a performance benefit.
knowing the DIT means knowing EXACTLY where an object is in the tree, like absolute paths. Knowing part of the dn is equivalent to knowing part on an absolute path, so it helps reducing search complexity.
This said, usually one runs generic queries on LDAP and doesn't care about the dn. Also, Stephen showed in previous slide that the performance is already very much acceptable for common queries.
However, it is clear that if a consumer (or user) *knows* the exact dn of an object, she can retrieve that object in O(1). If not, she has to run a search, and the search performance depends on the complexity of the query.
This is basically why infrastructure publishers and operators based their implementation on minimal DIT knowledge. It just makes things easier and faster to code. This is the same reason why we were advocating some kind of structure in XML.
As I said thousand times, ldap is a tree-like database, knowing the structure of the tree is *not needed* but *if you do* then you can query faster than if you don't. Moreover, you will have a set of databases structured more or less in the same way, that might even foster harmonization, interoperability and integration between information providers/publishers/generators, elevating GLUE2 to some kind of common datastructure. It's really as simple as this. But I don't want to start this topic again. I just wanted to answer to the who-benefits-of-what question.
Cheers, Florido
glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
_______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
Hi Florido, On 09/24/2013 05:46 PM, Florido Paganelli wrote:
Hi Laurence, all,
Some thoughts about Laurence claims that should help us sort out the issue.
Hi Florido,
The question is basically flat vs nested, similar to the XML, JSON discussions. As I mentioned in the previous mail, the decision at the time was to go with flat, i.e. not to make any assumption about the DIT. But then I really don't understand why you put a picture of the DIT in
On 2013-09-24 15:49, Laurence wrote: the rendering document, in the version before the one me and Balazs reviewed. If you wanted to go flat you shouldn't have suggested any DIT. I don't know who put it in or the reasoning behind it, but agree it should not be in the document as it is confusing rather than helpful.
The truth is that no LDAP implementation can be really flat. Here's the reasons:
Even if we move the whole DIT to external documents, the draft will leave these question open both for producers and consumers of information:
1) shall we mandate a tree root or not (i.e. a DN suffix)? (the already discussed o=glue thing) [I think this is good choice] The DN suffix discussion is part of the access protocol. The o=glue suffix is used to "access" GLUE 2.0 information. It is perfectly reasonable to query using HTTP (access protocol) and return LDIF (data model). The standarization of the access protocol has not started. 2) shall we suggest a use of GLUE2GroupID entries? I recall these are NOT part of the GFD147, it's an invention that came with LDAP rendering. [I already motivated why I like structures. no need to add here]
In spite of the above, I'd like to stress that we *CANNOT* suggest a complete FLAT tree structure, because if so, we shoudln't use groupings. Introducing a group introduces a dn structure and therefore a DIT. Be careful. We do not say that it MUST be flat, we say that you should not assume any specific DIT structure. The grouping entries are there for pragmatic reasons if for whatever reason someone would like to group objects, but we do not care how people do this as the structure should be ignored.
By defining 1) we are already mandating a minimal tree structure. All GLUE2 LDAP entries MUST be child of o=glue entry, like <glue></glue> in XML. No. We are defining part of an access protocol, which is out-of-scope for an information model discussion.
2) needs to be defined somewhat as it doesn't exist in the model, or removed forever. So the question is why is the grouping entry defined if we state that you should not assume any specific DIT structure?
Laurence
Hi Laurence, On 2013-09-24 21:09, Laurence Field wrote:
[...]
2) [GLUE2Group] needs to be defined somewhat as it doesn't exist in the model, or removed forever. So the question is why is the grouping entry defined if we state that you should not assume any specific DIT structure?
Ehr, yes, that was my question. Why was it part of the LDAP spec at all? It had always been there. I guess nobody can really answer this question. From your claims it seems it was just a suggestion for something that has apparently no practical use. Despite the fact that I think it does have a use... but I was not there when you guys wrote that. Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On
Behalf Of Florido Paganelli said: Ehr, yes, that was my question. Why was it part of the LDAP spec at all? It had always been there.
I guess nobody can really answer this question.
I can of course answer it, and have done repeatedly: it was a SUGGESTION. At the time the document was written we had no implementations, so it seemed useful to give some guidance on how someone could (not must) structure the tree. Now we have many implementations and the idea of a non-binding example seems to be regarded as incomprehensible, at least by you, so probably it should be removed to avoid any confusion. Stephen -- Scanned by iCritical.
Hi Stephen, Thanks for the (repeated) clarification. On 2013-09-26 12:58, stephen.burke@stfc.ac.uk wrote:
glue-wg-bounces@ogf.org [mailto:glue-wg-bounces@ogf.org] On
Behalf Of Florido Paganelli said: Ehr, yes, that was my question. Why was it part of the LDAP spec at all? It had always been there.
I guess nobody can really answer this question.
I can of course answer it, and have done repeatedly: it was a SUGGESTION. At the time the document was written we had no implementations, so it seemed useful to give some guidance on how someone could (not must) structure the tree. Now we have many implementations and the idea of a non-binding example seems to be regarded as incomprehensible, at least by you, so probably it should be removed to avoid any confusion.
I just think that it was a good idea, and some implementations that we nowadays have rely on that, so I don't manage to understand why it sounds that we want to *remove* it if it had been used. Whether it had been used succesfully or not, might be a matter of interest. But some members of the group sees the above as a matter of good practices and not realization specification. I am just trying to reason about the facts so we finalize this document. Sorry for being pedantic. Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
On Sep 26, 2013, at 7:32 AM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
I just think that it was a good idea, and some implementations that we nowadays have rely on that, so I don't manage to understand why it sounds that we want to *remove* it if it had been used.
Again. To "Rely on (DIT)" does NOT mean it's necessary in the rendering specification. Can you answer these questions? 1) could different federation adopt our GLUE2 LDAP rendering specification with different DITs where users can discover information across federations (two separate LDAP queries)? 2) could these federations cross-aggregate their LDAP information given different DITs? I assume they would need to know each other's DITs to know injection points (one perspective is that this is an operational issue and not a LDAP specification issue). JP
Dear JP,
1) could different federation adopt our GLUE2 LDAP rendering specification with different DITs where users can discover information across federations (two separate LDAP queries)?
Yes, it is the case now. Some ARC resources have a slightly different DIT that the rest of resources in the EGI infrastructure. You can discover information in both cases using slightly different queries.
2) could these federations cross-aggregate their LDAP information given different DITs? I assume they would need to know each other's DITs to know injection points (one perspective is that this is an operational issue and not a LDAP specification issue).
Yes, they could. We have tried to do it but failed so far: we needed two queries to be able to get ARC resource information whereas normally with one query we could get information for non ARC resources. We thought that the two queries were fine for both ARC and non ARC but later on discovered that we couldn´t get everything for non ARC so we had rolled back to the first query. I think we need to investigate a bit more, but I don´t think it´s technically impossible. The only problem is that we have to be very careful for not breaking what we currently have in production. Regards, Maria
Hi JP, On 2013-09-26 16:35, JP Navarro wrote:
On Sep 26, 2013, at 7:32 AM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
I just think that it was a good idea, and some implementations that we nowadays have rely on that, so I don't manage to understand why it sounds that we want to *remove* it if it had been used.
Again. To "Rely on (DIT)" does NOT mean it's necessary in the rendering specification.
Can you answer these questions?
I can, I however need to stress that today these questions you ask me are about an imaginary world in my mind and not about reality. Since I cannot predict the future I can evaluate on what I know about the past.
1) could different federation adopt our GLUE2 LDAP rendering specification with different DITs where users can discover information across federations (two separate LDAP queries)?
yes, but integrity of information is delegated to the client. Reason: No structure on the served documents means that all associations and relation between GLUE2 entities have to be computed by the client(s). The clients have to build their own database themselves. On the other hand it will happen for XML with the flat structure, so we're at least consistent on this side :)
2) could these federations cross-aggregate their LDAP information given different DITs? I assume they would need to know each other's DITs to know injection points (one perspective is that this is an operational issue and not a LDAP specification issue).
My istinctive answer would be no, but I'll smooth it to "hardly". Reason: We're still having problems within ARC and gLite after years of operation... even after the integration effort the architectures are having troubles agreeing on things. But maybe it's just the two of us. Interestingly, some of the issues are also on several attribute values, not just the DIT. But I guess these answers were obvious on my side, especially the second one is the whole reason why I do care about this. I'd like to hear other people's opinion on these. Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
JP Navarro [mailto:navarro@mcs.anl.gov] said:
1) could different federation adopt our GLUE2 LDAP rendering specification with different DITs where users can discover information across federations (two separate LDAP queries)?
As long as the queries are constructed in the right way the same queries would work on any implementation with the same schema, regardless of the DIT, with the exception that you need to know the "base DN", i.e. the root of the tree within which you will find the information. That's no harder than knowing the hostname and port of the server, and indeed we often specify them all together with LDAP URLs - for example the full information for the EGI production Grid is available from (among other places) ldap://lcg-bdii.cern.ch:2170/GLUE2GroupID=grid,o=glue. Note that a server running on a different port would be equally "non-interoperable" to anything which has the port hardwired!
2) could these federations cross-aggregate their LDAP information given different DITs? I assume they would need to know each other's DITs to know injection points (one perspective is that this is an operational issue and not a LDAP specification issue).
If you aggregated just by composition, for example you copied the entire trees from two Grids into two subtrees of a single united tree, it would be pretty much trivial. The reason Nordugrid is harder is that we're trying to integrate it at the level of individual sites (or even single services). Still it's basically just a technical issue - but you also have to understand what you expect to get at the end. For example, at the moment the top BDII has everything grouped by site (AdminDomain) so if we want to integrate an infrastructure which doesn't do that we have to make a choice, either restructure the information into that form or accept that the top BDII can have a mixed structure. Stephen -- Scanned by iCritical.
On Sep 23, 2013, at 1:50 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
I see. Differences in implementations due to fuzzy specification/specification not targeted for developers lead to an "interoperability problem".
I think we wanted to solve such interoperability issues with GLUE2, but it's clear I misunderstood all of it. My bad.
Florido, You are totally correct that developer and operator specifications/profiles/configurations may be necessary to implement a distributed interoperable GLUE 2 information systems. The point we're trying to make is that there are different types of specifications/profiles/configurations some of which belong in the rendering document, while others belong elsewhere. For example, in a grid of GLUE2 information systems authentication and authorization of distributed service interactions is critical, BUT that doesn't mean we want that information in the GLUE2 schema specification or the LDAP rendering specification. Specifications that enable a user to query and use LDAP information from multiple infrastructures belongs in the LDAP rendering specification. Specifications that implement a specific LDAP service topology (like EGI's) belong in a community practices document. Other communities may have a different topology, or NO topology with a single central LDAP service. So, we definitely must "solve such interoperability issues with GLUE2", but we need distinguish between problems that our GLUE2 LDAP rendering must solve, and problems that a particular infrastructure must solve, but which other infrastructures may solve differently and still be interoperable. JP
Hi JP, all First of all thanks for this effort of mediating to finish this LDAP document. On 2013-09-18 09:55, JP Navarro wrote:> Florido,
Sorry the phone connection failed.
I think you, Stephen, and the working group chairs (myself and/or
Shiraz) should go thru the document, as you suggest, and review all the changes, but first...
I think Maria or Laurence who are the BDII developers I worked with during EMI should be there as well, if they want, to give a technical point of view. I don't think we should underestimate implementation issues; it's clear from Stephen's talk that existing implementations matter. Trying to shorten this quite painful process, we reasoned with Balazs and we have a proposal. Looking at the document structure, we can move the DIT proposal in an appendix, as "Examples of existing implementations". Then the discussion, and the main topic of the document, will only be about the schema and naming choices, including a clarification of the meaning of GLUE2GroupID. Of course for this we will need another review round, but it's minor changes and we think this is a good compromise with Stephen's claim that we should make the document and the specification "independent" on the DIT. This said, some not-so-short comments to your requests:
So that we can produce a concise public summary of the changes,
As a matter of fact, this was already produced long time ago by me and Balazs. Here's the timeline: 1) An email with title: "Summary of changes in LDAP GLUE2 rendering as requested in last meeting" sent by me September 17th, 2012. that lists the changes in the LDAP Rendering document, and open questions we wanted to be answered. I guess it was not too fine grained, so that's why it failed to get the attention of the group. 2) Reasons why we decided to review the draft and do the integration work in EMI are explained in my OGF36 presentation (9th October 2012): http://redmine.ogf.org/dmsf_files/172?download= 3) For a list of changes of what has been done in practice by the software providers (that is, ARC and gLite), you should refer to a presentation that was due the 26th of February 2013, that I sent to JP and Shiraz with an email with title "GLUE2 LDAP State of the Art" on March the 21st. Since there was no time to present, I created an update of that presentation that I actually presented on the 21st of May 2013, and that is probably somewhere in Shiraz or JP machine. I didn't manage to find it on redmine. I'd be glad to upload it somewhere. As a side note, where do we store meeting's presentations?
I would like for us to agree on two things: 1) are the proposed changes in Stephen's slides complete and accurate?
As Balazs said, they're very gLite-centric. From this point of view I don't consider them that accurate, cause he doesn't have a broad view of what other LDAP GLUE2 implementations are out there (Mainly ARC, UNICORE are those I am better aware of). Therefore I'd like to present my slides as well, to give a hint of what others than gLite did, and argument a little about the issues we faced during integration (already presented in 21st of May 2013 presentation)
2) for you and Stephen to classify each change into a) good idea, b) neutral (no clear benefit and no harm), and c) not a good idea. We can do this during our next working group teleconference after Stephen presents the rest of his slides.
Before that next teleconference, I'd like to request that you reply to
This can be done based on the facts presented in slides I listed above, and the slides I sent to the meeting: http://redmine.ogf.org/dmsf_files/13121?download= this working group with the following information about the proposed changes that you know are contentious: DIT/insertion points and perhaps GLUE2GroupID.
- What role needs or benefits from this information in LDAP rendering specification: the user, the information provider, the BDII/ldap administrators?
It is no point in my opinion to proceed like this. Instead, I know the answer to this question for each of the changes in the LDAP document. I can explain who-benefits-of-what on the fly while we revise the document.
Florido and Stephen,
Are you available the next two GLUE WG teleconference dates September
24 or October 6 at 4 PM CET (9 AM CST) so that Stephen can present the rest of his slides?
I understand Stephen's need to finish expressing his views. In such case then, I'd like to show my slides as well. Thank you for your mediation action! Cheers, Florido
Thanks,
JP
On Sep 17, 2013, at 6:08 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi all,
I couldn't follow the end of the meeting as suddenly I couldn't hear you, I guess it's because Shiraz dropped out of the server.
I hope we will manage to find an agreement to close the LDAP document. I think me, Stephen and a mediator and whoever wants to follow the proceedings should sat down in front of the document and accept-reject changes. In this way we can also quickly show how things are done in reality and comment about it.
As I said, since the integration effort was done during EMI, the implemented technology follows almost all the lines described in that the July 2012 review. And we tried to stick to the real implementation as much as possible. So I don't understand Stephen's fear that current implementations do not follow these guidelines. They actually do!
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
I am in for some follow-up meeting to complete Stephen's slides, but mind that I sent a couple of slides with ARC's view on things. At the cost of repeating myself, Stephen slides are the the best summary I've seen about what happened in LDAP so far, as they focused discussion on real problems we had to face during integration.
Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ================================================== _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
Dear all, ________________________________________ I think Maria or Laurence who are the BDII developers I worked with during EMI should be there as well, if they want, to give a technical point of view. I don't think we should underestimate implementation issues; it's clear from Stephen's talk that existing implementations matter. I'm happy to be there, of course. I have explained the technical detailes in a previous mail in any case. Moreover, I don't forsee any other changes in the BDII. The BDII is currently compatible with ARC, gLite, dCache and Unicore resources as far as GLUE 2 is concerned. I'm not doing any further changes since the system is working and we have already seen that introducing changes in terms of DIT modifications is very risky and may break the production infrastructure (something that by the way happened some weeks ago and we had to roll back a version of the ldap info provider). Whatever it is decided in the meeting has to be compatible with what it is currently implemented. Regards, Maria
Florido,
Trying to shorten this quite painful process, we reasoned with Balazs and we have a proposal. Looking at the document structure, we can move the DIT proposal in an appendix, as "Examples of existing implementations".
OGF has profile and community practice document types that are better suited to hold examples.
As a side note, where do we store meeting's presentations?
http://redmine.ogf.org/dmsf/glue-wg?folder_id=4
Therefore I'd like to present my slides as well, to give a hint of what others than gLite did, and argument a little about the issues we faced during integration (already presented in 21st of May 2013 presentation)
You could present after Stephen finishes his presentation and we finish discussing his material.
It is no point in my opinion to proceed like this. Instead, I know the answer to this question for each of the changes in the LDAP document. I can explain who-benefits-of-what on the fly while we revise the document.
First we need to agree that the purpose of the specification is to enable "discovery interoperability" for users and the software they use. Second, we need to agree that to enable "glue 2 aggregation across publishers" we need a profile or community practice document. EGI, NorduGrid, and others may define different profiles.. Thanks, JP On Sep 19, 2013, at 8:59 AM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi JP, all
First of all thanks for this effort of mediating to finish this LDAP document.
On 2013-09-18 09:55, JP Navarro wrote:> Florido,
Sorry the phone connection failed.
I think you, Stephen, and the working group chairs (myself and/or
Shiraz) should go thru the document, as you suggest, and review all the changes, but first...
I think Maria or Laurence who are the BDII developers I worked with during EMI should be there as well, if they want, to give a technical point of view. I don't think we should underestimate implementation issues; it's clear from Stephen's talk that existing implementations matter.
Trying to shorten this quite painful process, we reasoned with Balazs and we have a proposal. Looking at the document structure, we can move the DIT proposal in an appendix, as "Examples of existing implementations".
Then the discussion, and the main topic of the document, will only be about the schema and naming choices, including a clarification of the meaning of GLUE2GroupID. Of course for this we will need another review round, but it's minor changes and we think this is a good compromise with Stephen's claim that we should make the document and the specification "independent" on the DIT.
This said, some not-so-short comments to your requests:
So that we can produce a concise public summary of the changes,
As a matter of fact, this was already produced long time ago by me and Balazs. Here's the timeline:
1) An email with title: "Summary of changes in LDAP GLUE2 rendering as requested in last meeting" sent by me September 17th, 2012. that lists the changes in the LDAP Rendering document, and open questions we wanted to be answered. I guess it was not too fine grained, so that's why it failed to get the attention of the group.
2) Reasons why we decided to review the draft and do the integration work in EMI are explained in my OGF36 presentation (9th October 2012):
http://redmine.ogf.org/dmsf_files/172?download=
3) For a list of changes of what has been done in practice by the software providers (that is, ARC and gLite), you should refer to a presentation that was due the 26th of February 2013, that I sent to JP and Shiraz with an email with title "GLUE2 LDAP State of the Art" on March the 21st. Since there was no time to present, I created an update of that presentation that I actually presented on the 21st of May 2013, and that is probably somewhere in Shiraz or JP machine. I didn't manage to find it on redmine. I'd be glad to upload it somewhere.
As a side note, where do we store meeting's presentations?
I would like for us to agree on two things: 1) are the proposed changes in Stephen's slides complete and accurate?
As Balazs said, they're very gLite-centric. From this point of view I don't consider them that accurate, cause he doesn't have a broad view of what other LDAP GLUE2 implementations are out there (Mainly ARC, UNICORE are those I am better aware of). Therefore I'd like to present my slides as well, to give a hint of what others than gLite did, and argument a little about the issues we faced during integration (already presented in 21st of May 2013 presentation)
2) for you and Stephen to classify each change into a) good idea, b) neutral (no clear benefit and no harm), and c) not a good idea. We can do this during our next working group teleconference after Stephen presents the rest of his slides.
This can be done based on the facts presented in slides I listed above, and the slides I sent to the meeting:
http://redmine.ogf.org/dmsf_files/13121?download=
Before that next teleconference, I'd like to request that you reply to this working group with the following information about the proposed changes that you know are contentious: DIT/insertion points and perhaps GLUE2GroupID. - What role needs or benefits from this information in LDAP rendering specification: the user, the information provider, the BDII/ldap administrators?
It is no point in my opinion to proceed like this. Instead, I know the answer to this question for each of the changes in the LDAP document. I can explain who-benefits-of-what on the fly while we revise the document.
Florido and Stephen,
Are you available the next two GLUE WG teleconference dates September
24 or October 6 at 4 PM CET (9 AM CST) so that Stephen can present the rest of his slides?
I understand Stephen's need to finish expressing his views. In such case then, I'd like to show my slides as well.
Thank you for your mediation action!
Cheers, Florido
Thanks,
JP
On Sep 17, 2013, at 6:08 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi all,
I couldn't follow the end of the meeting as suddenly I couldn't hear you, I guess it's because Shiraz dropped out of the server.
I hope we will manage to find an agreement to close the LDAP document. I think me, Stephen and a mediator and whoever wants to follow the proceedings should sat down in front of the document and accept-reject changes. In this way we can also quickly show how things are done in reality and comment about it.
As I said, since the integration effort was done during EMI, the implemented technology follows almost all the lines described in that the July 2012 review. And we tried to stick to the real implementation as much as possible. So I don't understand Stephen's fear that current implementations do not follow these guidelines. They actually do!
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
I am in for some follow-up meeting to complete Stephen's slides, but mind that I sent a couple of slides with ARC's view on things. At the cost of repeating myself, Stephen slides are the the best summary I've seen about what happened in LDAP so far, as they focused discussion on real problems we had to face during integration.
Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ================================================== _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
Hi JP, On 2013-09-23 18:17, JP Navarro wrote:
Florido,
Trying to shorten this quite painful process, we reasoned with Balazs and we have a proposal. Looking at the document structure, we can move the DIT proposal in an appendix, as "Examples of existing implementations".
OGF has profile and community practice document types that are better suited to hold examples.
I'll have a look at that, thanks.
As a side note, where do we store meeting's presentations?
I didn't mean OGF slides, these I know, I meant all the other meetings, i.e. various phone meetings and such.
Therefore I'd like to present my slides as well, to give a hint of what others than gLite did, and argument a little about the issues we faced during integration (already presented in 21st of May 2013 presentation)
You could present after Stephen finishes his presentation and we finish discussing his material.
Perfectly OK, these were meant as a follow-up
It is no point in my opinion to proceed like this. Instead, I know the answer to this question for each of the changes in the LDAP document. I can explain who-benefits-of-what on the fly while we revise the document.
First we need to agree that the purpose of the specification is to enable "discovery interoperability" for users and the software they use.
"For users" -- at this point I must say I don't understand the concept of "users" the group have. I must have misunderstood this when I started co-operating, clearly. As I understand better these concepts my mood wrt contributions is changing.
Second, we need to agree that to enable "glue 2 aggregation across publishers" we need a profile or community practice document. EGI, NorduGrid, and others may define different profiles..
Mh, ok. Allow me to be extremely polemic here. I had high hopes for GLUE2 to be a poweful interoperability tool for GRIDs. But after all these discussion I am more and more convinced that we are heading for a big fail, with a plethora of redefinitions of things that one has to read to be interoperable. If we ever wanted to keep it easy, we failed somewhere along the way, IMHO. Let's fix this LDAP thing soon then. Regards, Florido
Thanks,
JP
On Sep 19, 2013, at 8:59 AM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi JP, all
First of all thanks for this effort of mediating to finish this LDAP document.
On 2013-09-18 09:55, JP Navarro wrote:> Florido,
Sorry the phone connection failed.
I think you, Stephen, and the working group chairs (myself and/or
Shiraz) should go thru the document, as you suggest, and review all the changes, but first...
I think Maria or Laurence who are the BDII developers I worked with during EMI should be there as well, if they want, to give a technical point of view. I don't think we should underestimate implementation issues; it's clear from Stephen's talk that existing implementations matter.
Trying to shorten this quite painful process, we reasoned with Balazs and we have a proposal. Looking at the document structure, we can move the DIT proposal in an appendix, as "Examples of existing implementations".
Then the discussion, and the main topic of the document, will only be about the schema and naming choices, including a clarification of the meaning of GLUE2GroupID. Of course for this we will need another review round, but it's minor changes and we think this is a good compromise with Stephen's claim that we should make the document and the specification "independent" on the DIT.
This said, some not-so-short comments to your requests:
So that we can produce a concise public summary of the changes,
As a matter of fact, this was already produced long time ago by me and Balazs. Here's the timeline:
1) An email with title: "Summary of changes in LDAP GLUE2 rendering as requested in last meeting" sent by me September 17th, 2012. that lists the changes in the LDAP Rendering document, and open questions we wanted to be answered. I guess it was not too fine grained, so that's why it failed to get the attention of the group.
2) Reasons why we decided to review the draft and do the integration work in EMI are explained in my OGF36 presentation (9th October 2012):
http://redmine.ogf.org/dmsf_files/172?download=
3) For a list of changes of what has been done in practice by the software providers (that is, ARC and gLite), you should refer to a presentation that was due the 26th of February 2013, that I sent to JP and Shiraz with an email with title "GLUE2 LDAP State of the Art" on March the 21st. Since there was no time to present, I created an update of that presentation that I actually presented on the 21st of May 2013, and that is probably somewhere in Shiraz or JP machine. I didn't manage to find it on redmine. I'd be glad to upload it somewhere.
As a side note, where do we store meeting's presentations?
I would like for us to agree on two things: 1) are the proposed changes in Stephen's slides complete and accurate?
As Balazs said, they're very gLite-centric. From this point of view I don't consider them that accurate, cause he doesn't have a broad view of what other LDAP GLUE2 implementations are out there (Mainly ARC, UNICORE are those I am better aware of). Therefore I'd like to present my slides as well, to give a hint of what others than gLite did, and argument a little about the issues we faced during integration (already presented in 21st of May 2013 presentation)
2) for you and Stephen to classify each change into a) good idea, b) neutral (no clear benefit and no harm), and c) not a good idea. We can do this during our next working group teleconference after Stephen presents the rest of his slides.
This can be done based on the facts presented in slides I listed above, and the slides I sent to the meeting:
http://redmine.ogf.org/dmsf_files/13121?download=
Before that next teleconference, I'd like to request that you reply to this working group with the following information about the proposed changes that you know are contentious: DIT/insertion points and perhaps GLUE2GroupID. - What role needs or benefits from this information in LDAP rendering specification: the user, the information provider, the BDII/ldap administrators?
It is no point in my opinion to proceed like this. Instead, I know the answer to this question for each of the changes in the LDAP document. I can explain who-benefits-of-what on the fly while we revise the document.
Florido and Stephen,
Are you available the next two GLUE WG teleconference dates September
24 or October 6 at 4 PM CET (9 AM CST) so that Stephen can present the rest of his slides?
I understand Stephen's need to finish expressing his views. In such case then, I'd like to show my slides as well.
Thank you for your mediation action!
Cheers, Florido
Thanks,
JP
On Sep 17, 2013, at 6:08 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Hi all,
I couldn't follow the end of the meeting as suddenly I couldn't hear you, I guess it's because Shiraz dropped out of the server.
I hope we will manage to find an agreement to close the LDAP document. I think me, Stephen and a mediator and whoever wants to follow the proceedings should sat down in front of the document and accept-reject changes. In this way we can also quickly show how things are done in reality and comment about it.
As I said, since the integration effort was done during EMI, the implemented technology follows almost all the lines described in that the July 2012 review. And we tried to stick to the real implementation as much as possible. So I don't understand Stephen's fear that current implementations do not follow these guidelines. They actually do!
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
I am in for some follow-up meeting to complete Stephen's slides, but mind that I sent a couple of slides with ARC's view on things. At the cost of repeating myself, Stephen slides are the the best summary I've seen about what happened in LDAP so far, as they focused discussion on real problems we had to face during integration.
Cheers, Florido -- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ================================================== _______________________________________________ glue-wg mailing list glue-wg@ogf.org https://www.ogf.org/mailman/listinfo/glue-wg
-- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
-- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
Let's use the same folder for Teleconference material. I proposed directory names "Meeting yyyy-mm-dd". JP On Sep 23, 2013, at 1:44 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
As a side note, where do we store meeting's presentations?
I didn't mean OGF slides, these I know, I meant all the other meetings, i.e. various phone meetings and such.
Hi JP, On 2013-09-24 21:13, JP Navarro wrote:
Let's use the same folder for Teleconference material.
I proposed directory names "Meeting yyyy-mm-dd".
JP
I think is very good idea. Thanks
On Sep 23, 2013, at 1:44 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
As a side note, where do we store meeting's presentations?
I didn't mean OGF slides, these I know, I meant all the other meetings, i.e. various phone meetings and such.
-- ================================================== Florido Paganelli ARC Middleware Developer - NorduGrid Collaboration System Administrator Lund University Department of Physics Division of Particle Physics BOX118 221 00 Lund Office Tel: 046-2220272 Email: florido.paganelli@REMOVE_THIShep.lu.se Homepage: http://www.hep.lu.se/staff/paganelli ==================================================
On Sep 23, 2013, at 1:44 PM, Florido Paganelli <florido.paganelli@hep.lu.se> wrote:
Mh, ok. Allow me to be extremely polemic here. I had high hopes for GLUE2 to be a poweful interoperability tool for GRIDs. But after all these discussion I am more and more convinced that we are heading for a big fail, with a plethora of redefinitions of things that one has to read to be interoperable.
This seems like a well organized small set of documents me. Only the first three involve OGF. 1) GLUE Specification v. 2.0 (rendering independent data model) 2) GLUE 2 LDAP Rendering Specification 3) Community Practice/Profile Document(s) 4) Infrastructure specific implementation documentation (EGI/XSEDE/etc...) In my opinion, 1) and 2) are huge steps towards interoperability. Are they 100% correct and do they describe every detail necessary to have an interoperable infrastructure? Most likely not. We shouldn't throw away "the good" because we don't have "the perfect". In my view, the fact that both EGI and XSEDE are leveraging GLUE 2 means we already have two successes. Certainly there is more work and more successes to come. JP
Dear Florido,
I really think the only dispute is about the DIT tree. I already said many times I'd like at least implementation examples to be there, but if this is so bad, it can be dropped to some other document.
As far as the BDII is concerned, the DIT tree is described in the Sys admin guide that you can find in: http://gridinfo.web.cern.ch/sites/gridinfo.web.cern.ch/files/EMI_BDII_sysadm... Regards, Maria
participants (7)
-
Andre Merzky -
Florido Paganelli -
JP Navarro -
Laurence -
Laurence Field
-
Maria Alandes Pradillo -
stephen.burke＠stfc.ac.uk