Minutes from the OGF20 sessions
Hi everybody, the minutes (as well as the slides) from the RUS and RUS+UR session at the OGF20 in Manchester are now available on the GridForge in the following folder: http://forge.gridforum.org/sf/go/projects.rus-wg/docman.root.meeting_materia... Please have a look an tell me if something is wrong/missing/too much! Best Regards Gilbert Netzer
Hi Gilbert, hi everybody! Gilbert Netzer wrote:
Hi everybody,
the minutes (as well as the slides) from the RUS and RUS+UR session at the OGF20 in Manchester are now available on the GridForge in the following folder:
http://forge.gridforum.org/sf/go/projects.rus-wg/docman.root.meeting_materia...
Please have a look an tell me if something is wrong/missing/too much!
Very good minutes, very detailed, I can't imagine that something is missing :o) Many of us have been present at the sessions I think we should give those who haven't the possibility to express their opinions such that we can then take final decisions (although most are already now quite clear, but having other point of views can be interesting, so please read the minutes if you couldn't attend the sessions at OGF20!) I've already a few considerations to some of the things we said at the sessions, maybe they can serve as a starting point for a discussion. 1) Comments to the minutes of the joint UR/RUS session: Aggregation of usage: - Most of us think that a separate AUR specification would be most appropriate and allow to contemporarily work on both aggregation and the extension of the UR to resource consumers other than jobs (e.g. for storage accounting, etc). Also, a preliminary proposal has been made that we can start working on (and I'm sure Xiaoyu would take the lead on that as principal editor of the document) - Xiouyu, the version of the proposed AUR spec currently available at http://forge.ogf.org/sf/go/doc13708?nav=1 is quite old (before we modified it to have it more flexible for more use cases), can you provide the UR-WG (and RUS-WG) mailing list with the latest version, so that interested people can have a look at it and start a serious discussion on the topic? - About the name: currently Aggregate Usage Record (AUR), or would the name Summary Usage Record/SUR be more appropriate, since aggregation can also mean just a set of single URs without summing the usage values up? Proposed new properites of UR: - UserFQAN: The FQAN is indeed very well structured (as is the User DN), it is basically a string that indicates the VO and particular VO group a user belongs and has additional fields for the user's role/capabilities within that VO, something like: "/VO[/group][/Role=...[/Capability=...]]", for example: "/atlas/production/Role=admin". For a brief description see Section 2 of http://grid-auth.infn.it/docs/AC-RFC.pdf Note that a VOMS certificate may have multiple FQANs, such that the property should have an unbounded occurence. Since VOMS is widely used (and for example charging might depend on a user's role in a VO) the notion of a user's role should be supported although that need not necessarily be restricted to the VOMS FQAN. Maybe something more flexible like urf:UserRole might be ok (those who don't use VOMS might still be interested in distinguishing between "admins" and "mortal users", UserRole might simply be defined as a string and the exact form of it should then depend on the environment/use case. But that's just an idea. 2) Comments to the minutes of the joint UR/RUS session: [I keep the UR-WG in CC for this, some of you might be interested in what is going on eith RUS; if not, just ignore what is following ;o) ] Full XPath support vs. XPath limitations: - I wouldn't say that XPath queries destroy the performance, although they of course can slow it down, that depends on the complexity and also on the underlying database. Even with a relational DB schema the execution of a query can be awfully slow if an XPath has to be translated into an SQL statement that will cause many tables to be joint. Also, I think the most problematic part of the query is the selection of the requested records within the maybe millions of records in the database (being it relational or XML) and less the question whether, after these records have been found, they should be returned completely or just pieces of them (maybe returning complete records is even worse for the performance, for example if a relational DB is used which means the server will needs to reassemble the complete UR documents before returning them ...?). But I think although we should keep performance issues in mind (nonetheless we can't forsee many problems without having experienced a lot with this), we should not focus too much on that. The question is more whether we want the RUS interface to be completely XPath-compliant or leave the restrictions that are currently in place. I think full compliance is the better choice, above all since we're talking about standards :o) - If XPath will allow to retrieve only pieces/parts of URs: Why should a client need to validate what it got back against the UR spec. If I'm interested only in a list of job IDs than I will check only whether what I got back is a list of job IDs, not wether that list can be validated against the UR. And if I want to do that validation, this is perfectly fine, since I can always ask for entire records that I can then validate against the spec. It is up to the client to check what it gets back (knowing exactly what it wanted back; entire URs for validation, just a list of job IDs, or whatever ...) - I would oppose the idea to define a minimum set of XPath queries that have to be supported by a RUS server, since this interferes with implementation issues. If a RUS server, for example, doesn't want records to be found by UserIdentity than it always should be allowed to refuse the execution (maybe by returning a UserNotAuthorizedFault). If defining such a query in a mimimun set of queries that have to be support, the developer would loose that option. Generally all possible XPath queries should _theoretically_ be support but we need to give the developer the means to refuse their execution for some reasons (UserNotAuthorizedFault, QueryTooComplexFault, InternalServerFault, ...). - Also I think letting the RUS server inform the client about what queries it is willing to execute would be unfeasible since XPath is extremely flexible and listing all possible queries should be quite impossible (immagine all the possible combinations ...). - The idea of using XPath expressions for defining the mandatroy elements that have to be present for a UR to be accepted upon insertion is good since it is the most flexible one, also: if the client queries the RUS server for the mandatory elements it should be straightfoward for the client to test its UR documents against the servers requirements before trying to insert them into the RUS. Those are just a few ideas for starting a discussion. Cheers, Rosario.
Best Regards Gilbert Netzer -- rus-wg mailing list rus-wg@ogf.org http://www.ogf.org/mailman/listinfo/rus-wg
Rosario Michael Piro wrote:
I've already a few considerations to some of the things we said at the sessions, maybe they can serve as a starting point for a discussion.
I put my comments inline where I have them. In other cases I either just agree or otherwise have nothing to say. :-)
- About the name: currently Aggregate Usage Record (AUR), or would the name Summary Usage Record/SUR be more appropriate, since aggregation can also mean just a set of single URs without summing the usage values up?
I think I'd prefer Summary UR. It's much clearer about it's purpose.
Proposed new properites of UR: - UserFQAN: The FQAN is indeed very well structured (as is the User DN), it is basically a string that indicates the VO and particular VO group a [...] Since VOMS is widely used (and for example charging might depend on a user's role in a VO) the notion of a user's role should be supported [...] exact form of it should then depend on the environment/use case. But that's just an idea.
I don't think that it should be an update to the UR itself, since it isn't going to be at all meaningful outside systems that use VOMS (and not all grids do; there are other ways to manage the same general effect). I therefore think it should be an element in its own namespace that VOMS-using grids can use without raising hackles from everyone else.
2) Comments to the minutes of the joint UR/RUS session: [I keep the UR-WG in CC for this, some of you might be interested in what is going on eith RUS; if not, just ignore what is following ;o) ]
Full XPath support vs. XPath limitations: - I wouldn't say that XPath queries destroy the performance, although [...] focus too much on that. The question is more whether we want the RUS interface to be completely XPath-compliant or leave the restrictions that are currently in place. I think full compliance is the better choice, above all since we're talking about standards :o)
Not just that, there are things you can do in particular implementations to improve performance (e.g. managed caches of prepared statements) that mean that there would need to be experimental evidence that *common* queries are slow (even after applying such tricks) for it to be worth chucking out XPath. I think this is something where implementation experience will be needed to determine the best way forward.
- If XPath will allow to retrieve only pieces/parts of URs: Why should a [...] against the spec. It is up to the client to check what it gets back (knowing exactly what it wanted back; entire URs for validation, just a list of job IDs, or whatever ...)
It depends on exactly what you want to support. If you're supporting general XPath (or XQuery) queries, you need a resulting content model that permits mixed unvalidated nodes. If you're only ever using it for determining what (full) records to return (i.e. by only returning those records for which the XPath query matches something) you can apply the type constraint safely.
- Also I think letting the RUS server inform the client about what queries it is willing to execute would be unfeasible since XPath is extremely flexible and listing all possible queries should be quite impossible (immagine all the possible combinations ...).
XPath is a recursive language; it has no theoretic bounds on query size. I'm pretty sure that there's not even a *countably* infinite number of queries. Moreover, this is a standard you're writing; implementations can impose and document such extra restrictions if they choose, but we have no inherent reason to do so ourselves.
- The idea of using XPath expressions for defining the mandatroy elements that have to be present for a UR to be accepted upon insertion is good since it is the most flexible one, also: if the client queries the RUS server for the mandatory elements it should be straightfoward for the client to test its UR documents against the servers requirements before trying to insert them into the RUS.
One method that is used elsewhere in OGF specifications is to declare a (read-only) property of the server that describes the profiles on the records that it supports. Each profile would be described by a unique URI, and would document how the UR is restricted (e.g. "must have this element, must not have that element, and the other element over there must mean something special"). The advantages of this are that it allows more complex assertions of what is acceptable and yet it is easier for clients to understand it (no need to parse complex XPath expressions!) That's all for now. ;-) Donal.
- About the name: currently Aggregate Usage Record (AUR), or would the name Summary Usage Record/SUR be more appropriate, since aggregation can also mean just a set of single URs without summing the usage values up?
I think I'd prefer Summary UR. It's much clearer about it's purpose.
I also prefer "Summary" to "Aggregate". When I use the latter, I always seem compelled to embark on a long-winded explanation. "Summary" is clearer. Stephen
Hi, In the EGEE/WLCG accounting portal we also use "summary" instead of "aggregate" to refer to this type of records that we produce from the individual records. "Aggregate" could be reserved for the case where you collect records proceeding from different producers but without summarizing them. Cheers, Javier Stephen M Pickles wrote:
- About the name: currently Aggregate Usage Record (AUR),
or would the
name Summary Usage Record/SUR be more appropriate, since
aggregation can
also mean just a set of single URs without summing the
usage values up?
I think I'd prefer Summary UR. It's much clearer about it's purpose.
I also prefer "Summary" to "Aggregate". When I use the latter, I always seem compelled to embark on a long-winded explanation. "Summary" is clearer.
Stephen
-- ur-wg mailing list ur-wg@ogf.org http://www.ogf.org/mailman/listinfo/ur-wg
-- Dr. Javier Lopez Cacheiro Centro de Supercomputacion de Galicia (CESGA) Avda. de Vigo. s/n (Campus Sur) 15705 Santiago de Compostela (Spain) Tel: +34 981 56 98 10 ; Fax: +34 981 59 46 16 email: jlopez@cesga.es ; http://www.cesga.es/ ------------------------------------------------
participants (5)
-
Donal K. Fellows -
Gilbert Netzer -
Javier Lopez Cacheiro -
Rosario Michael Piro -
Stephen M Pickles