Dear JSDL wranglers, from the four sessions on GGF I compiled a list of topics that are unclear to me. My comments etc. are in square brackets ("[ and "]). Please feel free to comment. Session 1 - Fig. 1: suggestion to do a different document with scoping/explanation if this is controversial - multi-job [i.e. NAREGI] scenarios use container reference mechanism to pick up the needed pieces. [This would require identifying name attributes that should be of type NCName so that the container can use QNames to refer to needed parts. Alternatively, XPath or XQuery are an option, although not really handy in this case, IMHO] Session 2 - Units - declare as QNames? [I thought we agreed that units vanish and frequencies, bandwidth and storage are given, either as float or integer, with a fixed unit like Hz or Byte, respectively. However, can't find this in the notes stored at GridForge.] - open content on data staging [What does that mean?] - ref/name: check tooling support [Standard Java toolkits, SAX and W3C DOM, support this. However, AFAIK we didn't really clearly decide on using this.] Session 3 - A tag to indicate what the client wants to be understood or not? Everything should be understood vs [Hmm... vs what? What did we decide?] - jsdl:Application xsd:any##other with cardinality 1 vs jsdl:ApplicationType with type xsd:any##other and list all applications that are defined separately [No decisions on that so far, I think. It intertwines with Igor's proposal.] - need some tie-in to acs wrt [with respect to?] application definitions - [and these] limits should be under application executable [so these are then basically part of the normative JSDL extensions, or are they supposed to be child elements of the jsdl:Application element which is the parent element of the future normative extensions?] - [and the] limits are not used for resource selection; they are in the environment Session 4 - Get rid of all String types? - Ranges implicitly define operators, i.e. between 'a' and 'b' - Discuss on the list which resource constraints are global total constraints and which are per process constraints. [Are there constraints that can be both? For example, One may constrain the job to use no more than 10 Gigabytes of physical memory, but each process must not consume more than 10 Megabytes of physical memory. Is this a valid use case?] Cheers, Michel
Michel, thanks for this. I think you should carry on with the spec and address all those issues that are clear to you. Any that are not and are not addressed on the list, we should address at next week's telecon. You have the pen. Some comments below. Steve, Donal, anyone else any comments? Cheers, Ali On Tue, 22 Mar 2005, Michel Drescher wrote:
Session 1
- Fig. 1: suggestion to do a different document with scoping/explanation if this is controversial
Ali: Are we still planning to do a Primer based on v1.0 when that is released? Perhaps any extended clarification of scope should go in there.
- multi-job [i.e. NAREGI] scenarios use container reference mechanism to pick up the needed pieces.
[This would require identifying name attributes that should be of type NCName so that the container can use QNames to refer to needed parts. Alternatively, XPath or XQuery are an option, although not really handy in this case, IMHO]
Session 2
- Units - declare as QNames?
[I thought we agreed that units vanish and frequencies, bandwidth and storage are given, either as float or integer, with a fixed unit like Hz or Byte, respectively. However, can't find this in the notes stored at GridForge.]
- open content on data staging
[What does that mean?]
Ali: One for Andreas or Steve...?
- ref/name: check tooling support
[Standard Java toolkits, SAX and W3C DOM, support this. However, AFAIK we didn't really clearly decide on using this.]
Ali: I think the decision was that if this could be done relatively painlessly, then we should do it.
Session 3
- A tag to indicate what the client wants to be understood or not? Everything should be understood vs
[Hmm... vs what? What did we decide?]
Ali: "vs" nothing! "Everything should be understood" by a JSDL consumer.
- jsdl:Application xsd:any##other with cardinality 1 vs jsdl:ApplicationType with type xsd:any##other and list all applications that are defined separately
[No decisions on that so far, I think. It intertwines with Igor's proposal.]
Ali: IIRC, we decided to reference application type and have it defined externally.
- need some tie-in to acs wrt [with respect to?] application definitions
Ali: This is a new proposal. ACS is still being worked on and we should perhaps provide a mechanism to leverage their interface. Alternatively, perhaps their interface could look like our application type definition. One to discuss with them perhaps. Certainly outside v1.0 scope I think.
- [and these] limits should be under application executable
[so these are then basically part of the normative JSDL extensions, or are they supposed to be child elements of the jsdl:Application element which is the parent element of the future normative extensions?]
- [and the] limits are not used for resource selection; they are in the environment
Session 4
- Get rid of all String types?
Ali: So that parsers and engines match against QNames instead, right?
- Ranges implicitly define operators, i.e. between 'a' and 'b'
- Discuss on the list which resource constraints are global total constraints and which are per process constraints.
[Are there constraints that can be both? For example, One may constrain the job to use no more than 10 Gigabytes of physical memory, but each process must not consume more than 10 Megabytes of physical memory. Is this a valid use case?]
Ali: Right. So do we need an XML attribute to say if a constraint element applies at a global or per process level?
Cheers, Michel
-- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------
On Mar 22, Michel Drescher loaded a tape reading:
Session 2
- Units - declare as QNames?
[I thought we agreed that units vanish and frequencies, bandwidth and storage are given, either as float or integer, with a fixed unit like Hz or Byte, respectively. However, can't find this in the notes stored at GridForge.]
That is my recollection as well.
- open content on data staging
[What does that mean?]
I am not sure of the context of this note, but "open content on Foo" is synonymous with "presence of xsd:any of some sort in Foo". :-) I was asking that these extension spots remain in the staging elements so that we (Globus) could, for example, insert some extended metadata to use this with GRAM and not lose any functionality.
Session 3
- A tag to indicate what the client wants to be understood or not? Everything should be understood vs
[Hmm... vs what? What did we decide?]
I brought this up in relation to extensions, but I am not 100% positive it could not apply to basic JSDL terms as well. The essential problem with any complicated declarative syntax is whether all pieces are mandatory or not. There is a common idiom in extensible languages to be able to mark some fields to this effect, perhaps using mandatory as the default and having an extra optional bit to override the default. The LDAP protocol is a nice old-fashioned version of this. Being able to mark some elements as mandatory and others as optional means that a consumer, for whatever reason, can filter out optional pieces and consider the remaining document as "equivalent" for the needs of the producer. This allows the consumer to eliminate extensions that it does not understand as well as parts it understands but which conflict with its own policies. Conversely, anything marked as mandatory that is not understood or welcome will have to lead to "failure", whatever that means in the usage context. The consumer knows that it cannot fully support, for whatever reason, the declarative bits that the producer thinks are essential to the meaning of the document as a whole.
- [and these] limits should be under application executable
[so these are then basically part of the normative JSDL extensions, or are they supposed to be child elements of the jsdl:Application element which is the parent element of the future normative extensions?]
My understanding was that we would provide an 80% solution in the form of an executable application type where these limits would now live. I wasn't sure whether the base spec. would include any other application type, or just expect them to arrive via extension.
- Discuss on the list which resource constraints are global total constraints and which are per process constraints.
[Are there constraints that can be both? For example, One may constrain the job to use no more than 10 Gigabytes of physical memory, but each process must not consume more than 10 Megabytes of physical memory. Is this a valid use case?]
Yes, I think this is a good example and the way we discussed it in Seoul, the solution is to make two terms that explicitly capture the concept at the two different scopes. Something like: jsdl:TotalPhysicalMemory jsdl:PerResourcePhysicalMemory If we can find a prefixing scheme we all can tolerate, I would suggest modifying EVERY term to explicitly state its scoping in its name. No matter how certain we are that a particular term's scope is obvious, somebody out there will think the opposite. :-) karl -- Karl Czajkowski karlcz@univa.com
Karl Czajkowski wrote:
Being able to mark some elements as mandatory and others as optional means that a consumer, for whatever reason, can filter out optional pieces and consider the remaining document as "equivalent" for the needs of the producer. This allows the consumer to eliminate extensions that it does not understand as well as parts it understands but which conflict with its own policies.
The problem is that there's no way for us to force such attributes on extension elements (I think) and we don't need it for any non-ext elements in JSDL since the spec has everything as "must understand" (though a consumer could reject the doc if it doesn't "support" the term in question).
Conversely, anything marked as mandatory that is not understood or welcome will have to lead to "failure", whatever that means in the usage context. The consumer knows that it cannot fully support, for whatever reason, the declarative bits that the producer thinks are essential to the meaning of the document as a whole.
We need to make sure that the BES people remember to make their service state which terms are actually supported as part of their metadata. Alas, such things can't be done in JSDL itself as it's just a language spec and does not discuss the execution meta-environment. Donal.
On Mar 29, Donal K. Fellows loaded a tape reading:
Karl Czajkowski wrote:
Being able to mark some elements as mandatory and others as optional means that a consumer, for whatever reason, can filter out optional pieces and consider the remaining document as "equivalent" for the needs of the producer. This allows the consumer to eliminate extensions that it does not understand as well as parts it understands but which conflict with its own policies.
The problem is that there's no way for us to force such attributes on extension elements (I think) and we don't need it for any non-ext elements in JSDL since the spec has everything as "must understand" (though a consumer could reject the doc if it doesn't "support" the term in question).
We could do it via a wrapper... I am proposing this sort of mechanism in WS-Agreement as well: 1. treat elements in xsd:any extensibility slots as mandatory/critical 2. except, if wrapped in jsdl:noncriticalExtension which is an element w/ exactly one xsd:any##other in its body karl -- Karl Czajkowski karlcz@univa.com
Hi Karl, Donal, I think that the 'optional' vs. 'mandatory' attribute mark-up could work for JSDL. However, we decided a long time ago that all specified attributes in a JSDL doc MUST be satisfied, i.e. that there are NO attributes that a system MAY consider optional in a job description. I think we should perhaps reconsider this position. However, I also think that this is something that we can leave for a post-v1.0 version of the spec. It doesn't strike me as something that will break version 1.0 backward compatibility if introduced at a later stage. So, I would vote to postpone this issue until after v1.0 has been released and any bugs on that version are fixed. Any thoughts on this? Cheers and take care, Ali On Tue, 29 Mar 2005, Karl Czajkowski wrote:
On Mar 29, Donal K. Fellows loaded a tape reading:
Karl Czajkowski wrote:
Being able to mark some elements as mandatory and others as optional means that a consumer, for whatever reason, can filter out optional pieces and consider the remaining document as "equivalent" for the needs of the producer. This allows the consumer to eliminate extensions that it does not understand as well as parts it understands but which conflict with its own policies.
The problem is that there's no way for us to force such attributes on extension elements (I think) and we don't need it for any non-ext elements in JSDL since the spec has everything as "must understand" (though a consumer could reject the doc if it doesn't "support" the term in question).
We could do it via a wrapper... I am proposing this sort of mechanism in WS-Agreement as well:
1. treat elements in xsd:any extensibility slots as mandatory/critical
2. except, if wrapped in jsdl:noncriticalExtension which is an element w/ exactly one xsd:any##other in its body
karl
-- Karl Czajkowski karlcz@univa.com
-- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------
Given that JSDL is defining an ontology of terms so that we can all understand what is meant by "command line" or "file transfer", isn't the determination of mandatory or optional a bit out of scope? I can easily see a system based on negotiation where the two parties agree on which terms are consumable by the end system, so you can get this behaviour that way. I guess I just worry that the user of JSDL's expectations for a particular job submission behaviour are met. Wouldn't it be nice to know if a particular attribute is rejected or not? This seems to me to be part of the submission protocol layer. -- Chris On 29/3/05 01:30, "Ali Anjomshoaa" <ali@epcc.ed.ac.uk> wrote:
Hi Karl, Donal,
I think that the 'optional' vs. 'mandatory' attribute mark-up could work for JSDL. However, we decided a long time ago that all specified attributes in a JSDL doc MUST be satisfied, i.e. that there are NO attributes that a system MAY consider optional in a job description.
I think we should perhaps reconsider this position. However, I also think that this is something that we can leave for a post-v1.0 version of the spec. It doesn't strike me as something that will break version 1.0 backward compatibility if introduced at a later stage.
So, I would vote to postpone this issue until after v1.0 has been released and any bugs on that version are fixed.
Any thoughts on this?
Cheers and take care,
Ali
On Tue, 29 Mar 2005, Karl Czajkowski wrote:
On Mar 29, Donal K. Fellows loaded a tape reading:
Karl Czajkowski wrote:
Being able to mark some elements as mandatory and others as optional means that a consumer, for whatever reason, can filter out optional pieces and consider the remaining document as "equivalent" for the needs of the producer. This allows the consumer to eliminate extensions that it does not understand as well as parts it understands but which conflict with its own policies.
The problem is that there's no way for us to force such attributes on extension elements (I think) and we don't need it for any non-ext elements in JSDL since the spec has everything as "must understand" (though a consumer could reject the doc if it doesn't "support" the term in question).
We could do it via a wrapper... I am proposing this sort of mechanism in WS-Agreement as well:
1. treat elements in xsd:any extensibility slots as mandatory/critical
2. except, if wrapped in jsdl:noncriticalExtension which is an element w/ exactly one xsd:any##other in its body
karl
-- Karl Czajkowski karlcz@univa.com
--
---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------
You have a good point here Chris. I'm glad you're making your point and staying consistent with the views you expressed much earlier for JSDL :) We agreed at the telecon on Tuesday that we're leaving this discussion for post-v1.0, even post bug-fixes if we can. Cheers, Ali On Wed, 30 Mar 2005, Christopher Smith wrote:
Given that JSDL is defining an ontology of terms so that we can all understand what is meant by "command line" or "file transfer", isn't the determination of mandatory or optional a bit out of scope? I can easily see a system based on negotiation where the two parties agree on which terms are consumable by the end system, so you can get this behaviour that way.
I guess I just worry that the user of JSDL's expectations for a particular job submission behaviour are met. Wouldn't it be nice to know if a particular attribute is rejected or not? This seems to me to be part of the submission protocol layer.
-- Chris
On 29/3/05 01:30, "Ali Anjomshoaa" <ali@epcc.ed.ac.uk> wrote:
Hi Karl, Donal,
I think that the 'optional' vs. 'mandatory' attribute mark-up could work for JSDL. However, we decided a long time ago that all specified attributes in a JSDL doc MUST be satisfied, i.e. that there are NO attributes that a system MAY consider optional in a job description.
I think we should perhaps reconsider this position. However, I also think that this is something that we can leave for a post-v1.0 version of the spec. It doesn't strike me as something that will break version 1.0 backward compatibility if introduced at a later stage.
So, I would vote to postpone this issue until after v1.0 has been released and any bugs on that version are fixed.
Any thoughts on this?
Cheers and take care,
Ali
On Tue, 29 Mar 2005, Karl Czajkowski wrote:
On Mar 29, Donal K. Fellows loaded a tape reading:
Karl Czajkowski wrote:
Being able to mark some elements as mandatory and others as optional means that a consumer, for whatever reason, can filter out optional pieces and consider the remaining document as "equivalent" for the needs of the producer. This allows the consumer to eliminate extensions that it does not understand as well as parts it understands but which conflict with its own policies.
The problem is that there's no way for us to force such attributes on extension elements (I think) and we don't need it for any non-ext elements in JSDL since the spec has everything as "must understand" (though a consumer could reject the doc if it doesn't "support" the term in question).
We could do it via a wrapper... I am proposing this sort of mechanism in WS-Agreement as well:
1. treat elements in xsd:any extensibility slots as mandatory/critical
2. except, if wrapped in jsdl:noncriticalExtension which is an element w/ exactly one xsd:any##other in its body
karl
-- Karl Czajkowski karlcz@univa.com
--
---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------
-- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------
On Mar 30, Christopher Smith loaded a tape reading:
Given that JSDL is defining an ontology of terms so that we can all understand what is meant by "command line" or "file transfer", isn't the determination of mandatory or optional a bit out of scope? I can easily see a system based on negotiation where the two parties agree on which terms are consumable by the end system, so you can get this behaviour that way.
I guess I just worry that the user of JSDL's expectations for a particular job submission behaviour are met. Wouldn't it be nice to know if a particular attribute is rejected or not? This seems to me to be part of the submission protocol layer.
-- Chris
Yes, I guess I can buy that. It is a fine line, indeed, between trying to capture the meaning of the job and negotiating about variations on it. I think what was concerning me, while I had on a declarative-job-language hat, is that I expect the extensions mechanisms to be used out their in a world where types and specifications are not introduced in a coordinated fashion. Whether I can understand the meaning of a JSDL instance depends on whether I am aware of all the extension specifications the producer has used. I agree that this might be enough, and it is up to the enclosing protocol context to determine how one filters, projects, negotiates, or otherwise communicates the "important" job information from a particular producer to a particular consumer. From that point of view, critical/non-critical is a particular communications idiom used to increase the probability of such communication succeeding in heterogeneous environments. Sorry for the red herring... karl -- Karl Czajkowski karlcz@univa.com
Michel Drescher wrote:
Session 1
- Fig. 1: suggestion to do a different document with scoping/explanation if this is controversial
My feeling is that we need "production versions" of all the figures anyway. Everything there at the moment just feels like embedded powerpoint slides... :^) That aside, we are probably going to have to do some kind of primer.
- multi-job [i.e. NAREGI] scenarios use container reference mechanism to pick up the needed pieces.
[This would require identifying name attributes that should be of type NCName so that the container can use QNames to refer to needed parts. Alternatively, XPath or XQuery are an option, although not really handy in this case, IMHO]
Being able to name things is good, and using xsd:NCName and a targetNamespace attribute on the outermost container element is a good way of doing it because it is like other XML specs.
Session 2
- Units - declare as QNames?
[I thought we agreed that units vanish and frequencies, bandwidth and storage are given, either as float or integer, with a fixed unit like Hz or Byte, respectively. However, can't find this in the notes stored at GridForge.]
My recollection is that every element will have its type fixed, but we didn't go through and do that because it wasn't controversial but was time-consuming.
- open content on data staging
[What does that mean?]
I think it's something to do with putting <xsd:any##other/> everywhere.
- ref/name: check tooling support
[Standard Java toolkits, SAX and W3C DOM, support this. However, AFAIK we didn't really clearly decide on using this.]
I thought we did decide to support this on the grounds that it makes life much easier down the road and it would be messy to add later on.
Session 3
- A tag to indicate what the client wants to be understood or not? Everything should be understood vs
[Hmm... vs what? What did we decide?]
All core elements MUST be understood already. :^) There's also no way to constrain extensibility elements I think.
- jsdl:Application xsd:any##other with cardinality 1 vs jsdl:ApplicationType with type xsd:any##other and list all applications that are defined separately
[No decisions on that so far, I think. It intertwines with Igor's proposal.]
My recollection matches this.
- need some tie-in to acs wrt [with respect to?] application definitions
My impression is that they're a long way off providing anything. At the very least, we should ignore them for now and get 1.0 out the door. We can patch this all up later on.
- [and these] limits should be under application executable
[so these are then basically part of the normative JSDL extensions, or are they supposed to be child elements of the jsdl:Application element which is the parent element of the future normative extensions?]
My recollection is that limits are to be sibling elements of Executable, Argument, etc. and that the limit types are to be done as individual elements instead of all as sub-elements of a Limits element (so we can get the typing right).
- [and the] limits are not used for resource selection; they are in the environment
Correct. Limits aren't resources (though the resources might act as bounds on acceptable limits on some systems).
Session 4
- Get rid of all String types?
Where possible. Hostnames still have to be strings I think...
- Ranges implicitly define operators, i.e. between 'a' and 'b'
Yes, or rather ranges subsume every operator that's useful in resource satisfaction description (unless you're doing something really nasty).
- Discuss on the list which resource constraints are global total constraints and which are per process constraints.
[Are there constraints that can be both? For example, One may constrain the job to use no more than 10 Gigabytes of physical memory, but each process must not consume more than 10 Megabytes of physical memory. Is this a valid use case?]
The other question is whether we want to support such things even if we have use-cases for them, or whether any case complex enough to require such things is outside the scope of what we wish to describe in JSDL 1.0 (without necessarily constraining future versions, of course). Donal.
participants (5)
-
Ali Anjomshoaa -
Christopher Smith -
Donal K. Fellows -
Karl Czajkowski -
Michel Drescher