Hia folx, I stumbled over a couple of issues in the IDL spec, and would appreciate clarification / feedback. Some are minor, some are major. The bigger ones are likely caused by my limited understanding of the background which led to the design, so please bear with me - I don't want to reopen any discussions which have been closed for good... - the enum ResourceLimitType does not seem to be used anywhere - the enum JobTemplatePlaceholder does not seem to be used anywhere - I think the following is asymmetric. There is likely a reason, but I am not sure I understand it: - job session: create (name, contact), open (name), close (session), destroy (name) - reserv. session: create (name, contact), open (name), close (session), destroy (name) - monit. session: create ( contact), close (session) From the above, it seems like create/close are pairs? I would naively expect the following pairs: open/close and create/destroy, as usual - what is the rationale? Why is the Monitoring session handled differently, i.e. has no name/open/destroy? - jobInfo.exitStatus is a long. Shouldn't that be an int? Or is that an IDL artifact? - inErrorState: this is not explained - what does it mean? Am I getting reservation in an error state? What for? also, reservations don't seem to have state, really? (BTW: I did not check if all attributes are explained, just stumbled over this one. This should be checked). - JobWaitStarted/Terminated: The function returns always a Job object, in order to allow chaining, e.g. job.wait(JobStatus.RUNNING).hold(). The session-level functions I find that strange and inconsistent - chaining is not supported elsewhere AFAICS, why here? What is the advantage over 'job.wait(JobStatus.RUNNING); job.hold();'? this has exactly the same race conditions... - StringList getJobSessions(); StringList getReservationSessions(); All getXYZ methods in the API return XYZ, apart from these two - which return the name of XYZ. IMHO, they should either return XYZ, or should be called StringList listJobSessions(); StringList listReservationSessions(); - sessionManager.drmaaVersion: this seems to return the implementation version, not the DRMAA spec version (i.e. 2.0). I think this is useless without also reporting the drmaaName, i.e. the implementation name. Otherwise the user may report to you that she is using version 4.5, but what implementation?? ;-) - what is the use of the sessionManager interface? It cannot be created/destroyed, so is likely a singleton? This however is not explicit in the spec. A sessionManager does not seem to have state (apart from the new addition of a monitoring callback), but a stateless singleton is rather useless? All methods can easily be provided as free functions - is a language binding allowed to do that? - "sessionName: is supposed to be unique - if a session with that name was created before, an exception is thrown." Hmm, 'created before'? ever? by anybody? Or only for this session manager instance? Or this DRM system? etc etc. Cheers, Andre. -- Nothing is ever easy...
Hi,
The bigger ones are likely caused by my limited understanding of the background which led to the design, so please bear with me - I don't want to reopen any discussions which have been closed for good...
I like your attitude ;-) ...
- the enum ResourceLimitType does not seem to be used anywhere
It defines the valid keys for JobTemplate::resourceLimits, so it does not show up in the signatures.
- the enum JobTemplatePlaceholder does not seem to be used anywhere
It defines the valid macros usable in job template attributes, see Section 4.4. in Draft 6.
- I think the following is asymmetric. There is likely a reason, but I am not sure I understand it:
- job session: create (name, contact), open (name), close (session), destroy (name) - reserv. session: create (name, contact), open (name), close (session), destroy (name) - monit. session: create ( contact), close (session)
From the above, it seems like create/close are pairs? I would naively expect the following pairs: open/close and create/destroy, as usual - what is the rationale?
I don't get your point. Creation returns a new persistent XXXsession instance. Open allows you to re-access the persistent existing session. Close is for cleanup after both create and open. Destroy throws away the persistent information, and therefore does not demand an open session instance.
Why is the Monitoring session handled differently, i.e. has no name/open/destroy?
Monitoring session have no persistency, so they need no name for opening, and no destruction.
- jobInfo.exitStatus is a long. Shouldn't that be an int? Or is that an IDL artifact?
Yes. There are no INTs in IDL.
- inErrorState: this is not explained - what does it mean? Am I getting reservation in an error state? What for? also, reservations don't seem to have state, really? (BTW: I did not check if all attributes are explained, just stumbled over this one. This should be checked).
Oops, we removed that one, but I forgot to fix the IDL too. Thanks.
- JobWaitStarted/Terminated: The function returns always a Job object, in order to allow chaining, e.g. job.wait(JobStatus.RUNNING).hold(). The session-level functions
I find that strange and inconsistent - chaining is not supported elsewhere AFAICS, why here? What is the advantage over 'job.wait(JobStatus.RUNNING); job.hold();'? this has exactly the same race conditions...
Chaining was raised as important thing only for the wait functions. I am too lazy to search the minutes for this, but it is handy in OO languages. We didn't spend deeper thoughts on supporting this elsewhere. Proposals welcome.
- StringList getJobSessions(); StringList getReservationSessions();
All getXYZ methods in the API return XYZ, apart from these two - which return the name of XYZ. IMHO, they should either return XYZ, or should be called
StringList listJobSessions(); StringList listReservationSessions();
Sounds reasonable to me, I put that on the open issues list for the group.
- sessionManager.drmaaVersion: this seems to return the implementation version, not the DRMAA spec version (i.e. 2.0). I think this is useless without also reporting the drmaaName, i.e. the implementation name. Otherwise the user may report to you that she is using version 4.5, but what implementation?? ;-)
You get the DRMS type, this is your DRMAA implementation. I propose that there will be no two DRMAA implementations for the same DRM system in the same language.
- what is the use of the sessionManager interface? It cannot be created/destroyed, so is likely a singleton? This however is not explicit in the spec. A sessionManager does not seem to have state (apart from the new addition of a monitoring callback), but a stateless singleton is rather useless? All methods can easily be provided as free functions - is a language binding allowed to do that?
Yes, it's a singleton. Yes, you can do free functions in the language binding. IDL does not support that.
- "sessionName: is supposed to be unique - if a session with that name was created before, an exception is thrown." Hmm, 'created before'? ever? by anybody? Or only for this session manager instance? Or this DRM system? etc etc.
Argh, we missed that one. I put it on the list of open issues. Since sessions are expected to be submission-host specific, this should be also the answer here. Thanks for your help ! Peter.
2011/6/21 Peter Tröger <peter@troeger.eu>:
Hi,
The bigger ones are likely caused by my limited understanding of the background which led to the design, so please bear with me - I don't want to reopen any discussions which have been closed for good...
I like your attitude ;-) ...
;-) Thanks for the answers, that helps. Some comments inlined below. Cheers, Andre.
- I think the following is asymmetric. There is likely a reason, but I am not sure I understand it:
- job session: create (name, contact), open (name), close (session), destroy (name) - reserv. session: create (name, contact), open (name), close (session), destroy (name) - monit. session: create ( contact), close (session)
From the above, it seems like create/close are pairs? I would naively expect the following pairs: open/close and create/destroy, as usual - what is the rationale?
I don't get your point. Creation returns a new persistent XXXsession instance. Open allows you to re-access the persistent existing session. Close is for cleanup after both create and open. Destroy throws away the persistent information, and therefore does not demand an open session instance.
Why not close(name), so that all ops work with name as arg? Would be more symmetric.
Why is the Monitoring session handled differently, i.e. has no name/open/destroy?
Monitoring session have no persistency, so they need no name for opening, and no destruction.
If something has a create, I would expect it to have a destroy, too. That might just be me, but semantically those two go together... Anyway, your explanation helps!
- JobWaitStarted/Terminated: The function returns always a Job object, in order to allow chaining, e.g. job.wait(JobStatus.RUNNING).hold(). The session-level functions
I find that strange and inconsistent - chaining is not supported elsewhere AFAICS, why here? What is the advantage over 'job.wait(JobStatus.RUNNING); job.hold();'? this has exactly the same race conditions...
Chaining was raised as important thing only for the wait functions. I am too lazy to search the minutes for this, but it is handy in OO languages. We didn't spend deeper thoughts on supporting this elsewhere.
You may want to make sure that chaining was *not* introduced to prevent race conditions - because it does not.
Proposals welcome.
Proposal: don't do chaining, it makes error handling a nightmare - for almost all languages really, but in particular so for exception-less languages.
- sessionManager.drmaaVersion: this seems to return the implementation version, not the DRMAA spec version (i.e. 2.0). I think this is useless without also reporting the drmaaName, i.e. the implementation name. Otherwise the user may report to you that she is using version 4.5, but what implementation?? ;-)
You get the DRMS type, this is your DRMAA implementation. I propose that there will be no two DRMAA implementations for the same DRM system in the same language.
Woah - you MAY be able to ensure that for the current WG constituency, but you are writing a *standard*! You expect there will only *one* implementation using fork, for example? Only one implementation for PBS? You can never ensure this, and that contradicts the idea of an standard, really... The potentially missing API call is a minor point really, but your argument does not hold water ;-)
- what is the use of the sessionManager interface? It cannot be created/destroyed, so is likely a singleton? This however is not explicit in the spec. A sessionManager does not seem to have state (apart from the new addition of a monitoring callback), but a stateless singleton is rather useless? All methods can easily be provided as free functions - is a language binding allowed to do that?
Yes, it's a singleton.
This should be noted in the spec (unless I missed it).
Yes, you can do free functions in the language binding. IDL does not support that.
Ok, thanks. Cheers, Andre. -- Nothing is ever easy...
see inline Cheers, Daniel Am 22.06.2011 um 00:05 schrieb Andre Merzky:
2011/6/21 Peter Tröger <peter@troeger.eu>:
Hi, ...
- I think the following is asymmetric. There is likely a reason, but I am not sure I understand it:
- job session: create (name, contact), open (name), close (session), destroy (name) - reserv. session: create (name, contact), open (name), close (session), destroy (name) - monit. session: create ( contact), close (session)
From the above, it seems like create/close are pairs? I would naively expect the following pairs: open/close and create/destroy, as usual - what is the rationale?
I don't get your point. Creation returns a new persistent XXXsession instance. Open allows you to re-access the persistent existing session. Close is for cleanup after both create and open. Destroy throws away the persistent information, and therefore does not demand an open session instance.
Why not close(name), so that all ops work with name as arg? Would be more symmetric.
Yeah, but you make explicit that only with a valid handle (i.e. with an previously opened/created session) you are able to close it. The operation itself works on a (existing) session not on a string. My question is: In a OO language binding, wouldn't it be more natural to have this operation part of the Session itself? JobSesssion xy ... = ... JobSession.close()? Wouldn't the IDL spec now allow this (it should)? In case of a close("name") it definitely would not be possible. Like Peter said destroying works on the name not on an existing session. This could speed up session cleaning in some cases.
Why is the Monitoring session handled differently, i.e. has no name/open/destroy?
Monitoring session have no persistency, so they need no name for opening, and no destruction.
If something has a create, I would expect it to have a destroy, too. That might just be me, but semantically those two go together...
Anyway, your explanation helps!
It is a runtime object, IMHO even a singleton.
- JobWaitStarted/Terminated: The function returns always a Job object, in order to allow chaining, e.g. job.wait(JobStatus.RUNNING).hold(). The session-level functions
I find that strange and inconsistent - chaining is not supported elsewhere AFAICS, why here? What is the advantage over 'job.wait(JobStatus.RUNNING); job.hold();'? this has exactly the same race conditions...
Chaining was raised as important thing only for the wait functions. I am too lazy to search the minutes for this, but it is handy in OO languages. We didn't spend deeper thoughts on supporting this elsewhere.
You may want to make sure that chaining was *not* introduced to prevent race conditions - because it does not.
Proposals welcome.
Proposal: don't do chaining, it makes error handling a nightmare - for almost all languages really, but in particular so for exception-less languages.
- sessionManager.drmaaVersion: this seems to return the implementation version, not the DRMAA spec version (i.e. 2.0). I think this is useless without also reporting the drmaaName, i.e. the implementation name. Otherwise the user may report to you that she is using version 4.5, but what implementation?? ;-)
You get the DRMS type, this is your DRMAA implementation. I propose that there will be no two DRMAA implementations for the same DRM system in the same language.
Woah - you MAY be able to ensure that for the current WG constituency, but you are writing a *standard*! You expect there will only *one* implementation using fork, for example? Only one implementation for PBS? You can never ensure this, and that contradicts the idea of an standard, really...
The potentially missing API call is a minor point really, but your argument does not hold water ;-)
I would be fine with adding drmaaName, where implementors can add their vendor/organization name inside. Just for the case if...
- what is the use of the sessionManager interface? It cannot be created/destroyed, so is likely a singleton? This however is not explicit in the spec. A sessionManager does not seem to have state (apart from the new addition of a monitoring callback), but a stateless singleton is rather useless? All methods can easily be provided as free functions - is a language binding allowed to do that?
Yes, it's a singleton.
This should be noted in the spec (unless I missed it).
Yes, you can do free functions in the language binding. IDL does not support that.
Ok, thanks.
Cheers, Andre.
-- Nothing is ever easy... -- drmaa-wg mailing list drmaa-wg@ogf.org http://www.ogf.org/mailman/listinfo/drmaa-wg
See inline. Best, Peter. Am 22.06.2011 um 09:31 schrieb Daniel Gruber:
see inline
Cheers, Daniel
Am 22.06.2011 um 00:05 schrieb Andre Merzky:
2011/6/21 Peter Tröger <peter@troeger.eu>:
Hi, ...
- I think the following is asymmetric. There is likely a reason, but I am not sure I understand it:
- job session: create (name, contact), open (name), close (session), destroy (name) - reserv. session: create (name, contact), open (name), close (session), destroy (name) - monit. session: create ( contact), close (session)
From the above, it seems like create/close are pairs? I would naively expect the following pairs: open/close and create/destroy, as usual - what is the rationale?
I don't get your point. Creation returns a new persistent XXXsession instance. Open allows you to re-access the persistent existing session. Close is for cleanup after both create and open. Destroy throws away the persistent information, and therefore does not demand an open session instance.
Why not close(name), so that all ops work with name as arg? Would be more symmetric.
Yeah, but you make explicit that only with a valid handle (i.e. with an previously opened/created session) you are able to close it. The operation itself works on a (existing) session not on a string.
Exactly.
My question is: In a OO language binding, wouldn't it be more natural to have this operation part of the Session itself? JobSesssion xy ... = ... JobSession.close()? Wouldn't the IDL spec now allow this (it should)? In case of a close("name") it definitely would not be possible.
Close is paired with the others at the same location, since this allows a more natural mapping for C. OO languages typically have their own destruction methods, which might implicitely act as close operation. The language binding is free to define that - and we might mention this in the text.
Why is the Monitoring session handled differently, i.e. has no name/open/destroy?
Monitoring session have no persistency, so they need no name for opening, and no destruction.
If something has a create, I would expect it to have a destroy, too. That might just be me, but semantically those two go together...
Anyway, your explanation helps!
It is a runtime object, IMHO even a singleton.
Sounds reasonable. Is there a reason to open multiple monitoring session ? I don't think so ...
- JobWaitStarted/Terminated: The function returns always a Job object, in order to allow chaining, e.g. job.wait(JobStatus.RUNNING).hold(). The session-level functions
I find that strange and inconsistent - chaining is not supported elsewhere AFAICS, why here? What is the advantage over 'job.wait(JobStatus.RUNNING); job.hold();'? this has exactly the same race conditions...
Chaining was raised as important thing only for the wait functions. I am too lazy to search the minutes for this, but it is handy in OO languages. We didn't spend deeper thoughts on supporting this elsewhere.
You may want to make sure that chaining was *not* introduced to prevent race conditions - because it does not.
It wasn't. I am sure.
Proposal: don't do chaining, it makes error handling a nightmare - for almost all languages really, but in particular so for exception-less languages.
I agree, but the Java people love that *so* much. It is a nightmare for the error handling in the application, so if they use it, they have to deal with the consequences.
- sessionManager.drmaaVersion: this seems to return the implementation version, not the DRMAA spec version (i.e. 2.0). I think this is useless without also reporting the drmaaName, i.e. the implementation name. Otherwise the user may report to you that she is using version 4.5, but what implementation?? ;-)
You get the DRMS type, this is your DRMAA implementation. I propose that there will be no two DRMAA implementations for the same DRM system in the same language.
Woah - you MAY be able to ensure that for the current WG constituency, but you are writing a *standard*! You expect there will only *one* implementation using fork, for example? Only one implementation for PBS? You can never ensure this, and that contradicts the idea of an standard, really...
The potentially missing API call is a minor point really, but your argument does not hold water ;-)
I would be fine with adding drmaaName, where implementors can add their vendor/organization name inside. Just for the case if...
Sounds reasonable, I add it to the list.
- what is the use of the sessionManager interface? It cannot be created/destroyed, so is likely a singleton? This however is not explicit in the spec. A sessionManager does not seem to have state (apart from the new addition of a monitoring callback), but a stateless singleton is rather useless? All methods can easily be provided as free functions - is a language binding allowed to do that?
Yes, it's a singleton.
This should be noted in the spec (unless I missed it).
Ok, noted.
Yes, you can do free functions in the language binding. IDL does not support that.
Ok, thanks.
Cheers, Andre.
-- Nothing is ever easy... -- drmaa-wg mailing list drmaa-wg@ogf.org http://www.ogf.org/mailman/listinfo/drmaa-wg
2011/6/22 Daniel Gruber <dgruber@univa.com>:
- I think the following is asymmetric. There is likely a reason, but I am not sure I understand it:
- job session: create (name, contact), open (name), close (session), destroy (name) - reserv. session: create (name, contact), open (name), close (session), destroy (name) - monit. session: create ( contact), close (session)
From the above, it seems like create/close are pairs? I would naively expect the following pairs: open/close and create/destroy, as usual - what is the rationale?
I don't get your point. Creation returns a new persistent XXXsession instance. Open allows you to re-access the persistent existing session. Close is for cleanup after both create and open. Destroy throws away the persistent information, and therefore does not demand an open session instance.
Why not close(name), so that all ops work with name as arg? Would be more symmetric.
Yeah, but you make explicit that only with a valid handle (i.e. with an previously opened/created session) you are able to close it. The operation itself works on a (existing) session not on a string.
My question is: In a OO language binding, wouldn't it be more natural to have this operation part of the Session itself? JobSesssion xy ... = ... JobSession.close()? Wouldn't the IDL spec now allow this (it should)? In case of a close("name") it definitely would not be possible.
As the session has the name as property, I think close could (and should) very well be a member of the object, without the need for an argument.
Like Peter said destroying works on the name not on an existing session. This could speed up session cleaning in some cases.
ok.
Why is the Monitoring session handled differently, i.e. has no name/open/destroy?
Monitoring session have no persistency, so they need no name for opening, and no destruction.
If something has a create, I would expect it to have a destroy, too. That might just be me, but semantically those two go together...
Anyway, your explanation helps!
It is a runtime object, IMHO even a singleton.
If it is a singleton which is already instantiated (e.g. by loading the library), then it is being opened, not created? Anyway, I begin to understand the model - much appreciated... One might want to make those things clearer in the spec though, as others will likely stumble over similar questions? Cheers, Andre. -- Nothing is ever easy...
Am 22.06.2011 um 10:35 schrieb Andre Merzky:
2011/6/22 Daniel Gruber <dgruber@univa.com>:
....
Why is the Monitoring session handled differently, i.e. has no name/open/destroy?
Monitoring session have no persistency, so they need no name for opening, and no destruction.
If something has a create, I would expect it to have a destroy, too. That might just be me, but semantically those two go together...
Anyway, your explanation helps!
It is a runtime object, IMHO even a singleton.
If it is a singleton which is already instantiated (e.g. by loading the library), then it is being opened, not created?
It's just a name, if this makes it easier to understand, then we could simply change it to open(). I can't see any argument against. Cheers, Daniel
Anyway, I begin to understand the model - much appreciated... One might want to make those things clearer in the spec though, as others will likely stumble over similar questions?
Cheers, Andre.
-- Nothing is ever easy...
I added close() semantics and the singleton model decision to the list of open issues for the next conf call. Thanks, Peter. Am 22.06.2011 um 10:35 schrieb Andre Merzky:
2011/6/22 Daniel Gruber <dgruber@univa.com>:
- I think the following is asymmetric. There is likely a reason, but I am not sure I understand it:
- job session: create (name, contact), open (name), close (session), destroy (name) - reserv. session: create (name, contact), open (name), close (session), destroy (name) - monit. session: create ( contact), close (session)
From the above, it seems like create/close are pairs? I would naively expect the following pairs: open/close and create/destroy, as usual - what is the rationale?
I don't get your point. Creation returns a new persistent XXXsession instance. Open allows you to re-access the persistent existing session. Close is for cleanup after both create and open. Destroy throws away the persistent information, and therefore does not demand an open session instance.
Why not close(name), so that all ops work with name as arg? Would be more symmetric.
Yeah, but you make explicit that only with a valid handle (i.e. with an previously opened/created session) you are able to close it. The operation itself works on a (existing) session not on a string.
My question is: In a OO language binding, wouldn't it be more natural to have this operation part of the Session itself? JobSesssion xy ... = ... JobSession.close()? Wouldn't the IDL spec now allow this (it should)? In case of a close("name") it definitely would not be possible.
As the session has the name as property, I think close could (and should) very well be a member of the object, without the need for an argument.
Like Peter said destroying works on the name not on an existing session. This could speed up session cleaning in some cases.
ok.
Why is the Monitoring session handled differently, i.e. has no name/open/destroy?
Monitoring session have no persistency, so they need no name for opening, and no destruction.
If something has a create, I would expect it to have a destroy, too. That might just be me, but semantically those two go together...
Anyway, your explanation helps!
It is a runtime object, IMHO even a singleton.
If it is a singleton which is already instantiated (e.g. by loading the library), then it is being opened, not created?
Anyway, I begin to understand the model - much appreciated... One might want to make those things clearer in the spec though, as others will likely stumble over similar questions?
Cheers, Andre.
-- Nothing is ever easy...
participants (3)
-
Andre Merzky -
Daniel Gruber -
Peter Tröger