The reason we have vendor extensions is that it's much cleaner than the native specification, and it gives the DRMAA implementation the opportunity to understand the meaning of the job template without having to understand how to parse the native specification. In the SGE implementation, the native specification is passed directly to SGE. DRMAA knows nothing about what's in it. (Hence it's not sanity checked.) I could live with the idea of making the job template a value type and adding in C an undefined struct type field (e.g. struct drmaa2_vendor_attributes_t) that vendors are allowed to define themselves, with an empty struct being a valid definition. How would this approach affect the other non-OO languages, like Perl? Without thinking about it too hard, I think Perl would actually behaving like an OO language in this case. Are there other non-OO languages we'd need to worry about? I doubt it, but we need to be careful before closing the door on any implementations. I'm still chewing on whether it's really OK for the job template to allow bogus values, expecting that the run() method will point out the problems. We had a discussion about allowing DRMAA to filter the list of jobs to be returned from a search. If the job template were a value type, it could be used as the filter. Fill in the fields that interest you, and leave the rest empty. You'll get all the jobs that have those values set in those fields. And the easiest answer would be not to allow wildcarding. See the Jini discovery service for a complete implementation of this approach. Daniel On 03/26/10 07:23, Mariusz Mamoński wrote:
Hi,
I have tried to collect all the problems, suggestions, and also my experience with different APIs and created a scratch page on my wiki: "How the JobTamplate could look if.. ") with several different possible options:
http://fury.man.poznan.pl/~mmamonski/wiki/index.php/DRMAAv2/JobTemplate_-_Bi...
Please review every of the 4 options and provide your comments! We can always combine some ideas.
After writing this summary i asked one dummy question to myself: "What is the benefit of having vendor specific attributes if we already have nativeOptions (nativeSpecification)"?
Cheers,
On 24 March 2010 14:28, Andre Merzky<andre@merzky.net> wrote:
Quoting [Daniel Templeton] (Mar 24 2010):
Andre,
Don't worry about being biased. I'm unabashedly SGE-biased. :)
:-)
Unfortunately, though, what you proposed is what we're already discussing. It's the problem, not the solution. We're trying to have a consistent API across OO and non-OO languages. The issue goes away if we just add "except in C" to the description.
Hmm, I thought the problem was:
>> just only one concern: what if one of the DRM vendor wish to >> provide some JobTemplate attributes additional to those >> specified in DRMAA (as i remember SGE was an example)? I >> expect having different struct definition (different >> drmaa2.h ) leads to serious problems. I'm not saying no >> (actually having a self allocated struct in C is more >> convenient than using getter/setters - e.g. error handling), >> but this, for me, requires addressing the extension >> methodology on the IDL level.
The SAGA scheme would still map to C in a very simple way
// create job template drmaa_job_template_t jt = drmaa_job_template_create ();
// set a predefined DRMAA attribute // these calls would always succeed, basically. drmaa_job_template_set_attribute (jt, DRMAA_JOB_TEMPLATE_EXECUTABLE, "/bin/date"); drmaa_job_template_set_attribute (jt, "Executable", "/bin/date"); drmaa_job_template_set_executable (jt, "/bin/date");
// set a vendor specific attribute // this call would return an error if the attribute is not // supported drmaa_job_template_set_attribute (jt, "SGE:Executable-Type", "UniversalBinary");
// free job template drmaa_job_template_destroy (jt);
So, no special rule for C, and the same drmaa2.h for all vendors.
Well, the special rule for C is to have a Create and Destrpy method - but you have that for all classes/types, job template is not special in that respect...
Cheers, Andre.
Andre does bring up a valid point, though, which is that we really could just special-case the non-OO languages, just like we did in v1. Or, like I said before, we could leave the template management to the implementation, scoped to the session, with an overflow value of being allowed to free templates as needed.
Daniel
On 03/24/10 06:03, Andre Merzky wrote:
my biased point of view: use the saga model:
{ // create job template on the stack saga::job::description jd;
// 3 equivalent ways for predefined attribute jd.set_attribute (saga::job::description_executable, "/bin/date"); jd.set_attribute ("Executable", "/bin/date"); jd.set_executable ("/bin/date");
// vendor-defined attributes: jd.set_attribute ("SGE:Executable-Type", "UniversalBinary");
// job template gets deleted from the stack when leaving the scope }
Memory management of the hash table is done internally. The implementation also needs to maintain a list of predefined attributes - but that is anyway the case I think. C language bindings could of course still fall back to the explicit create/delete methods.
Again, sorry for being biased...
Best, Andre.
Quoting [Daniel Templeton] (Mar 23 2010):
It's not a bad idea. I would really like to find a way to make it work, because it would make life quite a bit simpler. Is there a way to make it work?
Daniel
On 03/23/10 13:56, Peter Tröger wrote:
Ok, it was a bad idea. Thanks for commenting.
/Peter.
Am 22.03.2010 um 16:14 schrieb Daniel Templeton:
> And that may be the original reason why the SGE C binding uses that > pseudo-OO structure. It is more or less a hash map, allowing > arbitrary > name-value pairs to be stored, facilitating vendor-defined job > template > attributes. If we can't find a way around that issue, it's a > show-stopper. We have to allow for vendor-defined attributes. > > Daniel > > On 03/22/10 07:29, Mariusz Mamo??ski wrote: >> Hi Peter, all, >> >> 2010/3/22 Peter Tröger<peter@troeger.eu>: >>> Dear all, >>> >>> we have an ongoing discussion about the possible removal of >>> "createJobTemplate" and "deleteJobTemplate". The last proposal was >>> to move this functions to the language bindings that need it. This >>> is currently only C - all other languages have their own way of >>> performing instantiation and termination explicitly. >>> >>> After some more thinking, I think I got the true underlying issue. >>> So far, we are treating job templates as instances with state and >>> behavior - objects, in most languages. The only reason for doing >>> this (so far) is the ability to throw errors on attribute access, >>> since we need that for DRMAA's understanding of optional job >>> template attributes. However, from all other perspectives, job >>> templates are just value types. If you got one, you fill it, and >>> then you pass the thing as a whole. In a RPC scenario, you would >>> also expect to send filled job templates as a whole, similar to a >>> filled JSDL document. >>> >>> So if we change that, what would that mean: >>> >>> C: Generic getter / setter functions for job template attributes >>> would go away. Instead, you would create / delete JobTemplate >>> structs directly. runJob() would take a pointer to this struct. >>> Explicit removal is no longer needed, since the stack is cleared >>> automatically. >>> >>> C# : JobTemplate class would become JobTemplate struct. >>> >>> Java / Python: JobTemplate class remains JobTemplate class, since >>> they have no struct concept. Creation and destruction can be >>> managed with OO mechanisms. >>> >>> Another effect would be a change in the semantics of optional >>> attributes. We already demand the syntactical inclusion of >>> optional attribute names in the class definition. In C, the usage >>> of a non-implemented optional attribute names gives a specialized >>> error. With the change, we would have struct members that you are >>> not allowed to fill in with some DRMAA libraries. But this would >>> be detected only at submission time, since struct changes do not >>> involve library code. >>> >> just only one concern: what if one of the DRM vendor wish to provide >> some JobTemplate attributes additional to those specified in DRMAA >> (as >> i remember SGE was an example)? I expect having different struct >> definition (different drmaa2.h ) leads to serious problems. I'm not >> saying no (actually having a self allocated struct in C is more >> convenient than using getter/setters - e.g. error handling), but >> this, >> for me, requires addressing the extension methodology on the IDL >> level. >> >>> And "createJobTemplate" resp. "deleteJobTemplate" ? Not needed at >>> all then, regardless of the language. >>> >>> Best, >>> Peter. >>> >>> >>> >>> >>> >>> -- >>> drmaa-wg mailing list >>> drmaa-wg@ogf.org >>> http://www.ogf.org/mailman/listinfo/drmaa-wg >>> >> >> >> Cheers, > -- > drmaa-wg mailing list > drmaa-wg@ogf.org > http://www.ogf.org/mailman/listinfo/drmaa-wg
-- drmaa-wg mailing list drmaa-wg@ogf.org http://www.ogf.org/mailman/listinfo/drmaa-wg
-- Nothing is ever easy. -- drmaa-wg mailing list drmaa-wg@ogf.org http://www.ogf.org/mailman/listinfo/drmaa-wg