Dear Oxana, many thanks for your feedback and comments. It is clear that you can make a valuable contribution to the work of the group. I am very sorry that you have joined us so very late in the day :( In order to understand the choices that we have made, you need to know the history of the group and our work over the years. Workflow, scheduling, and security are out of scope of JSDL. Of course there are a number of levels of granularity to each of these areas and it is very much up to the definitions that the group has assigned to these issues. e.g. a parametric job is workflow in our view of the job submission world. ...I would not expect that you would have 1000s of job documents for a parameter sweep, I would expect, however, that your submission/job management system can handle the fact that the same job description defines a parametric job where the value of a single argument for the job executable is changed accordingly. It is a matter of opinion as to whether adding this metadata to the JSDL is any better than adding it to a workflow layer or not. As to your question of how one might add new elements/concepts to the JSDL spec, I'm afraid that it takes more than just the popularity of adding that element amongst users. In order for you to suggest normative change it has to fit in with JSDL's purpose and scoping. ...I'm afraid that it is impossible for us to consider adding any substantially new elements for v1.0 of the spec and schema at this stage. We are on the way to putting this version out the door very soon and new additions will have to be lined up for discussion for a post-v1.0 spec and schema. We expect that you will be keen to participate in the public comment period for JSDL v1.0 this summer. You are right that some notion of logic and preference is needed in job scheduling, however, we have put this out of scope for v1.0 so that we can allow the description of simple, absolute job requirements. Grid work is very very complicated and as you say no one said it was going to be easy. However, the JSDL philosophy has always been "keep it simple", "learn to walk before you run" and be pragmatic. We want to try and satisfy simple needs of the majority of users first then look to satisfying the more complex scenarios. I do hope that you understand our points and that you'll keep tuned in to help us improve the JSDL spec and schema. The scope of JSDL will be made clearer in the spec very soon. If it still does not answer your questions, we're not doing our job and should try harder. Please also note that the team is working on releasing a primer following release of v1.0 of the spec and schema, and this is where we will, hopefully, provide practical examples of how to use JSDL. Many thanks for your comments and please continue to help us improve the JSDL. Cheers and take care, Ali On Wed, 11 May 2005, Oxana Smirnova wrote:
Hello Donal,
many thanks for replying, hope you (or other group members) also had time to look through the document. I am new to JSDL business, but have quite some experience with "real" job description languages used for great variety of applications in EDG/LCG/gLite (JDL) and NorduGrid/ARC (xRSL), thus I have some questions that perhaps are just FAQ of this group, see below.
a) This should be possible through either the existing subelements of the POSIXApplication element, or through extension. However I'd love to see more input from you on this as part of the development of the next version of the spec.
There are several elements that can be easily added; runtimeenvironment is just one of them. What is this group's criteria for an element to be added to the specs? What kind of arguments should I present to support each case? Would it be sufficient to tell that few hundred users find it convenient, and so do Grid m/w developers and site owners?
b) This should be done by wrapping the JSDL singletons inside some larger XML dialect that describes the relationship of the basic jobs to each other. Although we want to support workflows, we've always pushed the workflow bit of it out of scope so we can get agreement on the rest of everything. :^)
I don't think that describing several jobs in one script has anything to do with workflow. It's up to the workload management system to interpret it, but for the user's convenience it would be nice to include into JSDL a possibility to specify several jobs. No need for a "larger XML dialect", just one more super-element. Here's a use case: I want to submit 10000 jobs that differ only in one input parameter. I have a tool that generates JSDL; and I'd hate to fill my disk with 10000 small files. You are not going to create a dedicated workgroup to produce a two-line spec that describes JSDL concatenation, are you?
c) I think we're punting this one out to WS-Agreement (over a space of JSDL terms). It gets really complicated otherwise as you start to try to describe spaces of coupled terms...
Eh, nobody promised it's going to be easy :-) I understand that XML is just a markup language, not quite suited for task definition, because task description normally includes conditions "if then else", relations "and", "or", and negations "not". For example, "buy X amount of apples if they are red, else buy Y amount of pears and Z amount of bananas". This is a single task, these are not separate jobs. It does not describe workflow, because it does not specify in which shop these apples or pears should be purchased (doesn't even say it should be the same one), or when, and not even the price. JSDL presently can't describe such a job. I don't even see how two separate job descriptions can be merged by another *XML* dialect into one - XML is not really made for it. In real life, I've had plenty of jobs like "if a site has outbound connectivity, stage in this, else stage in something different". Globus' RSL allows this, and I'm sure it's a part of job description, not workflow. It is an instruction for the execution site, not for the workload manager/broker.
In summary, "why isn't it possible now", "out of scope", and "out of scope". But that's just a bit short and I think you'll prefer the longer outline above. ;^)
I'd like to learn what is the scope, I guess - and where to submit things that are out of it :-)
Oxana
-- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali@epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------