- jsdl-wg - lists.ogf.org

RE: [jsdl-wg] JSDL: Telecons
by Bill Harris 11 Apr '05

11 Apr '05

meeting tomorrow ........... information below telecomm only... Darren will be sending out web information William Harris Program Architect Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744 The contact details are: Dial In Numbers: Belgium - 0800-7053 Bezeq - 1-800-940-1786 France - 0800-903109 Germany - 0800-1815765 Hong Kong - 800-968932 Italy - 800874509 Japan - 00531123656 Korea-Tele - 00798-14-800-0512 Netherlands - 0800-0225447 Singapore - 800-1200766 Sweden - 020-7905043 Taiwan - 00801-13-8546 UK - 0800961294 US - 800-933-3594 --> ext 3000 pin: 6444 The telecon time is: _every_Tuesday_ 07:00 PST 08:00 MST 09:00 CST 10:00 EST 15:00 BST 16:00 CET 23:00 JST

1 0

Tracker status values proposal
by Andreas Savva 11 Apr '05

11 Apr '05

This is a procedural proposal about the specification tracker settings: At the moment the tracker artifact status are the gridforge default: open, pending, closed. These cannot be changed but new values can be defined. I propose to augment them with the following values: resolved, fixed, deferred. The meaning of these values would be as follows: Open: the issue has been proposed by a member but not yet accepted by the WG. Pending: the issue has been accepted and is still pending resolution by the WG Resolved: The WG has decided and accepted a resolution, but the actions are still pending Fixed: The editor has completed the proposed actions, but the actions are awaiting verification. Closed: The issue has been resolved and is reflected in the work products of the WG. The completion of the changes must be reported by the primary editor and verified by the secondary editor (or other WG member). Deferred: The WG has decided that the issue has some merit but that it should not be addressed in the upcoming revision of the spec. And deferred items would later be moved to the post-v1 tracker. Note that all this is not my invention. I am 'stealing' it from the ogsa-wg trackers. (And those were in turn 'stolen' from the ogsi ones or other places ....) -- Andreas Savva Fujitsu Laboratories Ltd

6 12

my view on execution user and group
by Karl Czajkowski 08 Apr '05

08 Apr '05

The concrete use case I have from Globus is to indicate under which POSIX username a job should be executed. I see group (actually, default or primary group) as a reasonable extension of the concept, though we do not do that today. This use case is job configuration details, just like POSIX limits, environment variables, etc. It is important in situations where there is not a one-to-one mapping of global user identity (PKI subject) to local accounts for the GRAM service to apply, e.g. when a user has several role-specific accounts or temporary "scratch" accounts. Of course, such configuration values are subject to authorization checks in practice. I am not clear on whether this is the intended purpose of the existing user and group fields in the User section, but if it is not then I am not clear on what it is for at all... I admit my bias is towards thinking of the use of JSDL in distributed protocols like GRAM. I guess if we were to use it in the local "adapter" interface between the GRAM service and the local scheduler scripts, we might have a use for specifying the authenticated user name, which for GRAM would be a PKI subject name. This makes sense because the adapter scripts inherently trust GRAM; GRAM would reject the presence of such a field in the over-the-wire transmission of JSDL, since the user identification comes from the binding-level security mechanisms. I can see this kind of info belonging in a User section for the adapter case, but I still need the POSIX username in the executable section. karl -- Karl Czajkowski karlcz(a)univa.com

12 28

RE: [jsdl-wg] RE: [drmaa-wg] making a custom file path for output
by Rajic, Hrabri 06 Apr '05

06 Apr '05

Tricky. It would need to be XTSL like preprocessing on both the client side and on the compute side. I wonder if the differences between Grid and Compute Center would require any specializations. There is much more legacy stuff that DRMAA needs to address that GRAM. -Hrabri >-----Original Message----- >From: Donal K. Fellows [mailto:donal.k.fellows@manchester.ac.uk] >Sent: Wednesday, April 06, 2005 8:33 AM >To: JSDL WG >Cc: Rajic, Hrabri; DRMAA Working Group >Subject: Re: [jsdl-wg] RE: [drmaa-wg] making a custom file path for output > >Ali Anjomshoaa wrote: >> we discussed this issue at some length at GGF13 and were unsure as to how >> to proceed. We would like to be able to reference labeled attributes, but, >> I think we're still discussing how to implement this issue in the xsd >> schema depending on current tooling! >> >> I think, unfortunately, our choice of how we implement this in xsd will >> influence how we specify how to do this in the spec. Clearly this is not >> ideal and should be the other way round! > >> On Tue, 5 Apr 2005, Rajic, Hrabri wrote: >>>Placeholders would be special variables like home directory or job id ($ >>>sign prefixed below), which DRMAA could substitute in job attributes >>>like the output path: >>> >>>drmaa_set_attribute( $jt, DRMAA_OUTPUT_PATH, >>> $DRMAA_HOME_DIR/MY_APP/my_job_output.$DRMAA_JOB_ID.out" > >Hmm, I have to say that I don't like arbitrary substitutions in XML >content, especially in styles like this. (It's one of the bits I do not >care for about Ant, BTW.) But the way I'd handle such a problem is >through something like submitting the job through a service that applied >an XSLT substitution to the document before producing a doc with "ground >terms". If you see what I mean. > >In the specific case of wanting to have file arguments given to the >application made relative to one of our virtual filesystem concepts, >there is an argument to be made for some improvement of the JSDL spec. >At the moment, the only mechanisms we have for arguments are "relative >to the job working directory" or "absolute", neither of which are great >for this use-case. (The same argument applies to environment variables, >of course.) > >Does this brain-dump indicate the sort of area where we need to address >some kind of solution? Or are there other cases which will need more >refinement before we work out how to fix it all? > >Donal.

1 0

RE: [drmaa-wg] making a custom file path for output
by Ali Anjomshoaa 06 Apr '05

06 Apr '05

Hi Hrabri, we discussed this issue at some length at GGF13 and were unsure as to how to proceed. We would like to be able to reference labeled attributes, but, I think we're still discussing how to implement this issue in the xsd schema depending on current tooling! I think, unfortunately, our choice of how we implement this in xsd will influence how we specify how to do this in the spec. Clearly this is not ideal and should be the other way round! Watch the JSDL space and let us know how you do this in the end. Cheers and take care, Ali On Tue, 5 Apr 2005, Rajic, Hrabri wrote: > Hi Ali, > > Placeholders would be special variables like home directory or job id ($ > sign prefixed below), which DRMAA could substitute in job attributes > like the output path: > > drmaa_set_attribute( $jt, DRMAA_OUTPUT_PATH, > $DRMAA_HOME_DIR/MY_APP/my_job_output.$DRMAA_JOB_ID.out" > > > > -Hrabri > > >-----Original Message----- > >From: Ali Anjomshoaa [mailto:ali@epcc.ed.ac.uk] > >Sent: Tuesday, April 05, 2005 10:59 AM > >To: Rajic, Hrabri > >Cc: Andreas Haas; Tim Harsch; DRMAA Working Group > >Subject: RE: [drmaa-wg] making a custom file path for output > > > > > >Hi Guys, > > > >If you could give me a quick synopsis of what you consider placeholders > >and in what context, perhaps I can help from the JSDL perspective. > > > >Cheers, > > > >Ali > > > > > >On Tue, 5 Apr 2005, Rajic, Hrabri wrote: > > > >> Andreas, > >> > >> I would suggest we approach the question as two issues: > >> > >> 1) what are the global placeholders in DRMAA > >> 2) in which (string typed) job attributes the placeholders could be > used > >> > >> > >> Here are few questions that go with the above: > >> > >> How is JDSL handling placeholders - nice to know? > >> Do all major DRM systems support placeholders well? > >> Do we need a new error code indicating failure in resolving a > >> placeholder? Any other errors needed? > >> > >> -Hrabri > >> > >> > >> >-----Original Message----- > >> >From: owner-drmaa-wg(a)ggf.org [mailto:owner-drmaa-wg@ggf.org] On > Behalf > >> Of > >> >Andreas Haas > >> >Sent: Tuesday, April 05, 2005 7:37 AM > >> >To: Tim Harsch > >> >Cc: DRMAA Working Group > >> >Subject: Re: [drmaa-wg] making a custom file path for output > >> > > >> >Hi Tim, > >> > > >> >I fully agree there is a need for this. Please see > >> > > >> > http://forge.gridforum.org/tracker/?aid=805 > >> > > >> >I raised it's priority to P2. I believe that issue must > >> >be solved with resubitted DRMAA spec. > >> > > >> >Regards, > >> >Andreas > >> > > >> > > >> >On Mon, 4 Apr 2005, Tim Harsch wrote: > >> > > >> >> I'm sorry I wasn't clear. I guess the heart of my question is: > How > >> can > >> >> create an arbitrarty template for the file and path name? > >> Specifically > >> >what > >> >> constant tells the DRM to insert job id into that filename > template. > >> >> > >> >> Said in Perl psuedo form: > >> >> > >> >> drmaa_set_attribute( $jt, > >> >> $DRMAA_OUTPUT_PATH + > ":/tmp/my_job_output.$DRMAA_JOB_ID.out" > >> ); > >> >> > >> >> # with that code job outputs look like > >> >> /tmp/my_job_output.1.out > >> >> /tmp/my_job_output.2.out > >> >> /tmp/my_job_output.3.out > >> >> > >> >> # without that code job outputs look like > >> >> /home/harsch/script_name.o1 > >> >> /home/harsch/script_name.o2 > >> >> /home/harsch/script_name.o3 > >> >> > >> >> Is there a DRMAA_JOB_ID constant? > >> >> > >> >> ----- Original Message ----- > >> >> From: "Rajic, Hrabri" <hrabri.rajic(a)intel.com> > >> >> To: "DRMAA Working Group" <drmaa-wg(a)gridforum.org> > >> >> Sent: Monday, April 04, 2005 12:02 PM > >> >> Subject: RE: [drmaa-wg] making a custom file path for output > >> >> > >> >> > >> >> There is a simple solution, provided it works for you: > >> >> to rely on the shell to supply the home directory path instead of > >> DRMAA. > >> >> > >> >> -Hrabri > >> >> > >> >> > >> >> >-----Original Message----- > >> >> >From: owner-drmaa-wg(a)ggf.org [mailto:owner-drmaa-wg@ggf.org] On > >> Behalf > >> >> Of > >> >> >Tim Harsch > >> >> >Sent: Monday, April 04, 2005 1:19 PM > >> >> >To: DRMAA Working Group > >> >> >Subject: [drmaa-wg] making a custom file path for output > >> >> > > >> >> >Hi all, > >> >> > I have a requirement for the app I'm writing ( using > >> >> Schedule::DRMAAc ) > >> >> >which I'm trying to implement. > >> >> > > >> >> >Normally the output files from jobs go to home directory and are > of > >> the > >> >> >form > >> >> ><jobname>.o<jobid> > >> >> > > >> >> >I want to move the jobs to a subdir in /tmp, but retain the > <jobid> > >> in > >> >> the > >> >> >filename. I tried this: > >> >> > > >> >> > ( $job_error, $diagnosis ) = drmaa_set_attribute( $jt, > >> >> > $DRMAA_OUTPUT_PATH, > >> >> > ":" . $tmpdir . '/' . > >> >> > __PACKAGE__ . '.' . > >> >> > $DRMAA_PLACEHOLDER_INCR . > >> >> > ".out" ); > >> >> > > >> >> >... but PLACEHOLDER is only for parametric jobs, I've been > scanning > >> the > >> >> >docs > >> >> >and have not found a placeholder for job id. Any ideas on how I > >> could > >> >> >accomplish this task? > >> >> > > >> >> >A similar task would be to move the job output path without > changing > >> >> the > >> >> >filename format at all. How could I do that? > >> >> > >> >> > >> > >> > >> > > > >-- > > > > ---------------------------------------------------- |epcc| - > > Ali Anjomshoaa > > EPCC, University of Edinburgh > > James Clerk Maxwell Building > > Mayfield Road E-mail: ali(a)epcc.ed.ac.uk > > Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 > > United Kingdom Fax: + 44 (0) 131 650 6555 > > ------------------------------------------------------------- > > > -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

3 3

phone conf today
by Michel Drescher 06 Apr '05

06 Apr '05

Guys, was anybody on the phone conference today? I tried, but I've been the only one attending. After some time I quit - Later the meeting "has ended" as the computer operator told me. What was going on? Bill, can you shed some light on this? Cheers, Michel

3 2

JSDL Meeting
by Bill Harris 05 Apr '05

05 Apr '05

You have been invited to attend a Microsoft Office Live Meeting. At the time of the meeting, click on the following link to attend. URL: http://www.placeware.com/cc/cadence/join?id=RWZ334&role=attend Subject: JSDL Meeting When: Tuesday, Apr 5, 2005 8:00 AM (MDT) Scheduled to Occur: Recurring Duration: 1 Hour 30 Minutes Audio Information: 1-800-933-3594 ext 3000 pin: 6444 Alternate Attend Instructions: Go to: http://www.placeware.com/cc/cadence/join Your Name: (enter your name) Meeting ID: RWZ334 William Harris Program Architect 150 West Civic Center Dr. Sandy, Utah 84070 Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744

1 0

JSDL v0.9.5-01
by Michel Drescher 31 Mar '05

31 Mar '05

Guys, here're a couple of new and updated documents: draft-ggf-jsdl-spec-0.9.5-01.doc The JSDL specification document itself. Contains all changes and decisions from the last phone conference. jsdl-0.9.5-01.xsd The JSDL schema consistent (well, I hope so...) with the latest specification document. apps-posix-0.1.xsd The schema defining the POSIXApplication element. POSIXApplication-Example.xml A sample XML file using the POSIXApplication extension. I will post a new list of issues for the next phone conference tomorrow (I hope). Cheers, Michel

3 3

GGF 13 notes and further discussion
by Michel Drescher 31 Mar '05

31 Mar '05

Dear JSDL wranglers, from the four sessions on GGF I compiled a list of topics that are unclear to me. My comments etc. are in square brackets ("[ and "]). Please feel free to comment. Session 1 - Fig. 1: suggestion to do a different document with scoping/explanation if this is controversial - multi-job [i.e. NAREGI] scenarios use container reference mechanism to pick up the needed pieces. [This would require identifying name attributes that should be of type NCName so that the container can use QNames to refer to needed parts. Alternatively, XPath or XQuery are an option, although not really handy in this case, IMHO] Session 2 - Units - declare as QNames? [I thought we agreed that units vanish and frequencies, bandwidth and storage are given, either as float or integer, with a fixed unit like Hz or Byte, respectively. However, can't find this in the notes stored at GridForge.] - open content on data staging [What does that mean?] - ref/name: check tooling support [Standard Java toolkits, SAX and W3C DOM, support this. However, AFAIK we didn't really clearly decide on using this.] Session 3 - A tag to indicate what the client wants to be understood or not? Everything should be understood vs [Hmm... vs what? What did we decide?] - jsdl:Application xsd:any##other with cardinality 1 vs jsdl:ApplicationType with type xsd:any##other and list all applications that are defined separately [No decisions on that so far, I think. It intertwines with Igor's proposal.] - need some tie-in to acs wrt [with respect to?] application definitions - [and these] limits should be under application executable [so these are then basically part of the normative JSDL extensions, or are they supposed to be child elements of the jsdl:Application element which is the parent element of the future normative extensions?] - [and the] limits are not used for resource selection; they are in the environment Session 4 - Get rid of all String types? - Ranges implicitly define operators, i.e. between 'a' and 'b' - Discuss on the list which resource constraints are global total constraints and which are per process constraints. [Are there constraints that can be both? For example, One may constrain the job to use no more than 10 Gigabytes of physical memory, but each process must not consume more than 10 Megabytes of physical memory. Is this a valid use case?] Cheers, Michel

5 9

Topics for the upcoming phone conference
by Michel Drescher 31 Mar '05

31 Mar '05

Dear JSDL wranglers, this is a list of topics I would like to be decided on at the today's phone conference. (1) JobDefinition/@JSDLVersion attribute The version of the JSDP spec is uniquely defined elsewhere, and encoded in the namespace. I do not see any use of this attribute. (2) JobDefinition/JobIdentification/User/UserCredential The spec contains a comment that this element "... can be handdled through the standard extension mechanism." Let's decide on the phone conference to keep it or to drop it (and release it to the extension mechanism). (3) range values Karl and Stephen made a valuable proposal for a clear and efficient definition of range values: <lowerBound>xsd:integer</lowerBound> ? <upperBound>xsd:integer</upperBound> ? <exact>xsd:integer</exact> * <range> <lowerBound>xsd:integer</lowerBound> <upperBound>xsd:integer</upperBound> </range> * I propose to replace our current definition of a range value with this complex structure. (4) xsd:integer or xsd:double Partly touching topic 3, there's a discussion on wether to use integers or fraction numbers. I propose to use xsd:double together with an optional attribute "epsilon=xsd:double" for precision restrictions, changing the range type to <lowerBound epsilon="xsd:double"? >xsd:double</lowerBound> ? <upperBound epsilon="xsd:double"? >xsd:double</upperBound> ? <exact epsilon="xsd:double"? >xsd:double</exact> * <range> <lowerBound epsilon="xsd:double"? >xsd:double</lowerBound> <upperBound epsilon="xsd:double"? >xsd:double</upperBound> </range> * (5) (Non-)Critical extensions There's a discussion at the moment to tag particular elements of JSDL to be (non-)critical in terms of understandability and support, similar to the SOAP attribute "mustUnderstand=xsd:boolean". I propose to postpone this mechanism to after the first official JSDL release. (6) naming elements To promote reusage, elements of JSDL documents should be identifiable. An appropriate technique is using QNames. This requires the elements that we want to be reused to get a naming attribute of type xsd:NCName. I propose to use this technique to promote reusage. If we agree to use this, a follow-up action point will be assiged to identify the elements that need this new naming attribute and to propose a naming syntax. (7) Resource constraints global total and/or per process During GGF13, the issue came up that a resource can be defined to be total global or as per process. A use case may be to constrain a job to 10 Gigabytes of physical memory and at the same time to constrain the job's processes to consume no more than 10 Megabytes of physical memory. I propose to reflect this in JSDL expressis verbis rather to punt it out to an extension. If we agree on incorporating this, a follow-up action point will be the investigation and proposal of solution alternatives. (8) Comments and flames on the current specification document Although this is very short term, I attached the current latest version of the spec document - unofficial as I am not the official editor! - in which I tried to reflect all the changes we agreed on at GGF13. It was quite much, so expect to face errors in what I created. Cheers, Michel

1 1