May 2005 - jsdl-wg - lists.ogf.org

JSDL Schema Doc
by Andrew Stephen McGough 12 May '05

12 May '05

All, I'm sat on a train so a perfect time to comment on the document... 1) Figure 1 is great now - though it isn't referred to in the text. Perhapse at the end of the paragraph beforehand: "Figure 1 illistrates places where a JSDL document may be used within a Grid environment. It should be noted that the JSDL document should not be limited to just these situations as many others may exist.". 2) First paragraph of section 3: Last scentence: Many of these languages are currently undefined though _they_ are required for the submission of jobs. 3) Page 8: Job Policy Thus -> This. 4) Page 8: Workflow: Incomplete scentence: Many other groups within (and without) the GGF. -> Many other groups witin (and without) the GGF are already working within this area. 5) Section 3.1 has an [add more text here] element. 6) Page 24: Example of a MoutSource The filesystem "HOME" must have at most 10GB capacity. The execution system is free to choose an appropriate MountPoint since no value is provided. This should mount the NFS export sputnick.ggf.org:/home/steve <jsdl:FileSystem name="jsdl:HOME" > <jsdl:Description> Steve's home </jsdl:Description> <jsdl:MountSource>nfs:sputnick.ggf.org/home/steve</jsdl:MountSource> <DiskSpace> <UpperBoundedRange>10737418240.0</UpperBoundedRange> </DiskSpace> <jsdl:FileSystemType>jsdl:normal</jsdl:FileSystemType> </jsdl:FileSystem> 7) my email is now just asm(a)doc.ic.ac.uk That's probably enuf for now. steve.. -- -- ------------------------------------------------------------------------ Dr A. Stephen McGough http://www.doc.ic.ac.uk/~asm ------------------------------------------------------------------------ Technical Coordinator, London e-Science Centre, Imperial College London, Department of Computing, 180 Queen's Gate, London SW7 2BZ, UK tel: +44 (0)207-594-8409 fax: +44 (0)207-581-8024 ------------------------------------------------------------------------ Assistant Warden, West Wing, Beit Hall, Imperial College, Prince Consort Road, London, SW7 2BB tel: +44 (0)207-594-9910 ------------------------------------------------------------------------

3 2

CREAM JDL vs JSDL
by Massimo Sgaravatto - INFN Padova 12 May '05

12 May '05

Dear all In the context of the Compute Resource Management Interfaces (CRM) initiative (the same mentioned by Oxana), the Job Description Language (JDL) used by the CREAM (Computing Resource Execution AND Management) service has been compared with JSDL. The CREAM JDL is a subset of the EGEE WMS JDL, that is the language used to specify characteristics and requirements for jobs submitted to the EGEE Workload Management System. This JDL is a high-level, user-oriented language based con Condor Classads for describing jobs and complex requests (aggregates of jobs). It is a fully extensible language, hence the user is allowed to use whatever attribute for the description of a request. However, only a certain set of attributes, referred as "supported attributes", is taken into account. The result of this evaluation is available at: http://www.pd.infn.it/grid/jra1/CREAMJDL-vs-JSDL.pdf A presentation summarizing these results is also available at: http://www.pd.infn.it/~sgaravat/Grid/CREAMJDL-vs-JSDL-EGEE3.pdf http://www.pd.infn.it/~sgaravat/Grid/CREAMJDL-vs-JSDL-EGEE3.ppt In short we found that for many functionality for which specific CREAM JDL attributes exist there aren't "default" JSDL directives that could be used. For most of them this is not surprising since they refer to features/functionality very specific to our architecture. In this case I guess JSDL extensions could be used. On the other hand (and here I agree with Oxana) there are some issues that in my opinion could/should probably be addressed by the default JSDL directives. In particular (and, again, I agree with Oxana): - the specification of workflows (in our case DAGs, job collections, parametric jobs) - the support for matchmaking, i.e. allowing specifying requirements (and preferences) on the target resource where the job has to be executed. Thanks for your attention Cheers, Massimo \\\|/// \\ ~ ~ // (/ @ @ /) -------oOOo-(_)-oOOo---------------------------------- Massimo Sgaravatto INFN Sezione di Padova Via Marzolo, 8 35131 Padova - Italy Tel: ++39 0498277047 Fax: ++39 0498277102 oooO E-mail: massimo.sgaravatto(a)pd.infn.it ( ) Oooo Home page: http://www.pd.infn.it/~sgaravat --------\ (----( )---------------------------------- \_) ) / (_/

2 1

JSDL vs NorduGrid xRSL: a comparison (fwd)
by Ali Anjomshoaa 11 May '05

11 May '05

Forwarding bounced mail on behalf of Oxana Smirnova, NorduGrid. Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 ------------------------------------------------------------- ---------- Forwarded message ---------- Date: Mon, 09 May 2005 14:52:43 +0200 From: Oxana Smirnova <oxana.smirnova(a)hep.lu.se> To: jsdl-wg(a)gridforum.org, egee-project-crm <egee-project-crm(a)cern.ch>, NorduGrid Discussion <nordugrid-discuss(a)nordugrid.org> Subject: JSDL vs NorduGrid xRSL: a comparison Dear all, as agreed by the Compute Resource Management Interfaces (CRM) initiative, JSDL has been evaluated by the participating Grid system experts against job description languages presently in use. NorduGrid's ARC uses extended Globus RSL (xRSL), which can be quite easily mapped to the POSIX Application of JSDL. However, there are several elements that are missing from JSDL but have been proven to be essential for proper Grid job description and execution - at least from the NorduGrid perspective. JSDL appears to be a proper framework for a more advanced Grid job description, provided it can be extended to: a) accomodate more elements (most notably, Run-time Environment for an application) b) enable multiple job definitions in a single "script"/file c) allow for alternative parameter requests (e.g., alternative OS, environment, host etc) A detailed comparison and related notes can be downloaded at http://www.nordugrid.org/documents/jsdl-vs-xrsl-revised2.sxw Hopefully, this can be taken as an input for further JSDL development. NorduGrid so far did not participate in JSDL-WG activities, but can contribute in future, if the WG members find it useful. Best regards, Oxana Smirnova, for the NorduGrid collaboration

4 5

JSDL: Telecon minutes and actions
by Ali Anjomshoaa 11 May '05

11 May '05

Hi all, the minutes from yesterday's telecon are attached. You can also find them at: https://forge.gridforum.org/projects/jsdl-wg/document/Minutes_-_2005-05-10/… Please take note of your Actions from these minutes. We're very close to closing the issues for v1.0, after which an editorial polish is due. Version 1.0 is due for submission to the GGF Editor for AD review on Wed. 25th May - 2 weeks from today. The plan is to freeze the contents next Wed. 18th. Cheers and take care, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

1 0

JSDL Meeting
by Darren Pulsipher 10 May '05

10 May '05

When: Tuesday, May 10, 2005 7:00 AM (MDT) Where: Online *~*~*~*~*~*~*~*~*~* -+-----+-----+-----+-----+-----+-----+-----+-----+- Darren Pulsipher has invited you to attend a Microsoft Office Live Meeting. Subject: JSDL Meeting Meeting URL: https://www.livemeeting.com/cc/xango/join Meeting ID: WXCXR7 Attendee Key: KF~44D{'b Role: Attendee This Office Live Meeting invitation is a personal invitation meant only for you. It should not be forwarded. If you received this email by mistake or require Live Meeting Assistance, please refer to the Live Meeting Assistance Center at http://r.office.microsoft.com/r/rlidLiveMeeting?p1=7&p2=en_US&p3=LMInfo&p4=…

1 0

Fwd: JSDL action point 17: Extending JSDL
by Michel Drescher 09 May '05

09 May '05

Folks, since this is something that is required for JSDL v1 and I unfortunately did not get Steve's comments on this yet, I decided to give the JSDL community the opportunity to comment on this before tomorrow's telecon. Cheers, Michel Begin forwarded message: > From: Michel Drescher <Michel.Drescher(a)uk.fujitsu.com> > Date: 3 May 2005 17:45:49 BST > To: Andrew Stephen McGough <asm(a)doc.ic.ac.uk> > Subject: Re: JSDL action point 17: Extending JSDL > > Hi Steve, > > thx for the review. I now come up with the following, having added a > section for jsdl:other and an example for attribute extension: > > Extending JSDL > ============== > JSDL provides for the overall structure to define jobs. To extend JSDL > to best fit to the required needs, JSDL provides two mechanisms for > extension: Extending using attributes and extending using elements. In > general using extension mechanisms will break interoptability of JSDL > consumers and as such these mechanisms should be used spareingly and > only where need. It is recommended that people interested in extending > the functionality of JSDL should first verify that no other group has > provided an extension which meets their requirements. > > Attribute based extensions > -------------------------- > Every JSDL element allows for additional attributes, as many as > needed, provided these attributes have a namespace other than the > normative namespaces JSDL defines. > > Example: > This example introduces an order that has to be followed when staging > in files (shortened for brevity): > > <?xml version="1.0" encoding="UTF-8"?> > <jsdl:JobDefinition xmlns="http://www.example.org/" > xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/04/jsdl" > xmlns:o="http://www.example.org/order-of-execution"> > <jsdl:JobDescription> > <jsdl:DataStaging o:order="1"> > <jsdl:FileName>foo</jsdl:FileName> > <jsdl:CreationFlag>overwrite</jsdl:CreationFlag> > > <jsdl:Source><jsdl:URL>http://www.nowhere.com/foo-file</jsdl:URL></ > jsdl:Source> > </jsdl:DataStaging> > <jsdl:DataStaging o:order="2"> > <jsdl:FileName>bar</jsdl:FileName> > <jsdl:CreationFlag>overwrite</jsdl:CreationFlag> > > <jsdl:Source><jsdl:URL>http:://www.nowhere.com/bar-file</jsdl:URL></ > jsdl:Source> > </jsdl:DataStaging> > </jsdl:JobDescription> > </jsdl:JobDefinition> > > > > Element based extensions > ------------------------ > Where applicable within the overall structure of the document, JSDL > allows for additional elements that are not normatively defined by > JSDL. As with attribute extension, these elements must assume a > namespace different than any namespace JSDL normatively defines. > > Example (shortened for brevity): > The following example extends JSDL by introducing reservations: > <?xml version="1.0" encoding="UTF-8"?> > <jsdl:JobDefinition xmlns="http://www.example.org" > xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/04/jsdl"> > <jsdl:JobDescription> > <jsdl:Resource> > <jsdl:CPUCount>  > <jsdl:Exact>1.0</jsdl:Exact> > </jsdl:CPUCount> > <jsdl:DiskSpace>  > > <jsdl:LowerBoundedRange>1073741824.0</jsdl:LowerBoundedRange> > </jsdl:DiskSpace> > <ug:Reservation > > xmlns:ug="http://schemas.unigrids.org/2005/05/reservation"> > <ug:Ticket>h933fsolenri900wnmd90mm34</ug:Ticket> > </ug:Reservation> > </jsdl:Resource> > </jsdl:JobDescription> > </jsdl:JobDefinition> > > Semantics of jsdl:other > ----------------------- > JSDL defines enumerations of values for processor architectures and > operating system. Both enumerations are not assumed to be exhaustively > complete. These enumerations are embedded in wrapper types to allow > for extension, "jsdl:CPUArchitectureType" and > "jsdl:OperatingSystemTypeType", respectively. > > To extend the enumerations, the special keyword "other" MUST be used > for the values of the elements "jsdl:CPUArchitectureName" and > "jsdl:OperatingSystemName", respectiveely, and the element extension > pattern as described above MUST be used. > > Example: > Given there exist a CPU that natively executes Java Bytecode named > "JavaCPU", this CPU would have to be specified as follows in the > resource section of a JSDL instance document: > ... > <Resource> > <CPUArchitecture> > <CPUArchitectureName>other</CPUArchitectureName> > <ex:NewCPUTypes > xmlns:ex="http://www.example.org/NewCPUTypes">JavaCPU</ex:NewCPUTypes> > </CPUArchitecture> > </Resource> >

1 0

JSDL: Updates
by Ali Anjomshoaa 06 May '05

06 May '05

Hi everyone, yesterday's telecon minutes are now on GridForge: https://forge.gridforum.org/projects/jsdl-wg/document/Minutes_-_2005-05-03/… PLEASE note that next week's telecon starts an hour earlier and goes on for 2 hours, starting at: 06:00 PST 07:00 MST 08:00 CST 09:00 EST 14:00 BST 15:00 CET 22:00 JST Trackers updated to reflect minutes from yesterday's telecon. Remember, this week's the big push. Next week we're closing down the issues for v1.0. PLEASE ALL READ THE SPEC. Many thanks in advance. Cheers and take care, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

3 4

JSDL compliance levels
by Michel Drescher 04 May '05

04 May '05

Folks, I got the assignment to the following two action points from last week's phone conference: >> 8) JSDL compliance levels >> How much of jsdl a system must support, not just process. >> Probably a topic for later. >> >> 9) The statement "If not supported then the consuming system MUST >> reject the document." needs a bit more explanation or re-write To solve these action points, I refer to the specification document (v0.8.5-02) ch 2 "Notational conventions". I propose to replace the chapter's last three paragraphs (the text after table 1) with the following paragraphs: "The term JSDL documentrefers to a XML document that has been created following this specification. It is a schema instance document derived from the normative schema definition inAppendix 2:Normative Schema. The terms JSDL element and JSDL attribute indicate that the corresponding language construct is represented as an XML element and XML attribute in a JSDL document. The key word presentrefers to a construct being present in a JSDL document. The key word support refers to a consuming system being able to apply the following functions to a JSDL element or JSDL attribute: · Parse the JSDL element or JSDL attribute into a DOM tree · Validate the parsed DOM tree against the appropriate schema(s) · Interpret the JSDL element or attribute and assign the semantics according to this specification. The JSDL specification does not require the consuming system to actually implement the semantics of JSDL elements and attributes, so that the described job is executed on a computing system. A consuming system MUST support all normative JSDL elements and JSDL attributes. If a consuming system implements JSDL, it MUST implement all defined JSDL elements and attributes. This allows not only systems that execute jobs but also systems that provide higher level services, for example job broker, super scheduling systems (that possibly change JSDL documents before they get submitted to lower level job execution systems), etc." Note: The last paragraph possibly fits better in a JSDL primer document. This way any consuming system that deals with JSDL must support the defined JSDL elements and attributes, but it does not need to "ground" or "incarnate" or even execute a JSDL job. But if it implements JSDDL, then all JSDL elements and attributes must be implemented. So this basically eliminates any JSDL compliance levels. As we already reduced JSDL to the lowest common denominator, there is no need of compliance levels at all - at least for the moment. The consequence would be that we can delete all these "If not supported then ... " sentences. Cheers, Michel

2 1

Action Item 26, Tracker 1391
by Donal K. Fellows 03 May '05

03 May '05

Issue 26: Normative Reference for POSIX: ISO/IEC 9945:2003, Portable Operating System Interface, version 3 [Note: Not online, but thumpingly definitive!] Tracker 1391: Semantics of RangeValueType Adapted from Karl's mail message, please find attached document fragment to be inserted between the pseudoschema for the document and the example. Donal.

1 0

Meeting minutes May 2
by Darren Pulsipher 03 May '05

03 May '05

May 2, 2005 Attendees Darren Steve Ali Bill Donal Michel Agenda Action Items from last call Document Status =============================================== Other ----- Possible JSDL meeting to go with the OGSA F2F meeting in London. Agenda & Minutes ---------------- 5) Section 3 (spec doc) general revision and polishing up - To be done... - Action: Andreas - Owners for: Glossary : Extending JSDL --> On Going --> On Going --> On Going 6) Update explanation for jsdl:other usage - jsdl:other is the value for e.g. other arch and OS name etc. - Have defined this and behaviours - have to be able to process it and must reject the document if can't be processed. - To add extra information that is allowed to be ignored if can't be processed, but, jsdl:any can't be ignored. - Section 4.2 - placeholder (maybe better in the Extending JSDL section). --> Action: Steve (with Michel) - as number 17 --> On Going (03-May-2005) --> On Going (Almost Done) 8) JSDL compliance levels How much of jsdl a system must support, not just process. Probably a topic for later. - Placeholder, will come up later! [DONE] - Action: Michel --> Action: ALL to review Michel's e-mail to list: JSDL compliance levels http://www-unix.gridforum.org/mail_archive/jsdl-wg/2005/04/msg00111.html --> Action: Ali to edit Michel's proposed entry (03-May-2005) 9) The statement "If not supported then the consuming system MUST reject the document." needs a bit more explanation or re-write. - Related to number 8 above. - Refer to notational convention [DONE] - Action: Michel --> Action: ALL to review Michel's e-mail to list: JSDL compliance levels http://www-unix.gridforum.org/mail_archive/jsdl-wg/2005/04/msg00111.html --> Action: Ali to edit Michel's proposed entry (03-May-2005) 11) Filesystem name and type and filesystem reference in DataStaging - This needs to be specified? Needs more explanation? - Action: Andreas - FileSystemID changed to FileSystemName - Define well known filesystem names: HOME and TMP --> On Going --> On Going --> On Going *** 12) Mountpoint environment variable. Is the current approach ok, or should we provide a separate mechanism, e.g, a tag that if present provides the name of the environment variable in which the MountPoint value must be present. - If a Mountpoint environment variable is not specified, then the name of the filesystem is used for the value of the env var. - Action: ALL to review spec section 5.5.12 and get back to the list - Issue to be discussed on the list. --> On Going --> On Going --> On Going 14) DRMAA use case for defining filenames. We have to at least be able to show an example of how to define and stage out an <Output> file. - Action: Andreas [DONE]- Action: Donal to suggest alternative - Action: ALL to review Donal's suggestion on the mailing list. - Action: Donal to chase this on the list... --> On Going --> On Going 16) Process Topology needs review. In particular are the examples ok or not, are the definition correct and so on. Delete the yellow comments once that is done. Not sure if we should define the precise allocation behaviour or allow for that to be defined instead. - Action: ALL to review this section (Section 6 in 0.9.5-01). - ProcessCount to be put back into POSIX Application. - ProcessCountLimit in the POSIX Application. [DONE] --> Action: Chris to bring issues to list if any - Action: ALL to review discussion on this on the list - Action: Karl to send summary of discussion to the list - Action: Andreas to put final recommendation from Karl's summary in the spec. --> On Going --> On Going --> On Going 17) Section "Extending JSDL" needs an owner. I expect this section to be about 1 page long so should not be too much trouble. - Action: Steve (with Michel as wingman) --> On Going --> On Going (03-May-2005) --> On Going (Almost Done) 19) Re-check terminology for process topology and delete comments - Part of 16 Review --> See 16 above - On Going 24) Complete author information (see my entry as reference) - Leave to last... Placeholder --> Action: Ali to pick this up. --> On Going --> On Going --> On Going 25) Update contributors and acknowledgements - Leave to last... Placeholder --> Action: Ali to pick this up. --> On Going --> On Going --> On Going 26) Complete list of Normative references, e.g., CIM is still missing. - Leave to last... Placeholder --> One for CIM (http://www.dmtf.org/standards/cim/) version 2.8 --> Action: Donal, pass ref for lists from CIM to Andreas. - Action: Donal - Add reference for POSIX. --> On Going --> On Going --> On Going 31) Complete example sub-sections - Need examples for elements. --> On Going --> On Going --> On Going v1.0 GridForge Trackers ----------------- 1271 - post-1.0 1272 - post-1.0 1273 - DONE 1275 - Steve 1277 - DONE, CLOSE 1279 - DONE 1280 - DONE 1281 - DONE 1282 - post-1.0 1283 - post-1.0 1284 - post-1.0 1285 - Michel: Andreas' Intro Slides 1286 - post-1.0 1310 - DONE 1326 - DONE 1327 - Ali: Andreas' Intro Slides 1330 - post-1.0 1331 - DONE 1332 - post-1.0 1334 - DONE 1335 - DONE 1358 - post-1.0 1391 - Donal

1 0