February 2005 - jsdl-wg - lists.ogf.org

TileSize and ProcessCount
by claudio cacciari 02 Mar '05

02 Mar '05

Dear All, I 'm trying to understand the meaning of TileSize and ProcessCount elements to describe a MPI + OpenMPI job. For example, in a job submission request for a batch scheduler (LoadLeveler, PBS and so on) I can specify : tasks per job=3 threads per task=4 processors per task=4 in order to get 1 thread per processor ( processors per task = threads per task ). I can set "nodes per job"=2 (with a a configuration of 8 cpus per node) or leave the scheduler sets this value. For what I understand in JSDL there are : [tileSize] tasks (processes) per group [ProcessCount] tasks (processes) per job [CPUCount] processors per job (with ResourceCount=1) [CPUCount + ResourceCount] processors per node + nodes per job from which I can get : tasks per job = [ProcessCount] = 3 threads per task = [ ? ] = 4 processors per task = [ CPUCount * ResourceCount / ProcessCount ] = 4 nodes per job = [ ResourceCount ] = 2 And so what is a "group of processes (tasks)" ? Is there a way to specify "threads per task (process)" ? Best Regards, Claudio Cacciari ========================================================= Claudio Cacciari CINECA - Via Magnanelli 6/3 40033 High Performance System Group Casalecchio di Reno - Bologna - ITALY e-mail: c.cacciari(a)cineca.it http://www.cineca.it Tel: 051/6171411 - Fax: 051/6132198 =========================================================

4 5

Question on jsdl:HostName element
by Michel Drescher 01 Mar '05

01 Mar '05

From: Michel.Drescher(a)uk.fujitsu.com Subject: Various questions sourcing from external Date: 28 February 2005 14:06:02 GMT To: jsdl Dear JSDL wranglers, I got assigned to relay the following question to the JSDL working group: 1) jsdl:FileSystem A very popular use case for this element is to mount the user's home directory to the execution environment. How do I do that using JSDL? Solution a) - "Magic interpreter" The JSDL snippet would be <jsdl:FileSystem id="Home"> <jsdl:Description>foo</jsdl:Description> <jsdl:FileSystem> and the genie interpreter would recognise the id attribute to mount the execution user's home dir. JDSL goes "Genie in the bottle", if I may say so. ;-) Solution b) - Platform specific incarnation The JSDL snippet would be <jsdl:FileSystem id="Home"> <jsdl:Description>foo</jsdl:Description> <jsdl:MountPoint>/home/mdrescher</jsdl:MountPoint> <jsdl:FileSystem> This solution lacks abstraction since I cannot resubmit that very job to another site where my home directory is mounted to another path (i.e. /users/drescher). Brokering this job is also hardly possible even if I try to mess around with an incredibly large set of jsdl:Profile elements (to match all possible mount points for home directories). Solution b) - adding "abstractable" directories to jsdl:FileSystemType The JSDL snippet would be <jsdl:FileSystem id="Home"> <jsdl:Description>foo</jsdl:Description> <jsdl:FileSystemType>home</jsdl:FileSystemType> <jsdl:FileSystem> This way, JSDL would introduce the "home directory" syntactically. The downside is that $HOME is not a filesystem. But, then, is "tmp" always a different filesystem? Solution c) - Allowing globbing in jsdl:MountPoint The JSDL snippet could be <jsdl:FileSystem id="Home"> <jsdl:Description>foo</jsdl:Description> <jsdl:MountPoint>{$HOME}</jsdl:FileSystemType> <jsdl:FileSystem> or something similar This mechanism would introduce the well-known globbing mechanisms from PERL or the shells like tcsh, bash etc. Solution d) - A revised jsdl:FileSystem element I don't have any idea yet to propose, but personally, I think JSDL would benefit from an abstraction of file systems (like UNICORE or Globus provide). Any thoughts? Cheers, Michel

2 3

FW: JSDL Meeting
by Bill Harris 24 Feb '05

24 Feb '05

William Harris Program Architect Cadence Design Systems 150 West Civic Center Dr. Sandy, Utah 84070 Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 554 0187 From: Bill Harris Sent: Thursday, February 24, 2005 10:01 AM To: Bill Harris Subject: JSDL Meeting You have been invited to attend a Microsoft Office Live Meeting. At the time of the meeting, click on the following link to attend. http://www.placeware.com/cc/cadence/join?id=RWZ334&role=attend Subject: JSDL Meeting When: Tuesday, Mar 1, 2005 7:00 AM (MST) Scheduled to Occur: Recurring Duration: 1 Hour 30 Minutes Audio Information: 1-800-933-3594 pin: 6444 Click to add to your Outlook calendar: http://www.placeware.com/cc/cadence/meetingICS?id=RWZ334&role=attend&i=i .ics Alternate Attend Instructions: Go to: http://www.placeware.com/cc/cadence/join Your Name: (enter your name) Meeting ID: RWZ334 Dial In Numbers >Phone Numbers: >Belgium - 0800-7053 >Bezeq - 1-800-940-1786 >France - 0800-903109 >Germany - 0800-1815765 >Hong Kong - 800-968932 >Italy - 800874509 >Japan - 00531123656 >Korea-Tele - 00798-14-800-0512 >Netherlands - 0800-0225447 >Singapore - 800-1200766 >Sweden - 020-7905043 >Taiwan - 00801-13-8546 >UK - 0800961294 >US - 800-933-3594

1 0

No meeting on Tuesday.
by Darren Pulsipher 22 Feb '05

22 Feb '05

We will not be having a meeting this week. Darren

1 0

Limits Update and Apologies
by Donal K. Fellows 14 Feb '05

14 Feb '05

Firstly, here are the revised versions of the sections on the Limits and Limit elements. Query points (to be removed in editing) are highlighted in yellow, and what's there should be a drop-in replacement for the limits stuff in v0.9 of the main document. It includes a couple of examples and recommends that this goes in the same namespace as the major sub-elements of Application that we're defining (limits aren't really relevant to non-exec jobs). Secondly, I'm sorry but I won't be able to attend the telecon this week. I've been sent across the country to a funding-body meeting at short notice. Bleah. :^( Donal.

1 0

JSDL Meeting
by Darren Pulsipher 14 Feb '05

14 Feb '05

BEGIN:VCALENDAR PRODID:-//Microsoft Corporation//Outlook 11.0 MIMEDIR//EN VERSION:2.0 METHOD:REQUEST BEGIN:VEVENT ATTENDEE;ROLE=REQ-PARTICIPANT;RSVP=TRUE:MAILTO:jsdl-wg@gridforum.org ORGANIZER:MAILTO:darren@pulsipher.org DTSTART:20050215T140000Z DTEND:20050215T153000Z LOCATION:On Line TRANSP:OPAQUE SEQUENCE:0 UID:040000008200E00074C5B7101A82E0080000000050ED2A716612C5010000000000000000100 0000065916A0D85796149914F96C744558B0E DTSTAMP:20050214T142604Z DESCRIPTION:When: Tuesday\, February 15\, 2005 7:00 AM-8:30 AM (GMT-07:00) Mountain Time (US & Canada).\nWhere: On Line\n\n*~*~*~*~*~*~*~*~*~*\n\nYouhave been invited to attend a Microsoft Office Live Meeting. \n\nAt the time of the meeting\, click on the following link to attend. \nURL: \n http://www.placeware.com/cc/cadence/join?id=JSDL <http://www.placeware.com/cc/cadence/join?id=JSDL&role=attend> &role=attend \n\nSubject: JSDL Meetings \nN\nWhen: Tuesday\, Feb 15\, 2005 7:00 AM (MST) \nScheduled to Occur: Recurring \nDuration: 1 Hour 30 Minutes \nAudio Information: x3000 Passcode 6447 \n\nClick to add to your Outlook calendar: \nhttp://www.placeware.com/cc/cadence/meetingICS?id=JSDL <http://www.placeware.com/cc/cadence/meetingICS?id=JSDL&role=attend&i=i.ic s> &role=attend&i=i.ics \n\nAlternate Attend Instructions: \n Go to: http://www.placeware.com/cc/cadence/join \n Your Name: (enter your name) \n Meeting ID: JSDL \n\n\nDial In Numbers \nPhone Numbers:\nBelgium - 0800-7053\nBezeq - 1-800-940-1786\nFrance - 0800-903109\nGermany - 0800-1815765\nHong Kong - 800-968932\nItaly - 800874509\nJapan - 00531123656\nKorea-Tele - 00798-14-800-0512\nNetherlands - 0800-0225447\nSingapore - 800-1200766\nSweden - 020-7905043\nTaiwan - 00801-13-8546\nUK - 0800961294\nUS - 800-933-3594 \n\n\n\n SUMMARY:JSDL Meeting PRIORITY:5 X-MICROSOFT-CDO-IMPORTANCE:1 CLASS:PUBLIC BEGIN:VALARM TRIGGER:-PT15M ACTION:DISPLAY DESCRIPTION:Reminder END:VALARM END:VEVENT END:VCALENDAR

1 0

[Fwd: [ogsa-wg] Basic EMS BOF]
by Andreas Savva 10 Feb '05

10 Feb '05

FYI. This is along the same thread as the emails that Ali forwarded yesterday. -- Andreas Savva Fujitsu Laboratories Ltd

1 0

[ogsa-wg] Teleconference minutes - February 7, 2005 (fwd)
by Ali Anjomshoaa 09 Feb '05

09 Feb '05

FYI, attached are the minutes of the OGSA-WG telecon. Can I draw your attention to the Job Submission (or whatever its name ends up as) BoF being proposed. Cheers, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 ------------------------------------------------------------- ---------- Forwarded message ---------- Date: Tue, 08 Feb 2005 16:32:16 +0900 From: Andreas Savva <andreas.savva(a)jp.fujitsu.com> To: 'ogsa-wg' <ogsa-wg(a)gridforum.org> Subject: [ogsa-wg] Teleconference minutes - February 7, 2005 Minutes attached. Gridforge: https://forge.gridforum.org/projects/ogsa-wg/document/minutes-20050207/en/1 or http://tinyurl.com/5fxfd/minutes-20050207/en/1 [Note that there is no call planned for Wednesday and next week is the face to face.] -- Andreas Savva Fujitsu Laboratories Ltd OGSA teleconference - 7 February 2005 ====================================== * Participants Jay Unger (IBM) Andreas Savva (Fujitsu) Dave Snelling (Fujitsu) Hrabri Rajic (Intel) Mark Morgan (UVa) Tom Maguire (IBM) Hiro Kishimoto (Fujitsu) Andrew Grimshaw (UVa) Dave Berry (NeSC) Minutes: Andreas Savva * Minutes for January 31 approved with no changes ** BoF preparation ** Simple Job Submission BoF - Driven by the need to define normatively a minimal subset of EM so that an EM pofile might go ahead. - Andrew is pulling things together - He has talked to Bill Nitzberg (area director) - He has told Stacey that a BoF application will be sent by Friday, the deadline for BoF scheduling requests. - Working on the draft application - Simple Job Submission may not be the right title. (It is more along the lines of 'sub-EM' or 'minimal EM.') - The proposed schedule is likely to be quite aggresive in order to meet various dependencies (and GGF expectations). For example, - First draft by June; and end of year as target delivery date - Clarified that there are 3 relevant documents: 1. BoF request (required to schedule a BoF) 2. Draft charter (good to have but not essential to schedule a BoF) 3. Starting point draft (as (2)) - For Friday's submission (1) and possibly part of (2) is good enough. - There is a minimal EM draft. (Based on the refactoring/renaming that Andrew, Mark, Chris did last year to address OGSA version 1.0 terminology problems.) - Starting with the EMS draft, ignore all services except container. Require a description of what to start and where (which container to use); and start it up. Return an EPR (or perhaps a Name). - Andrew and Mark worked with other UK participants in the mini-EMS F2F on this draft last week while in the UK. The result is a further simplification: http://www-unix.gridforum.org/mail_archive/ogsa-wg/2005/02/msg00009.html - It looks like there is a lot of interest. Many names were mentioned as possible chairs for the group. (No decisions yet.) - Agreed that when sending out the announcement all relevant people should also be cc-ed. - Need to also try to get Globus, SUN, and other parties involved. ** ByteIO BoF - Similar approach to the one described above. BoF request by Friday, tell the relevant area directors and so on. - Dave (B) will do the BoF application. - Again the BoF proposal was discussed during last week's UK meeting and many names came up as likely chairs. (Nothing fixed yet.) * UK meeting updates [The minutes of these meetings are being prepared by Mark and will be distributed once the participants review them. Very brief notes here.] ** OGSA Data mini-F2F - Dave (B) and Andrew Grimshaw gave an update. - Things seem to be coming together for a general data architecture - Data transport as an issue left for discussion at GGF13 ** OGSA EMS mini-F2F - Two day meeting with a number of local participants - Worked on simplifying the EMS draft. Produced the draft sent to the list (see url above). Some UML included. (Note that people listed have not necessarily endorsed the content yet.) ** Monday meeting (e-Science) - Geoffrey Fox presentation on 2 core model - People will not converge on one model any time soon. - UK e-Science should be prepared to cover both WSRF & WS-I++ - Still not sure what the objections are to WSRF; WS-I++ ends up trying to do same things in slightly different way. - Proposal to discuss at a later time, perhaps during the F2F, the issue of OGSA dependence on WSRF as well as the opposition to WSRF. (Meanwhile Andrew is following up by talking with some people known to object to WSRF in order to get a better idea of the issues.) - There seem to be both political and technical issues. OGSA-WG should try to understand and work through the technical issues and try to reduce the division, at least within the grid community. - OGSA Profiles: 'conformance' and 'compliance' - Some issues were raised about what is in the BP - Andrew emphasized that the BP is not finished so people who have comments should (and are welcome to) contribute. * Update from WSRF/WSN F2F ** WSRF - Good progress. Many issues resolved. (Issue of which version of WS-Addressing to reference is still open). - A committee draft should be out soon (perhaps within the next couple of months). - There are still some questions with the Service Group definition. ** Base Notification/Eventing There are some places where Base Notification is more complex than WS-Eventing. It is possible to refactor Base Notification but there is no point doing so until the WS-Eventing co-authors respond to the offer sent to them. * F2F logistics - Phone line cost (for bridge) will be covered by Fujitsu. - Meeting room cost will be covered by Andrew. - Network access: - It is too expensive to get a network connection in the meeting room. - There is free wireless in lobby - And high speed network access in the rooms (not free) - Agreed that this is a reasonable compromise. - Projector will be provided by Takuya Mori - Deal is dependent on enough people staying at the hotel. Hiro got a $125/day room rate which is better than the online one. - Hiro has to finalize the deal before people can make room reservations at this rate. And the cutoff date is likely to be very soon. People attending should make sure to reserve soon after Hiro announces it on the list. [In case of questions email Hiro or Andreas.] * Next call: Agreed to cancel Wednesday's call since many people cannot make it due to GlobusWorld. Hiro will send F2F updates to the list.

1 0

JSDL: Re: [ogsa-wg] OGSA EMC BoF on Simple job submission (fwd)
by Ali Anjomshoaa 09 Feb '05

09 Feb '05

FYI. Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 ------------------------------------------------------------- ---------- Forwarded message ---------- Date: Mon, 07 Feb 2005 11:56:50 -0600 From: Ian Foster <foster(a)mcs.anl.gov> To: "Rajic, Hrabri" <hrabri.rajic(a)intel.com>, Tom Goodale <goodale(a)cct.lsu.edu> Cc: saga-rg(a)ggf.org, ogsa-wg(a)ggf.org, Bill Nitzberg <bill(a)computer.org>, Stephen Pickles <stephen.pickles(a)man.ac.uk> Subject: Re: [ogsa-wg] OGSA EMC BoF on Simple job submission Hrabri: I would think that the job submission thing we need is almost completely trivial, as noted in earlier emails on ogsa-wg(a)ggf.org: it is basically a "create" call, with a JSDL string; the (also very simple) WSRF mechanisms then address the few other functions needed. The complexities all live in the JSDL, not the WSDL. On the API side, I'd strongly encourage the SAGA guys to look at the GT4 GRAM API, as it is makes it easy to do simple things, while also addressing many other things like staging, delegation, etc. See http://www-unix.globus.org/toolkit/docs/development/4.0-drafts/execution/ws…. Ian. At 09:15 AM 2/7/2005 -0800, Rajic, Hrabri wrote: >Tom, > >Are OGSA EMC people aware of the most recent SAGA-RG work? Would >SAGA-RG people like to engage OGSA EMC and possibly co-sponsor a Simple >job submission WG? This could be a nice opportunity to tie in the >classic API and the WSDL approach. > >One thing to consider: Do we really need APIs in SRM, APME, and now in >ARCH areas? > >Hrabri > > >---- from "[ogsa-wg] Proposed agenda for Feb. 7th call" mail by Hiro >------- > >4) Simple job submission BoF preparation (30 min) > From the minutes of Jan. 19 call. > > - There is a plan to do an EM-related BoF. Stacey should know about > > it but it is not the job of the OGSA-WG to tell her---the > > proposed-chairs should do it. Preparation has not progressed to > > that point yet so Andrew as Area Director will take care of it. > > > Action: Andrew as Area director to send an email to the directors of > > the relevant Area (SRM: Bill, Stephen) to let them know of > > the proposed BoF; also to Stacey to reserve a slot. > > > - Andrew mentioned that on the Data call there was a discussion for > > a similar BoF for Data. > > -Hrabri > _______________________________________________________________ Ian Foster www.mcs.anl.gov/~foster Math & Computer Science Div. Dept of Computer Science Argonne National Laboratory The University of Chicago Argonne, IL 60439, U.S.A. Chicago, IL 60637, U.S.A. Tel: 630 252 4619 Fax: 630 252 1997 Globus Alliance, www.globus.org

1 0

Splitting Application
by Donal K. Fellows 09 Feb '05

09 Feb '05

This is the summary of the other part of Igor's proposal. Basically, the proposal is that all elements inside Application should be done as extensibility elements except for ApplicationType and Description. Then, the other current (v0.9 of the doc) elements inside Application should be moved to another XML namespace that describes "executable" type jobs (i.e. jobs that are fundamentally the running of an executable even if that happens to be hidden away inside). Then, the other types of jobs (particularly web-services invocations and SQL queries) that have been mentioned involve other sets of extensibility elements inside. For the "executable" elements, I suggest the following namespace: http://www.ggf.org/namespaces/2005/02/jsdl-executable.xsd This should apply to the following elements: ApplicationName ApplicationVersion Argument Environment Error Executable Input Output ProcessCount TileSize WorkingDirectory The interpretations of these elements laid out in section 4.2.2.8 (under the 'executable' type) is correct and consistent. Other types that have been suggested to me which fit under this banner include 'bat' and 'dll' (both applicable to DOS/Win only; they're different types because they both really need different invokation sequences - 'cmd.exe /c <blah>' and 'rundll.exe <blah>' respectively - with these being details that it is usually a good idea to abstract away.) We need to add a note that the ApplicationType is *not* describing the format of the executable. (Thanks to Chin Chee-Kai for raising these issues.) Arguably, Limits should be done this way too, since they're only meaningful to executable-type jobs. For a "webService" invocation, the remaining body of the Application element (apart from the ApplicationType and Description) should be a normal SOAP envelope. The processing engine should use the JSDL Resource section(s) to decide where to run hand off the invocation to and update the SOAP header as appropriate (I think this is done by the wsa:To or wsa:Via elements, but we'll need to get this checked by someone who knows more than I do.) I don't know what to do about handling the response from the invocation. ;^) For a "sql" query, unless there is something already specified that we can use, I suggest that the following extra elements be defined in the namespace: http://www.ggf.org/namespaces/2005/02/jsdl-sql.xsd Elements: Query Parameter Output OutputFormat The Query element should contain an SQL query (formally as a string?). Following on from what I wrote yesterday on a document-oriented schema, it may also appear as a direct child of jsdl:JobDefinition, may be imported, and may have ref and name attributes. The query should be suitable for use as a Prepared Statement if any Parameter elements are present. There should be one Query as a child of jsdl:Application. The Parameter element may appear multiple times, and each represents a single parameter to a prepared statement set up using the Query element. Order matters (which is normal for these things it seems). The Output and OutputFormat elements (should the format be an attribute of the Output element?) specify where the result of the query is to go and in what format. (Is there someone else's spec we can use here?) I think that covers the summary of what's implied in Igor's proposal for splitting up the Application element's contents. Note that what I've written about SQL is not much more than a kite I'm flying. ;^) Donal.

2 1