August 2006 - jsdl-wg - lists.ogf.org

[Fwd: Proposed extension]
by Michel Drescher 12 Sep '06

12 Sep '06

Folks, meanwhile I figured out (thanks Joel!) that the Argonne mailman doesn't like ZIP files. :-( I attached a renamed ZIP file - rename it to "Archive.zip" and it should work out of the box. Cheers, Michel -------- Original Message -------- Subject: Proposed extension Date: Fri, 07 Jul 2006 13:55:10 +0100 From: Michel Drescher <Michel.Drescher(a)uk.fujitsu.com> To: JSDL WG <jsdl-wg(a)gridforum.org> Folks, sorry for the delay - my fault. I sent the revised proposal to the "wrong" mail address jsdl-wg(a)ggf.org. Obviously, this mail address is a dead end. :-( Please find attached two proposed JSDL extensions. The attachment contains the schemas, and a list of examples. NOTE: The multi job extension requires two backwards compatible changes to the JSDL v1.0 schema itself: a) jsdl:JobDescription gets an "id" attribute of type xsd:ID b) jsdl:JobDefinition allows more than one jsdl:JobDescription element. I will update/create the necessary specification documents after the next round of rotten eggs. ;-) Cheers, Michel -- Michel <dot> Drescher <at> uk <dot> fujitsu <dot> com Fujitsu Laboratories of Europe +44 20 8606 4834 -- Michel <dot> Drescher <at> uk <dot> fujitsu <dot> com Fujitsu Laboratories of Europe +44 20 8606 4834

6 13

POSIXApplication Limits
by Peter G. Lane 07 Sep '06

07 Sep '06

I'm confused about many of the limits in POSIXApplication. Some seem redundant with respect to the Resources requirement elements. Other's simply don't seem clear in their spec descriptions. In general, it seems unclear whether "job" refers to the entire job or a single process. For example, MemoryLimit says "...the maximum amount of physical memory that the job should use when executing", whereas CPUTimeLimit says "...the number of CPU time seconds a job is allowed to consume before a SIGXCPU signal is sent to the job". In the first case I would interpret "job" to mean the one or more processes a job spawns in the process of achieving completion. In the second case it seems that "job" means an individual process. Is it then impossible to specify a total job CPU limit for all processes, or is that what TotalCPUTime in Resources is meant for? But then it's not so clear why MemoryLimit is not redundant with TotalPhysicalMemory. Here are some more specific questions: WallTimeLimit -- Why isn't there a hard wall time limit if this is a soft limit? DataSegmentLimit -- What is meant by a data segment? VirtualMemoryLimit -- Similar to MemoryLimit, what's the difference between this and an UpperBoundedRange value for Resources' TotalVirtualMemory? Is there some reason someone would want to request nodes with a total virtual memory capacity greater than they are limiting their job to use? Thanks! Peter

3 4

Re: [ogsa-hpcp-wg] Minutes for OGSA HPC Profile telecon (Aug 18 2006)
by Donal K. Fellows 07 Sep '06

07 Sep '06

humphrey(a)cs.virginia.edu wrote: > OGSA HPC Profile Minutes – Fri Aug 18 2006 [...] > AI-HPCP-0818a: Donal Fellow to review HPC Profile Application. > Specifically to ensure that normative text use is correct. Also, adding > normative text. Hopefully done before the next JSDL call on Wed. As promised, I've reviewed the HPC Profile Application extension document and applied a few changes (though fewer than I originally thought). I've uploaded a change-tracked version to GridForge (in the JSDL documentation tree) which has my changes in it together with comments detailing a few issues that I think could do with addressing before it goes final. (Some comments just describe reasoning behind one or two of the changes I've made though.) https://forge.gridforum.org/sf/docman/do/viewDocument/projects.jsdl-wg/docm… The main thing of note I suppose is that I've beefed up the Security Considerations to state that it is important that the UserName element not be allowed to do end-runs around security. But that needs checking by someone else to ensure that I've not gone too far and required that implementations *have* a security context that could allow deduction of what username to use. That's just the sane thing to do... :-) Cc:ing the JSDL WG so that they can get a chance to review the doc (plus my comments) too. Donal.

5 14

Questions about JSDL specification
by Alain Roy 01 Sep '06

01 Sep '06

Hello, I have been reading the JSDL specification, and I have a few questions about it. 1) I want to make sure that I understand "support" and "satisfy" as defined on page 7. If a system can parse a JSDL document, then it supports it, and if it can do everything requested by the JSDL document, then it satisfied it. Is that correct? 2) Just to make sure I understand support, when it says "The JSDL core element set contains the semantics for elements that are defined by JSDL 1.0. All elements MUST be supported by JSDL 1.0 compliant consuming systems.", that simply means that they need to be parsed, not that the need to be accepted, correct? 3) ApplicationName identifies the executable. I don't understand where the executable comes from. Do we expect that the underlying system can figure out which program to run from the name and version, and that it is pre-staged? Can applications bring their executable with them by staging it? Am I supposed to use the POSIXApplication element? I'm confused how this fits together. 3) When I specify the operating system: how do I specify linux version? Kernel version? Distro + version? Is it dependent on the underlying system? 3) IndividualCPUCount allows me to specify a range in terms of double: what does this mean? For example, if I specify that I want 3.14 CPUs, it's a legal specification, but I don't know what it means. Ditto for IndividualPhysicalMemory and IndividualVirtualMemory and IndividualDiskSpace. 3) I don't undersatnd IndividualNetworkBandwidth: bandwidth to where? Does this refer just to the local NIC? What if there are multiple NICs? 4) I'm confused how IndividualDiskSpace interacts with the filesystem element. The FileSystem element specifies how much disk space is needed on a particular file system: the IndividualDiskSpace says something about disk space, but not about where the disk space is located. Which disk space is it? What does it mean if I specify a FileSystem and IndividualDiskSpace? 5) I don't understand the difference between IndividualCPUCount and TotalCPUCount. Can I think of it as the number of CPUs on a single node, and the total number needed across all nodes? Or does it mean something different? 6) When I stage files, the destination might be a filesystem on NFS. If I'm running many jobs at the same time, the files might clobber each other unless I give the files unique names: is the user responsible for doing so, or is there some way to specify a unique identifier in the file name? For example, in Condor I can say something like File.$(Process) to get a unique filename based on the job id. Thanks for any assistance in understanding! -alain

5 7

Friday (Sept 1) conf call will cover JSDL basic execution extension, interop plans
by Marvin Theimer 30 Aug '06

30 Aug '06

Hi; This Friday's HPC Profile call will be at the usual time and "place": * Friday, 9/1, at 10AM EDT (7AM PDT) * 1-866-500-6738 (from US & Canada) or 1-203-480-8000 * Passcode: 1157025 The tentative agenda is the following: * Cover the suggested changes to the JSDL basic execution extension specification. (We didn't have quorum at today's JSDL conf. call, so we are pushing discussion to the HPC Profile conf. call and will then be sending a "summary" email to the JSDL mailing list requesting comments.) * Discuss interop plans and compliance specifications. Marvin.

1 0

Teleconference next week (August 30th, Wednesday) (fwd)
by Ali Anjomshoaa 30 Aug '06

30 Aug '06

Hi JSDL-ers, reminder of today's telecon. Details at the end of the fwded e-mail below. Cheers, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa PhD EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh, EH9 3JZ Phone: + 44 (0) 131 651 3479 United Kingdom Fax: + 44 (0) 131 650 6555 ------------------------------------------------------------- ---------- Forwarded message ---------- Date: Tue, 22 Aug 2006 19:14:17 +0900 From: Andreas Savva <andreas.savva(a)jp.fujitsu.com> To: JSDL WG <jsdl-wg(a)gridforum.org> Subject: [jsdl-wg] Teleconference next week (August 30th, Wednesday) The next call is **next week, Wednesday (Aug.30)** at the usual time; see bottom of email for complete details. A tentative agenda follows. Email the chairs if you want to add anything to this. 1. Review Extension drafts There are 3-4 drafts in various stages of completion. Draft owners should send out the latest versions before the call. 2. Confirm GGF18 plan (espl. people presenting/leading) See minutes of last call (8/16) for details. https://forge.gridforum.org/sf/go/doc13751?nav=1 3. Other business ----- Date: ** 30 August 2006 ** Time: 3pm BST / 10am EDT / 7 am PDT / 11pm JST Duration: 1 hour Dial-in numbers (toll): US: +1 718 3541071 (New York) or +1 408 9616509 (San Jose) UK: +44 207 3655269 (London) Japan: +81 3 3570 8225 (Tokyo) Complete dial-in information: http://tinyurl.com/eg48j Participant PIN: 4371991 -- Andreas Savva Fujitsu Laboratories Ltd

2 1

Apologies
by Michel Drescher 30 Aug '06

30 Aug '06

Folks, unfortunately, I won't be able to join the phone conference today, as I'll be in the air at that time. Sorry, Michel -- Michel <dot> Drescher <at> uk <dot> fujitsu <dot> com Fujitsu Laboratories of Europe +44 20 8606 4834

1 0

RE: [ogsa-wg] OGSA F2F meeting agenda on Sept. 14 & 15
by Marvin Theimer 29 Aug '06

29 Aug '06

Hi; I will only be able to attend the F2F meetings on Thursday as I have to be back in Redmond on Friday the 15th. I'd like to hold an HPC Profile session on Thursday, if possible. Parts of that session would very probably involve BES and JSDL matters as they relate to the HPC Profile. Thanks, Marvin. ________________________________________ From: owner-ogsa-wg(a)ggf.org On Behalf Of Hiro Kishimoto Sent: Tuesday, August 22, 2006 12:33 AM To: OGSA WG Subject: [ogsa-wg] OGSA F2F meeting agenda on Sept. 14 & 15 Hi all, GGF18 and back-to-back OGSA meeting are just around the corner and we need to finalize meeting agenda very soon. The following is current proposal and if you have any request or comment, please let me know ASAP. We will review and finalize meeting agenda at the upcoming Thursday call. Also, if you plan to come (or dial-in) but your name is not on the attendee list, please let me know. You should be on the list to enter facility. https://forge.gridforum.org/sf/go/wiki1555?nav=1 Thursday afternoon (2:30-6pm) * A joint EMS/Data session (Dave and Andreas, 1-2hr depending on people's availability) o Since many Data members will be leaving on Thursday it would be good to arrange some time early on Thursday. * Info Model (Ellen and Fred, 2H or half day) o agenda is overflow from work that will not get done during the 2 GGF sessions o first draft of base model objects for HPC/BES/JSDL o probably XQuery examples. Friday morning (8am-noon) * OGSA general session (Tom and Hiro, 2H) o OGSA direction and priority o document status review o winter F2F schedule o telecon schedule * JSDL session (Steve McGough, 1-2 hr) o Bashing out ideas for JSDL 2.0 Friday afternoon (1-4pm) * OGSA-DMI Update (Michel Drescher, 1hr) o This session is to update the OGSA-WG and all interested parties on the status of the OGSA-DMI work. It is expected that a first draft of the functional interface is drafted at GGF18. * OGSA-RSS (Donal Fellows, ??) o discuss the EPS and its relation to the rest of the EMS arch -- Hiro Kishimoto

2 5

call for implementers of the OGSA HPC Profile
by Marty Humphrey 28 Aug '06

28 Aug '06

Folks, As chairs of the OGSA HPC Profile WG, Marvin Theimer and I think that we're getting sufficiently close to having a profile defined that we'd like to issue the call for implementers of the OGSA HPC Profile. While there is clearly some more work to be done on the current HPC Profile draft (and the BES spec), we believe that certain ambiguities in the current HPC Profile can only be identified and resolved when parties try to get their implementations to speak with each other. We seek implementers to (a) help solidify the profile through direct interoperability testing, and (b) help define the compliance suite. Therefore, we encourage people interested in prototyping an implementation to email me (Marty Humphrey). The current version of the HPC Profile spec can be found at: http://forge.ggf.org/sf/go/doc13736?nav=1 Thanks a bunch! Marty Marty Humphrey Assistant Professor Department of Computer Science University of Virginia

1 0

IndividualCPUCount
by Peter G. Lane 22 Aug '06

22 Aug '06

Would it be possible to change the description of IndividualCPUCount to say simply that it specifies the required number of CPUs instead of the number to be allocated. The way it's worded now, match making based on the actual number of CPUs (i.e Condor) seems improperly represented by this element. Making it a little more general allows for the interpretation that a host must at very least have a certain number of CPUs, but it does not dictate the number of CPUs actually being allocated for the job. Resource managers such as LSF and PBS that only operate using CPU allocations could still be considered included in the description because if an allocation is successful then obviously the host must have that many CPUs. In addition, the following elements are worded similarly to IndividualCPUCount so as to effectively prohibit purely match making as opposed to strict allocation of host resources: IndividualVirtualMemory IndividualDiskSpace TotalDiskSpace For what it's worth, the rest of Resources' elements seem to be worded more generally (i.e. "required by the job" as opposed to "allocated to the job"). Derive from that what you will. ;-) Peter

2 2