April 2005 - jsdl-wg - lists.ogf.org

summary of topology thread
by Karl Czajkowski 13 May '05

13 May '05

As promised on the telecon, I try to summarize the "topology thread." The basic status is that Chris and I seem to be in agreement about a range of resource constraints that need expression and there is a minor open question about how (if at all) the binding of application to resource is expressed. I have talked to Chris and others from Platform a lot over the years, so I guess it is not clear whether we've made progress explaining this position to others. :-/ PROPOSED SET OF CONCEPTS What Chris and I think is that there are three critical concepts that need expression and they all relate to what allocation the job will receive. We think these should be expressed in the resource section: A. Total Number of CPUs for job B. Number of CPUs per resource (e.g. per LSF host) C. Total Number of resources I think both (B) and (C) are tiling constraints and need to default to "unconstrained" if left out of the JSDL doc. It is important to note that by expressing all of (A), (B), and (C) one can overconstrain the selection, e.g. describe redundant things or impossible things. We think this is OK and maybe even necessary in order to allow expression of the important things. All three values should be range-value for generalized constraints w/ step values, limits, etc. Another minor point is whether we want to express shared/exclusive access to resources. Saying "4 CPUs per resource" DOES NOT mean that all allocated hosts will have 4 installed CPUs. It means the job is granted the right to use 4 of the CPUs, but there could be more which are kept idle or allocated to other jobs. REPRESENTATION OF CONCEPTS The total value (A) is the difficult one, because it applies across all Resource elements but does not really belong in the Application section in our view. I suggest that a wrapping layer could be placed around Resource: <jsdl:Resources> <jsdl:TotalCPUs>jsdl:rangeValue</jsdl:TotalCPUs> ? ... other totals for RAM, VM, etc. ... <jsdl:Resource> <jsdl:Count>jsdl:rangeValue</jsdl:Count> ? <jsdl:CPUs>jsdl:rangeValue</jsdl:CPUs> ? ... other per-resource constraints ... </jsdl:Resource> + </jsdl:Resources> Make jsdl:TotalCPUs default to "<exact>1</exact>". Make jsdl:Count and jsdl:CPUs default to unconstrained. The remaining question, again, is how to represent the binding of application to resource. This is where the GRAM single/multi/mpi/condor stuff fits in to state different launch techniques. It might be appropriate to say the default is "site specific" and a reasonable assumption is that a real resource topology is selected by the scheduler and somehow communicated to the job in the form of a resources file, environment setting, or actual process set startup. One can imagine arbitrarily strange and complex scenarios here for the interesting cases. USE CASES TO ILLUSTRATE CONCEPTS Chris made a nice concise statement of use cases that need to be addressed, and I think the implication is that JSDL examples should use these cases rather than existing confusing ones: 1. Simple MPI job. Wants 32 processors with 1 processor per resource (in JSDL, a host is a "resource"). <jsdl:Resources> <jsdl:TotalCPUs><exact>32</exact></jsdl:TotalCPUs> ... <jsdl:Resource> <jsdl:CPUs><exact>1</exact></jsdl:CPUs> ... </jsdl:Resource> </jsdl:Resources> <jsdl:Resources> <jsdl:TotalCPUs><exact>32</exact></jsdl:TotalCPUs> ... <jsdl:Resource> <jsdl:Count><exact>32</exact></jsdl:Count> ... </jsdl:Resource> </jsdl:Resources> <jsdl:Resources> <jsdl:TotalCPUs><exact>32</exact></jsdl:TotalCPUs> ... <jsdl:Resource> <jsdl:Count><exact>32</exact></jsdl:Count> <jsdl:CPUs><exact>1</exact></jsdl:CPUs> ... </jsdl:Resource> </jsdl:Resources> All three of these lead to exactly the same set of possible allocations. I will not show equivalent permutations for the rest of the examples but this gives the basic idea. 2. OpenMPI job. Wants 32 processors with 8 processors per resource. <jsdl:Resources> <jsdl:TotalCPUs><exact>32</exact></jsdl:TotalCPUs> ... <jsdl:Resource> <jsdl:CPUs><exact>8</exact></jsdl:CPUs> ... </jsdl:Resource> </jsdl:Resources> 3. An OpenMP job. Wants 32 processors. Shared mem of course, so one resource. <jsdl:Resources> <jsdl:TotalCPUs><exact>32</exact></jsdl:TotalCPUs> ... <jsdl:Resource> <jsdl:Count><exact>1</exact></jsdl:Count> ... </jsdl:Resource> </jsdl:Resources> 4. A "homegrown" master/slave parallel job (say a ligand docking job). Wants 32 processors. No tiling constraints at all. <jsdl:Resources> <jsdl:TotalCPUs><exact>32</exact></jsdl:TotalCPUs> ... <jsdl:Resource> ... </jsdl:Resource> </jsdl:Resources> * Note that I'm specifically leaving out the Naregi "coupled simulation" use case (sorry guys), since we determined at the last GGF that it was a case which could be decomposed into multiple JSDL documents. Of course, these use cases leave out many interesting "loose" constraints where range-value is used to allow many possibly heterogeneous solutions to the constraint system of the JSDL doc. I would add that I think the "coupled simulation" use case can be addressed by the resource constraints we are proposing but it is in the "binding of application to resource" where multiple JSDL docs are required. For example, we can select heterogeneous resources if the single application knows how to run each different "simulation" on subsets of the allocation without expressing that binding in JSDL: <jsdl:Resources> ... <jsdl:Resource> <jsdl:Count><exact>8</exact></jsdl:Count> <jsdl:CPUs><exact>2</exact></jsdl:CPUs> ... </jsdl:Resource> <jsdl:Resource> <jsdl:Count><exact>16</exact></jsdl:Count> <jsdl:Count><exact>1</exact></jsdl:Count> ... </jsdl:Resource> </jsdl:Resources> If we wanted to address such complex scenarios more elegantly, we could allow more than one Resources element or even allow Resources elements to recursively contain Resources elements in order to express logical assemblies of resources which scope the "total" constraints expressed within them as a summation of the contained assemblies or "leaf" resource elements. I do not know if anyone cares about this sort of hierarchical resource map. karl -- Karl Czajkowski karlcz(a)univa.com

3 7

JSDL compliance levels
by Michel Drescher 04 May '05

04 May '05

Folks, I got the assignment to the following two action points from last week's phone conference: >> 8) JSDL compliance levels >> How much of jsdl a system must support, not just process. >> Probably a topic for later. >> >> 9) The statement "If not supported then the consuming system MUST >> reject the document." needs a bit more explanation or re-write To solve these action points, I refer to the specification document (v0.8.5-02) ch 2 "Notational conventions". I propose to replace the chapter's last three paragraphs (the text after table 1) with the following paragraphs: "The term JSDL documentrefers to a XML document that has been created following this specification. It is a schema instance document derived from the normative schema definition inAppendix 2:Normative Schema. The terms JSDL element and JSDL attribute indicate that the corresponding language construct is represented as an XML element and XML attribute in a JSDL document. The key word presentrefers to a construct being present in a JSDL document. The key word support refers to a consuming system being able to apply the following functions to a JSDL element or JSDL attribute: · Parse the JSDL element or JSDL attribute into a DOM tree · Validate the parsed DOM tree against the appropriate schema(s) · Interpret the JSDL element or attribute and assign the semantics according to this specification. The JSDL specification does not require the consuming system to actually implement the semantics of JSDL elements and attributes, so that the described job is executed on a computing system. A consuming system MUST support all normative JSDL elements and JSDL attributes. If a consuming system implements JSDL, it MUST implement all defined JSDL elements and attributes. This allows not only systems that execute jobs but also systems that provide higher level services, for example job broker, super scheduling systems (that possibly change JSDL documents before they get submitted to lower level job execution systems), etc." Note: The last paragraph possibly fits better in a JSDL primer document. This way any consuming system that deals with JSDL must support the defined JSDL elements and attributes, but it does not need to "ground" or "incarnate" or even execute a JSDL job. But if it implements JSDDL, then all JSDL elements and attributes must be implemented. So this basically eliminates any JSDL compliance levels. As we already reduced JSDL to the lowest common denominator, there is no need of compliance levels at all - at least for the moment. The consequence would be that we can delete all these "If not supported then ... " sentences. Cheers, Michel

2 1

a couple of questions
by Ariel Oleksiak 29 Apr '05

29 Apr '05

Hi All, That's me again. Please find another portion of questions and comments concerning the latest versions of JSDL specification and schema. These issues arisen mainly when we were trying to map JSDL to other job description languages. 1. In one of my previous emails I asked about specification of a directory in the DataStaging element and I got the answer that this is possible. Therefore, in my opinion, what we need is the attribute which distinguishes between regular file and a directory. I think that a scheduler may often need this knowledge to perform data staging (e.g. it may have to know whether a given logical name is the name of file or directory in order to call appropriate methods of a data management system). 2. I guess the NetworkBandwidth element defines the maximal bandwidth of the networking interface for a given resource. Such a metric is rather useless in a resource selection process because indeed only the actual bandwith between two nodes is really important. So didn't you think about the element defining bandwith between two nodes, e.g. containing an attribute to specify the second node? 3. Some time ago we discussed defining of software dependencies, e.g. necessary libraries etc. and I sent some simple use-cases. I do not see this element in the spec. Have you come to the conclusion that this is not general enough? 4. What is the meaning of the "other" value in the list of OSs, processor architectures etc.? Does it mean that (i) a scheduler has to select resources limited to those having OS OTHER than the remaining OSs from the list or (ii) that a certain OS, not included in the list, is required and its exact name is included in an additional element (which one?). In my opinion the latter is more rational (actually I don't see any use case for the former) but, if so, how this OS should be specified - using the Description element, extension? 5. The next question is related to the semantics of resource requirements in JSDL. Are resource requirements limited to required amounts of resources available for user's application (e.g. free CPUs, free memory etc.) in a given point of time or they also enable a user to specify required total amounts of resources (e.g. total number of CPUs, total virtual memory etc.)? Of course the former (dynamic) values are much more important during resource selection, however, information about some of them may not be available in all grid environments. Anyway this distinction must be clearly defined. 6. I see that units and operators attributes disappeared from the schema. I understand that operators can be expressed using RangeValueType and units are defined in the specification now? 7. It is not clear for me what is the difference between Range elements and Upper/LowerBoundedRange in the RangeValueType. Sorry if it has been already explained elsewhere. 8. I cannot find a way to specify a type of application in terms of its architecture (single, mpi etc.). I suppose that you consider it as too specific and as a possible extension or maybe this is an oversight? Thanks, Ariel

2 1

CIM Reference (Again)
by Donal K. Fellows 27 Apr '05

27 Apr '05

The CIM reference I was using at the F2F was a preliminary release of CIM 2.9, and currently we're only using the operating system types from them (in their human-readable versions[*]). There is also a processor typing model available, but it's just about unusable from our PoV because CIM does not support partial order relations on its enumeration types. (The problem is also there with OSes, but the amount of cross-emulation is much less.) The enumeration was taken from the ValueMap for System -> Operating System -> CIM_OperatingSystem -> OSType part of the overall model (Device sub-model, Operating System page, CIM_OperatingSystem class, OSType field), though it is necessary to read the MOFs or the XML to actually get the values. (Messy!) The processor part of the model would be the ValueMap from Device -> Processors -> Processor -> Family, except I find it hard to see many people being happy at distinguishing between "AMD Athlon(TM) 64 Processor Family" and "AMD Opteron(TM) Processor Family" for almost all applications! The reference to the final version of CIM 2.9.0 is at http://www.dmtf.org/standards/cim/cim_schema_v29 but I actually used http://www.dmtf.org/standards/cim/cim_schema_v29_prelim if that matters. Note that as far as we're concerned, I think updates to the list of OS types shouldn't be a problem. Donal. [* The core model is actually numeric, but we really don't want to go that route; it overexposes the underlying implementation details. ]

1 0

JSDL: Today's telecon minutes
by Ali Anjomshoaa 26 Apr '05

26 Apr '05

Hi everyone, today's telecon minutes are on GridForge at: https://forge.gridforum.org/projects/jsdl-wg/document/Minutes_-_2005-04-26/… For those on the call, your actions are in the minutes. Please ask for help on the list with feedback or effort. Thanks everyone. Cheers and take care, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

1 0

Limit Text
by Donal K. Fellows 26 Apr '05

26 Apr '05

The following is my suggested text for the MemoryLimit and PipeSizeLimit element definitions. The text in [brackets] is non-normative. I have omitted the text stating what it must be a sub-element of (according to the principles of allowing the schema to be used as an ontology as well as a document definition) but if we really want that, it's trivial to add it. MemoryLimit: Definition This element is a positive integer that describes the maximum amount of physical memory (in bytes) that the job should use when executing. It MUST be supported. If this is not present, then it is unlimited. [This is really the Resident Set Size limit, and systems might impose a granularity of a kilobyte.] PipeSizeLimit: Definition This element is a positive integer that describes the maximum size of pipelines (in bytes) created for and by the processing of the job. It MUST be supported. If this is not present, then the system should use its default value. [No system permits arbitrary sized pipe buffers, but their defaults are usually entirely acceptable. Systems typically impose a granularity of 512 bytes.] Donal.

1 0

Coupling PosixApplication to DataStaging
by Donal K. Fellows 26 Apr '05

26 Apr '05

Hi everyone! We have something of a problem in that we need a way to join the virtual filesystem locations that we stage files to/from in DataStaging to the places referred to in various places in PosixApplication (especially Argument and Environment, though this also applies to Input, Output and Error; possibly WorkingDirectory too). The problem is that we want to virtualize the location within the real target filesystem(s) that the files are located in, but we do not have any standard substitution scheme; so far, we've only got the ability to specify literal strings which gives either absolute locations (non-portable) or locations that are relative to the working directory (great for the simple cases, but awful for anything complex). In DRMAA, this problem is solved by allowing magical tokens to be put at the start of applicable parameters and defining a substitution scheme for those magical parameters. (This is also the UNICORE approach). That works, but I use the term "magical" with good reason; the values are not easily manipulable with XML processing tools. The other basic alternative, processing with XSLT on the way in, has the problem in that it tends to end up binding things to specific locations far too soon. What I think we need is a way to say that a certain term is to be made relative to a particular virtual filesystem root. As far as I'm concerned, the easiest way of doing this is to put optional attributes on the elements needing this treatment (listed above) that say that the element value needs to be interpreted as being relative to the root of the virtual filesystem named in the attribute, with an absent attribute indicating that the attribute value is to be used literally. Alternatively, we could put a complex type inside the element, but that's much messier IMO. I suggest that the attribute should be called jsdl:filesystem (or rather, whatever we use in DataStaging but I don't remember for sure right now) and we might use it like this fragment: <jsdl:Application> <jsdl:ApplicationName> Gaussian </jsdl:ApplicationName> <jsdl-exec:POSIXApplication> <jsdl-exec:Input jsdl:filesystem="HOME"> input.txt </jsdl-exec:Input> <jsdl-exec:StackSpaceLimit> 8388608 </jsdl-exec:StackSpaceLimit> </jsdl-exec:POSIXApplication> </jsdl:Application> We'd need to decide what predefined virtual filesystems to support, of course. Or have we done that already? :^) Donal.

2 2

OGSA-BES group
by Darren Pulsipher 26 Apr '05

26 Apr '05

For everyone's information the OGSA-BES group is having meetings on Thursday Mornings at the same time as the JSDL meetings on Tuesdays. To join the email list send mail to majordomo(a)ggf.org subscribe ogsa-bes-bof Darren

1 0

JSDL
by Darren Pulsipher 26 Apr '05

26 Apr '05

When: Tuesday, Apr 26, 2005 8:00 AM (MDT) Where: On Line *~*~*~*~*~*~*~*~*~* Here is the web conference for tomorrow -+-----+-----+-----+-----+-----+-----+-----+-----+- Darren Pulsipher has invited you to attend a Microsoft Office Live Meeting. Subject: JSDL Meeting URL: https://www.livemeeting.com/cc/xangoit/join Meeting ID: KC2MK6 Attendee Key: S|h:@K<9K Role: Attendee This Office Live Meeting invitation is a personal invitation meant only for you. It should not be forwarded. If you received this email by mistake or require Live Meeting Assistance, please refer to the Live Meeting Assistance Center at http://r.office.microsoft.com/r/rlidLiveMeeting?p1=7&p2=en_US&p3=LMInfo&p4=…

1 0

RE: JSDL Meeting on Apr 26 2005
by Bill Harris 25 Apr '05

25 Apr '05

Audio for meeting on Tuesday information 1 800-933-3594 at message dial 3000 id is 6444 and confirmed ;-) Talk to you in the morning William Harris Program Architect Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744 The contact details are: Dial In Numbers: Belgium - 0800-7053 Bezeq - 1-800-940-1786 France - 0800-903109 Germany - 0800-1815765 Hong Kong - 800-968932 Italy - 800874509 Japan - 00531123656 Korea-Tele - 00798-14-800-0512 Netherlands - 0800-0225447 Singapore - 800-1200766 Sweden - 020-7905043 Taiwan - 00801-13-8546 UK - 0800961294 US - 800-933-3594 --> ext 3000 pin: 6444 The telecom time is: wed April 26 this week 07:00 PST 08:00 MST 09:00 CST 10:00 EST 15:00 BST 16:00 CET 23:00 JST William Harris Program Architect Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744

1 0

Process Topology
by Christopher Smith 21 Apr '05

21 Apr '05

Ok ... here are my thoughts on process topology and what's currently expressible in JSDL. First, I'll list some use cases (they're all parallel jobs): 1. Simple MPI job. Wants 32 processors with 1 processor per resource (in JSDL, a host is a "resource"). 2. OpenMPI job. Wants 32 processors with 8 processors per resource. 3. An OpenMP job. Wants 32 processors. Shared mem of course, so one resource. 4. A "homegrown" master/slave parallel job (say a ligand docking job). Wants 32 processors. No tiling constraints at all. * Note that I'm specifically leaving out the Naregi "coupled simulation" use case (sorry guys), since we determined at the last GGF that it was a case which could be decomposed into multiple JSDL documents. Second ... what is process topology? It provides the user a way to express how resources should be _allocated_ given the characteristics of the job (usually in terms of IO patterns ... e.g. network communication, disk IO channel contention, etc). Thus, it's used when the resource manager is _allocating_ the resources, not when the job is being started/launched. Therefore, none of the elements used to express process topology should be in the POSIXApplication section What we have in JSDL now: ResourceCount (how many "resources" i.e. hosts I want) CPUCount (how many processors _per resource_) TileSize (how many processors to allocate per resource as a unit) ProcessCount (total number of _processes_ that the job will use to execute the job) I will argue that ProcessCount is useless for the purposes of process topology, since a) it isn't about allocation, and b) there isn't enough information to tell me how to start/launch a parallel job. It isn't about allocation since it is irrelevant to the scheduler whether I'll be computing using threads or processes. It isn't useful for launching because it doesn't tell me how to spread the ProcessCount processes given a particular allocated topology. So that leaves the rest of them. TileSize and CPUCount are pretty much the same thing. At least for 80% (or more) of the uses I've seen. The only thing that might cause them to differ is that I could possibly allocate more than one tile on a host. Given that CPUCount is a range and that we could express step values in the range (we can express step values in the range, right?), we don't need TileSize any more. Note: I'm making an assumption here that CPUCount is the number of cpus that I want from the resource, rather than an expression of how many cpus the host needs to have configured. If it is the latter, then we do need TileSize, and replace CPUCount in my examples below with TileSize. So let's see how these map to the use cases. 1. ResourceCount == 32, CPUCount == 1 -> LSF : "-n 32 -R span[ptile=1]" -> PBS : "-l nodes=32:ppn=1" (ppn=1 might be the default) 2. ResourceCount == 4, CPUCount == 8 -> LSF : "-n 32 -R span[ptile=8]" -> PBS : "-l nodes=4:ppn=8" 3. ResourceCount == 1, CPUCount == 32 -> LSF : "-n 32 -R span[hosts=1]" (hosts=1 equivalent to ptile=<-n val>) -> PBS : "-l nodes=1:ppn=32" 4. ResourceCount == 32, CPUCount == 1 -> oops ... it doesn't care about tiling ResourceCount == 1, CPUCount == 32 -> hmm ... artificial constraint ... would suck on a blade cluster ResourceCount == 1-32, CPUCount == 1,32 -> oops again ... I might get a total allocation of 32*32 cpus * there seems to be a gap! If we had a term called "TotalCPUCount" for the entire job, I could do: 4. TotalCPUCount == 32 -> LSF : "-n 32" -> PBS : "not sure how to express" It basically means to grab 32 cpus, regardless of how they are spread. Basically I just need cpus. This is used a whole hell of a lot within our customer base. So ... in summary ... I propose: CPUCount (as is if it's allocated cpus per resource) TileSize (iff CPUCount is an expression of configured cpus in a host) ResourceCount (as is ... hmmm ... maybe the default value needs to change) TotalCPUCount (how many cpus this jobs needs to run in total) -- Chris

3 12

JSDL: Telecon minutes 19 Apr 2005
by Ali Anjomshoaa 21 Apr '05

21 Apr '05

Hi all, the minutes of yesterday's telecon are on GridForge: https://forge.gridforum.org/projects/jsdl-wg/document/Minutes_-_2005-04-19/… Those on the call, you can find your actions in-lined in the minutes. Please ask for help with your actions if you don't have time or need input. Target someone you know is able to help and perhaps has time. Catch you on the call next Tuesday, when we can tick off a few more issues. Cheers, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

3 2

Re: [jsdl-wg] Process Topology
by Karl Czajkowski 20 Apr '05

20 Apr '05

On Apr 20, Andreas Savva loaded a tape reading: > Afaik Resource should be 0-n.... Thanks for pointing it out. I'll > confirm it and change the spec. > > When were heterogeneous resource mixes discussed and are there any notes > of that discussion? > I think it came up at GGF-13 and I thought it was 0-n too at the time. I got confused by the 0-1 section title in the draft... I thought we said the general model was a sequence of Resource elements each with its own localally scoped attributes and the resource count was just a shorthand to avoid writing redundant sequences for homogeneous scenarios. If it remains 0-n, I wonder if we should have an enclosing Resources element or something like that to hold the "total" metrics that summarize allocation across all resources? karl -- Karl Czajkowski karlcz(a)univa.com

2 3

Tracker status value proposal - take 2
by Andreas Savva 20 Apr '05

20 Apr '05

Ok, here's a slightly revised proposal for the status values. Again note that this is just for the *specification* tracker and not for the general 'Agenda/Topic' tracker. At the moment the tracker artifact status values are the gridforge default: open, pending, closed. These cannot be changed but new values can be defined. I propose to augment them with the following values: resolved, fixed "open", "pending", "resolved" and "fixed" would be of status class Open "closed" of status class Closed Status value definitions will be clearly documented on the tracker summary page. A typical workflow would be open->pending->resolved->fixed->closed (Michel's email gives a more detailed proposal for these transitions.) The meaning of these values (for the typical path) would be as follows: Open: the issue has been proposed by a member but not yet accepted by the WG. Pending: the issue has been accepted and is still pending resolution by the WG. (Includes 'waiting response from author' mentioned by Donal.) Resolved: The WG has decided and accepted a resolution, but the actions are still pending Fixed: The editor has completed the proposed actions, but the actions are awaiting verification. Both the schema and spec may have to be revised in order to reach this state. Closed: The issue has been resolved and is reflected in the work products of the WG. The completion of the changes must be reported by the primary editor and verified by the secondary editor (or other WG member). Issues that have been deferred or rejected also get closed with an appropriate comment. Issues that have been deferred are moved to the post-v1 tracker. (After checking the tracker settings again, "deferred" or "rejected" should probably be 'resolution' values rather than status values. The resolution field exists but is not enabled at the moment. I propose we first try out the new status values. We can come back and enable the 'resolution' field if we decide we really need it.) Andreas

4 5

[karlcz@univa.com: Re: [jsdl-wg] Process Topology]
by Karl Czajkowski 20 Apr '05

20 Apr '05

oops, hit reply instead of reply-all and missed the JSDL list... -- Karl Czajkowski karlcz(a)univa.com

2 1

process count fields
by Karl Czajkowski 19 Apr '05

19 Apr '05

I wasn't sure if Chris was taking on sorting out the posix element, so I wanted to add my memory regarding the process count fields because they seem misleading in the minutes document from the call. There are two mostly unrelated concepts: 1) a process or task count for the job, which should use the range value type 2) a posix limit "max user processes" that should have integer type and is a sibling to the other limits like stack size, VM size, file descriptors, etc. it is probably a mistake to make these have names which look too similar and incorrectly imply that they are very closely related... karl -- Karl Czajkowski karlcz(a)univa.com

1 0

None
by Ali Anjomshoaa 19 Apr '05

19 Apr '05

http://www.epcc.ed.ac.uk/~ali/WORK/GGF/JSDL-WG/index.txt -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

1 0

JSDL: Here it is: MeetingPhone.co.uk Conference Call Confirmation
by Ali Anjomshoaa 19 Apr '05

19 Apr '05

Try this: Conference Details Date: 19 April 2005 Start Time: 3:15 p.m. British Summer Time End Time: 5:10 p.m. British Summer Time UK number: Primary Dial-in Number: 0871.711.1515 (UK 10p National Rate) International number: * +44.870.088.5700 (For Callers Outside of the U.K.) * 01805.004.101 (For Callers in Germany) Participant Access Code: 20050419 Touch-Tone Quick Reference for Conference Call 0 Exit the Conference 4 Access the Help Menu 6 Toggle Mute/Unmute 8 Time Remaining in the Conference See you there!

1 0

Re: [jsdl-wg] JSDL: Telecon...
by Ali Anjomshoaa 19 Apr '05

19 Apr '05

Hang on folks, I'm about to set up a another call... Be with you in a minute. Ali On Tue, 19 Apr 2005, Michel Drescher wrote: > "... The meeting has ended." > > WTH? > > Michel > > > On 19 Apr 2005, at 14:45, Ali Anjomshoaa wrote: > > > > > ...on the hour in 15 mins... > > > > > > Agenda (last week's minutes): > > > > https://forge.gridforum.org/projects/jsdl-wg/document/Minutes_-_2005 > > -04-13/en/1 > > > > > > Web: > > > > Subject: JSDL > > Meeting > > URL: https://www.livemeeting.com/cc/xangoit/join > > Meeting ID: JSDL > > Attendee Key: Wxx*}$Hk4 > > Role: Attendee > > > > > > Phone: > > > > --> ext 3000 pin: 6444 > > > > Belgium - 0800-7053 > > Bezeq - 1-800-940-1786 > > France - 0800-903109 > > Germany - 0800-1815765 > > Hong Kong - 800-968932 > > Italy - 800874509 > > Japan - 00531123656 > > Korea-Tele - 00798-14-800-0512 > > Netherlands - 0800-0225447 > > Singapore - 800-1200766 > > Sweden - 020-7905043 > > Taiwan - 00801-13-8546 > > UK - 0800961294 > > US - 800-933-3594 > > > > --> ext 3000 pin: 6444 > > > > > > Cheers, > > > > Ali > > > > -- > > > > ---------------------------------------------------- |epcc| - > > Ali Anjomshoaa > > EPCC, University of Edinburgh > > James Clerk Maxwell Building > > Mayfield Road E-mail: ali(a)epcc.ed.ac.uk > > Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 > > United Kingdom Fax: + 44 (0) 131 650 6555 > > ------------------------------------------------------------- > > -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

2 1

JSDL: Telecon...
by Ali Anjomshoaa 19 Apr '05

19 Apr '05

...on the hour in 15 mins... Agenda (last week's minutes): https://forge.gridforum.org/projects/jsdl-wg/document/Minutes_-_2005-04-13/… Web: Subject: JSDL Meeting URL: https://www.livemeeting.com/cc/xangoit/join Meeting ID: JSDL Attendee Key: Wxx*}$Hk4 Role: Attendee Phone: --> ext 3000 pin: 6444 Belgium - 0800-7053 Bezeq - 1-800-940-1786 France - 0800-903109 Germany - 0800-1815765 Hong Kong - 800-968932 Italy - 800874509 Japan - 00531123656 Korea-Tele - 00798-14-800-0512 Netherlands - 0800-0225447 Singapore - 800-1200766 Sweden - 020-7905043 Taiwan - 00801-13-8546 UK - 0800961294 US - 800-933-3594 --> ext 3000 pin: 6444 Cheers, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

3 2

Issues for todays phone conference
by Michel Drescher 19 Apr '05

19 Apr '05

Folks, here's a list of issues. It's huge in numbers, but most of it is (IMHO) quite quickly fixable. 1) Fig. 1 revision 2) Fig. 2 revision 3) Prefixes and namespaces (check and update) 4) Namespace should be in the table (and update) 5) Section 3 (spec doc) general revision and polishing up 6) Update explanation for jsdl:other usage 7) Possibly reformat ch. 4.2.1 8) JSDL compliance levels How much of jsdl a system must support, not just process. Probably a topic for later. 9) The statement "If not supported then the consuming system MUST reject the document." needs a bit more explanation or re-write. 10) When/why did HostName change from 0-1 to 0-n? (reverted) 11) Filesystem name and type and filesystem reference in DataStaging 12) Mountpoint environment variable. Is the current approach ok, or should we provide a separate mechanism, e.g, a tag that if present provides the name of the environment variable in which the MountPoint value must be present. 13) Should FileSystemID be changed (back) to FileSystemName. 14) DRMAA use case for defining filenames. We have to at least be able to show an example of how to define and stage out an <Output> file. 15) CreationFlag. Was this made optional or not? Confirm and delete comment. 16) Process Topology needs review. In particular are the examples ok or not, are the definition correct and so on. Delete the yellow comments once that is done. Not sure if we should define the precise allocation behaviour or allow for that to be defined instead. 17) Section "Extending JSDL" needs an owner. I expect this section to be about 1 page long so should not be too much trouble. 18) Normative Extension is out of place 19) re-check terminology for process topology and delete comments 20) "Bytes (octets of bits)" 21) Memory limit incomplete text 22) pipe size limit incomplete text 23) Update of security considerations in line with latest decisions (when we actually reach firm decision) 24) Complete author information (see my entry as reference) 25) Update contributors and acknowledgements 26) Complete list of Normative references, e.g., CIM is still missing. 27) Glossary owner 28) Translation tables need to be added Proposal: Karl for Globus/GRAM, Chris for LSF, Saga-san for Naregi, ??? for Condor, ... 28a) Put the translation tables into a separate document? 29) Promised examples for the appendix (But maybe this should be split off into a Primer) 30) Make citations in text of document reference real cross-references. 31) Complete example sub-sections --- For those that read until here: Is anything missing? Cheers, Michel

7 20

Issues for All to Comment on
by Donal K. Fellows 19 Apr '05

19 Apr '05

The following issues (from the last telecon) need comment from multiple people; please feel free to jump in with anything you want to say on these topics. 12) Mapping mountpoints to environment variables (see spec 5.5.12) 16) Process topology section needs review 32) POSIXApplication/Executable cardinality (=1 vs <=1) On issue 32, my feeling is that having the Executable as optional is a good thing since this permits the processing engine to derive the executable name from the overall application name and version. I've known the same application be installed with multiple different names across multiple architectures within the same overall organization, some of which are non-trivial to guess[*], so concealing that sort of stuff from users is a distinctly good thing in my view. (Very few users want to know, and sometimes there are details of the job running process which we would prefer them not to see either.) Does anyone strongly disagree on this point re cardinality? Donal. [* Both Gaussian and Amber are nasty, but in different ways. ]

3 2

JSDL: Telecon minutes
by Ali Anjomshoaa 18 Apr '05

18 Apr '05

Hi all, the minutes of yesterday's telecon are on GridForge: https://forge.gridforum.org/projects/jsdl-wg/document/Minutes_-_2005-04-13/… Those on the call, you can find your actions in-lined in the minutes. Please ask for help with your actions if you don't have time or need input. Target someone you know is able to help and perhaps has time. Catch you on the call next Tuesday, when we can tick off a few more issues. Cheers, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

1 1

FW: Madeline Pulsipher
by Bill Nitzberg 18 Apr '05

18 Apr '05

Subject: Madeline Pulsipher From: "Darren Pulsipher" <Darren.Pulsipher(a)xango.net> To: "Ashlee Rich" <Ashlee.Rich(a)xango.net>, "Stacy Pulsipher" <stacy(a)pulsipher.org>, <jsdl-wg(a)ggf.org> Madeline joined this world at 12:21 pm on Sunday April 17,2005. 7 lbs 11 oz. 20 inches long. She looks like her older sister did which means we will have another beautiful little girl in our family. Mommy and daughter are doing great and will be home soon. Darren

1 0

Madeline Pulsipher
by darren＠pulsipher.org 18 Apr '05

18 Apr '05

Madeline joined this world at 12:21 pm on Sunday April 17,2005. 7 lbs 11 oz. 20 ½ inches long. She looks like her older sister did which means we will have another beautiful little girl in our family. Mommy and daughter are doing great and will be home soon. Darren

1 0

RE: [jsdl-wg] Any JSDL parser around
by Vivian Li 15 Apr '05

15 Apr '05

Hi, Try if this works? <?xml version="1.0" encoding="UTF-8"?> <wsdl:definitions name="xxx" targetNamespace="http://xxx" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" <wsdl:types> <xsd:schema targetNamespace="http://xxx"> <xsd:include schemaLocation="xxx/jsdl.xsd"/> </xsd:schema> </wsdl:types> </wsdl:definitions> name the above codes into xxx.wsdl, then pass to Axis. you need to replace xxx with proper ones. You could also give it a package name where you want the classes to be generated when you tool it. Regards, Vivian > -----Original Message----- > From: owner-jsdl-wg(a)ggf.org [mailto:owner-jsdl-wg@ggf.org] On Behalf Of > Aleksandr Konstantinov > Sent: 15 April 2005 12:46 > To: jsdl-wg(a)ggf.org > Subject: Re: [jsdl-wg] Any JSDL parser around > > It seems Axis can't digest jsdl.xsd. Any suggestions on how to convert it > to wsdl (if possible at all)? > > > A.K. > > > > On Fri, 15 Apr 2005 13:11:05 +0200 > Roger Menday <r.menday(a)fz-juelich.de> wrote: > > > > > On Fri, 2005-04-15 at 11:11 +0100, William Lee wrote: > > > When you say a parser, do you mean a XML-to-Object binding for JSDL > > > schema? JSDL is specified in XML, therefore any XML tools (e.g. DOM4j, > > > JDOM, JAXB in Java) will do the job. > > > > In java, I'm currently using xmlbeans, http://xmlbeans.apache.org/ for > > this kind of thing. IMHO the XML schema support is unparallelled. It > > eats the JSDL.xsd with no problems ! I guess everyone has their > > favourite in such matters though. > > > > Roger > > > > > If you are looking for a JSDL > > > consumer, Imperial College has a project called GridSAM > > > (http://www.lesc.ic.ac.uk/gridsam) which translates JSDL into > > > DRM-specific (e.g. Condor, SGE) actions. UCL has a project for > > > translating JSDL into Condor ClassAds. > > > > > > William > > > > > > Aleksandr Konstantinov wrote: > > > > > > > Hello JSDL gurus, > > > > > > > > Does anybody know any common purpose JSDL parser implementation? Or > project which is going to release such? > > > >Preferably with C/C++ API. > > > > > > > >Thanks, > > > > > > > > > > > >A.K. > > > > > > > > > > > > > > > > > > > >

2 1

Any JSDL parser around
by Aleksandr Konstantinov 15 Apr '05

15 Apr '05

Hello JSDL gurus, Does anybody know any common purpose JSDL parser implementation? Or project which is going to release such? Preferably with C/C++ API. Thanks, A.K.

4 7

Re: [jsdl-wg] Any JSDL parser around
by Chee-Kian Koh 15 Apr '05

15 Apr '05

Hi William, GridSAM looks cool! What DRMs support now? Will we see one for Grid Engine soon? > On Fri, 15 Apr 2005 11:11:44 +0100 > William Lee <wwhl(a)doc.ic.ac.uk> wrote: > > > > > When you say a parser, do you mean a XML-to-Object binding for > JSDL > > schema? JSDL is specified in XML, therefore any XML tools (e.g. > DOM4j, > > JDOM, JAXB in Java) will do the job. If you are looking for a > JSDL > > consumer, Imperial College has a project called GridSAM > > (http://www.lesc.ic.ac.uk/gridsam) which translates JSDL into > > DRM-specific (e.g. Condor, SGE) actions. UCL has a project for > > translating JSDL into Condor ClassAds. > > > > William ----------------------------------------- Melvin Koh Chee Kian Grid Research Engineer Asia Pacific Science & Technology Center Sun Microsystems Inc. Website: http://apstc.sun.com.sg/ -----------------------------------------

1 0

Naming elements
by Michel Drescher 14 Apr '05

14 Apr '05

Folks, from the last phone conference, I had the action assigned to follow the suggestion of naming elements. Based on some thinking and toying around, I propose the following layout: JSDL allows to define a) applications that are supposed to be executed, b) sets of resources that are to be consumed, c) files to be staged in and out and, finally, d) establishes associations between applications, resources and data sets. An exhaustive 3-tuple of associations is called "JobDescription". A JobDescription can also contain a set of identifiable elements serving various kind of purposes, such as accounting, billing, etc. These elements are aggregated in an element called "JobIdentification". All this is aggregated in a root element called "JobDefinition" (a name I would like to question, but more on this at a later point). This short description from a bird's eye view may not be totally new, but it gives some valuable hints on what is worth being referencable and what not: a) resources, b) applications and c) data sets. A JobDescription does not necessarily has to be referencable, since it is usually a unique association between other elements that, in this configuration, serve a unique purpose. It is adequatically specified through the document's namespace. Similar jobs (who have a different namespace) may want to use exactly the same set of resources or the same application etc. Most likely, however, the resource sets or applications etc. that have been defined for another job will not fit exactly the new job's needs, so a mechanism to tweak existing definitions could be handy. So, more technically and closer to the spec/schema, the following elements should be made referencable: - /JobDefinition/JobDescription/Application - /JobDefinition/JobDescription/Resource - /JobDefinition/JobDescription/DataStaging - Additionally, a similar mechanism exists for /JobDefinition/JobDescription/Resource/FileSystem and /JobDefinition/JobDescription/DataStaging/FileSystemID. I also propose to change FileSystemID to the type xsd:QName. - Following the convention used with the construct "xsd:group", I propose to use an attribute "name" of type "xsd:NCName" to make an element referencable. An attribute named "ref" of type "xsd:QName" is proposed to be used as a reference to a referencable element. - Sibling to JobDescription, applications, resource sets and data staging elements are allowed to be specified. They all get an attribute "name" of type "xsd:NCName" to be made referencable. - JobDescription defines local elements(!) with the same name (for applications, resource sets and data staging) and each gets an optional attribute named "ref" of type "xsd:QName" to refer to already defined elements (either in the same document, or externally). - These locally defined elements allow exactly the same appropriate groups of elements as the elements sibling to JobDescription. - Elements specified in JobDescription's local elements override elements with the same name in referred elements. - Finally, I propose to change the name of "JobDefinition" to something else, e.g. "JSDL" or something that does *not* have "Job" in its name since it does not define jobs alone anymore. (I just have no clue on a reasonable replacement.) The following proposals are of more concern to schema hackers and are related to technical issues of design and "taste". They can easily be skipped by others although they may touch higher level concerns: - I further propose to follow one of the two scenarios: a) define exactly one global element ("JobDefinition") so that any valid JSDL document always has "JobDefinition" as root element b) define a limited number of global elements, i.e. JobDefinition, JobDescription, Resource (rename to "ResourceSet"?) Application and DataStaging so that any valid JSDL document may have any of these globally defined elements as root element - Any other elements and types should be locally defined. Or do we want to allow i.e. "jsdl:NetworkBandwidth" to be root element? - Use xsd:group to group elements together and refer to these groups instead of repeating element definitions Cheers, Michel

4 12

JSDL 0.9.5-02 uploaded
by Andreas Savva 14 Apr '05

14 Apr '05

I have uploaded JSDL spec. 0.9.5-02 and corresponding schema files to gridforge. This release contains a number of changes, mainly by Michel, especially around limits as well as the rangevalue definition and explanation. Also the POSIXApplication is now split off into a separate schema. ** This is a snapshot release so it is known to have inconsistencies ** and inaccuracies. Use at your own risk. The plan is to go back to releasing a snapshot every week (or bi-weekly if there are insufficient changes) ideally before each call. People with text assignments please send me, or the list, your updates to this version in time for me to put things together. Ali when releasing the previous version said: > I propose that any outstanding issues from GGF13 be added to the spec, > incrementing the double digits at the end of the version number at each > iteration for this purpose. i.e. 0.9.5-00 to 0.9.5-01 etc. I've kept the numbering scheme for this release (note that -01 was never formally released) but I would like to propose instead to start calling the spec "JSDL version 1.0, draft XX" where the XX corresponds to the version number of the document on gridforge. E.g., this release would be "JSDL version 1.0, draft 17." URLs: https://forge.gridforum.org/projects/jsdl-wg/document/draft-ggf-jsdl-spec/e… https://forge.gridforum.org/projects/jsdl-wg/document/jsdl.xsd/en/8 https://forge.gridforum.org/projects/jsdl-wg/document/jsdl-posix.xsd/en/1 Equivalent shorter urls: http://tinyurl.com/3taa2/draft-ggf-jsdl-spec/en/17 http://tinyurl.com/3taa2/jsdl.xsd/en/8 http://tinyurl.com/3taa2/jsdl-posix.xsd/en/1 -- Andreas Savva Fujitsu Laboratories Ltd

1 0

RE: JSDL Meeting
by Bill Harris 13 Apr '05

13 Apr '05

I apologize for fat fingering the meeting time to 7:00 AM instead of eight, no excuses just in a hurry. Meeting are scheduled for 8:00 AM mountain time every Tuesday in the future. Darren has asked that we reschedule for Wed this week so meeting time is 8 AM Mountain Wed April 13, 2005. Darren has sent out the issues list for discussion. my apologies again billh meeting tomorrow ........... information below telecomm only... Darren will be sending out web information William Harris Program Architect Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744 The contact details are: Dial In Numbers: Belgium - 0800-7053 Bezeq - 1-800-940-1786 France - 0800-903109 Germany - 0800-1815765 Hong Kong - 800-968932 Italy - 800874509 Japan - 00531123656 Korea-Tele - 00798-14-800-0512 Netherlands - 0800-0225447 Singapore - 800-1200766 Sweden - 020-7905043 Taiwan - 00801-13-8546 UK - 0800961294 US - 800-933-3594 --> ext 3000 pin: 6444 The telecom time is: wed April 13 this week 07:00 PST 08:00 MST 09:00 CST 10:00 EST 15:00 BST 16:00 CET 23:00 JST William Harris Program Architect Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744

3 2

POSIX UID/GID
by Donal K. Fellows 13 Apr '05

13 Apr '05

The telecon has decided to move the UID/GID bits from User to the POSIX version of Application (whatever that's called; I'm behind tracking the exact name of things right now). But a question has come up about the meaning of the content of these elements. It is obvious that they have to be able to accept usernames in their xsd:string content (and I'm aware of systems with spaces in usernames, even if this is not something I'd ever do myself) since they're how POSIX user identities are exposed to users normal. However, with names like they have, should they also accept numeric UIDs? I don't have a use case for this (it's a detail I'd want to always hide) and I'm not aware of anyone ever having a wholly numeric username. As I see it, we have three options: * Current name, user-name semantics * Current name, user-name-and-uid semantics * Change name (to UserName?), user-name semantics Which do people prefer? (GID handling should mirror UID handling) Donal.

3 3

JSDL: Telecon...
by Ali Anjomshoaa 13 Apr '05

13 Apr '05

...on the hour in 15 mins... Agenda: http://www-unix.gridforum.org/mail_archive/jsdl-wg/2005/04/msg00054.html Web: Subject: JSDL Meeting URL: https://www.livemeeting.com/cc/xangoit/join Meeting ID: JSDL Attendee Key: Wxx*}$Hk4 Role: Attendee Phone: --> ext 3000 pin: 6444 Belgium - 0800-7053 Bezeq - 1-800-940-1786 France - 0800-903109 Germany - 0800-1815765 Hong Kong - 800-968932 Italy - 800874509 Japan - 00531123656 Korea-Tele - 00798-14-800-0512 Netherlands - 0800-0225447 Singapore - 800-1200766 Sweden - 020-7905043 Taiwan - 00801-13-8546 UK - 0800961294 US - 800-933-3594 --> ext 3000 pin: 6444 Cheers, Ali -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

1 0

telecon error "the meeting has ended"?
by Karl Czajkowski 12 Apr '05

12 Apr '05

I cannot seem to dial in using the call info sent earlier to the list... anyone know what is up? karl -- Karl Czajkowski karlcz(a)univa.com

1 0

Requirements and preferences on resources
by Massimo Sgaravatto - INFN Padova 12 Apr '05

12 Apr '05

Dear all I have just recently joined this mailing list, and so maybe the question that I am going to raise have already been discussed: my apologies if this is the case. My question concerns the Resource element ... It is not clear to me if/how it is possible to specify not only requirements, but also preferences on resources, for example to allow a smart brokering service to choose among multiple resources matching the resource requirements. If you are familiar with Condor, I am referring to something similar to Requirements and Rank of Condor classads Thanks Cheers, Massimo \\\|/// \\ ~ ~ // (/ @ @ /) -------oOOo-(_)-oOOo---------------------------------- Massimo Sgaravatto INFN Sezione di Padova Via Marzolo, 8 35131 Padova - Italy Tel: ++39 0498277047 Fax: ++39 0498277102 oooO E-mail: massimo.sgaravatto(a)pd.infn.it ( ) Oooo Home page: http://www.pd.infn.it/~sgaravat --------\ (----( )---------------------------------- \_) ) / (_/

2 1

RE: [jsdl-wg] JSDL: Telecons
by Bill Harris 11 Apr '05

11 Apr '05

meeting tomorrow ........... information below telecomm only... Darren will be sending out web information William Harris Program Architect Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744 The contact details are: Dial In Numbers: Belgium - 0800-7053 Bezeq - 1-800-940-1786 France - 0800-903109 Germany - 0800-1815765 Hong Kong - 800-968932 Italy - 800874509 Japan - 00531123656 Korea-Tele - 00798-14-800-0512 Netherlands - 0800-0225447 Singapore - 800-1200766 Sweden - 020-7905043 Taiwan - 00801-13-8546 UK - 0800961294 US - 800-933-3594 --> ext 3000 pin: 6444 The telecon time is: _every_Tuesday_ 07:00 PST 08:00 MST 09:00 CST 10:00 EST 15:00 BST 16:00 CET 23:00 JST

1 0

Tracker status values proposal
by Andreas Savva 11 Apr '05

11 Apr '05

This is a procedural proposal about the specification tracker settings: At the moment the tracker artifact status are the gridforge default: open, pending, closed. These cannot be changed but new values can be defined. I propose to augment them with the following values: resolved, fixed, deferred. The meaning of these values would be as follows: Open: the issue has been proposed by a member but not yet accepted by the WG. Pending: the issue has been accepted and is still pending resolution by the WG Resolved: The WG has decided and accepted a resolution, but the actions are still pending Fixed: The editor has completed the proposed actions, but the actions are awaiting verification. Closed: The issue has been resolved and is reflected in the work products of the WG. The completion of the changes must be reported by the primary editor and verified by the secondary editor (or other WG member). Deferred: The WG has decided that the issue has some merit but that it should not be addressed in the upcoming revision of the spec. And deferred items would later be moved to the post-v1 tracker. Note that all this is not my invention. I am 'stealing' it from the ogsa-wg trackers. (And those were in turn 'stolen' from the ogsi ones or other places ....) -- Andreas Savva Fujitsu Laboratories Ltd

6 12

my view on execution user and group
by Karl Czajkowski 08 Apr '05

08 Apr '05

The concrete use case I have from Globus is to indicate under which POSIX username a job should be executed. I see group (actually, default or primary group) as a reasonable extension of the concept, though we do not do that today. This use case is job configuration details, just like POSIX limits, environment variables, etc. It is important in situations where there is not a one-to-one mapping of global user identity (PKI subject) to local accounts for the GRAM service to apply, e.g. when a user has several role-specific accounts or temporary "scratch" accounts. Of course, such configuration values are subject to authorization checks in practice. I am not clear on whether this is the intended purpose of the existing user and group fields in the User section, but if it is not then I am not clear on what it is for at all... I admit my bias is towards thinking of the use of JSDL in distributed protocols like GRAM. I guess if we were to use it in the local "adapter" interface between the GRAM service and the local scheduler scripts, we might have a use for specifying the authenticated user name, which for GRAM would be a PKI subject name. This makes sense because the adapter scripts inherently trust GRAM; GRAM would reject the presence of such a field in the over-the-wire transmission of JSDL, since the user identification comes from the binding-level security mechanisms. I can see this kind of info belonging in a User section for the adapter case, but I still need the POSIX username in the executable section. karl -- Karl Czajkowski karlcz(a)univa.com

12 28

RE: [jsdl-wg] RE: [drmaa-wg] making a custom file path for output
by Rajic, Hrabri 06 Apr '05

06 Apr '05

Tricky. It would need to be XTSL like preprocessing on both the client side and on the compute side. I wonder if the differences between Grid and Compute Center would require any specializations. There is much more legacy stuff that DRMAA needs to address that GRAM. -Hrabri >-----Original Message----- >From: Donal K. Fellows [mailto:donal.k.fellows@manchester.ac.uk] >Sent: Wednesday, April 06, 2005 8:33 AM >To: JSDL WG >Cc: Rajic, Hrabri; DRMAA Working Group >Subject: Re: [jsdl-wg] RE: [drmaa-wg] making a custom file path for output > >Ali Anjomshoaa wrote: >> we discussed this issue at some length at GGF13 and were unsure as to how >> to proceed. We would like to be able to reference labeled attributes, but, >> I think we're still discussing how to implement this issue in the xsd >> schema depending on current tooling! >> >> I think, unfortunately, our choice of how we implement this in xsd will >> influence how we specify how to do this in the spec. Clearly this is not >> ideal and should be the other way round! > >> On Tue, 5 Apr 2005, Rajic, Hrabri wrote: >>>Placeholders would be special variables like home directory or job id ($ >>>sign prefixed below), which DRMAA could substitute in job attributes >>>like the output path: >>> >>>drmaa_set_attribute( $jt, DRMAA_OUTPUT_PATH, >>> $DRMAA_HOME_DIR/MY_APP/my_job_output.$DRMAA_JOB_ID.out" > >Hmm, I have to say that I don't like arbitrary substitutions in XML >content, especially in styles like this. (It's one of the bits I do not >care for about Ant, BTW.) But the way I'd handle such a problem is >through something like submitting the job through a service that applied >an XSLT substitution to the document before producing a doc with "ground >terms". If you see what I mean. > >In the specific case of wanting to have file arguments given to the >application made relative to one of our virtual filesystem concepts, >there is an argument to be made for some improvement of the JSDL spec. >At the moment, the only mechanisms we have for arguments are "relative >to the job working directory" or "absolute", neither of which are great >for this use-case. (The same argument applies to environment variables, >of course.) > >Does this brain-dump indicate the sort of area where we need to address >some kind of solution? Or are there other cases which will need more >refinement before we work out how to fix it all? > >Donal.

1 0

RE: [drmaa-wg] making a custom file path for output
by Ali Anjomshoaa 06 Apr '05

06 Apr '05

Hi Hrabri, we discussed this issue at some length at GGF13 and were unsure as to how to proceed. We would like to be able to reference labeled attributes, but, I think we're still discussing how to implement this issue in the xsd schema depending on current tooling! I think, unfortunately, our choice of how we implement this in xsd will influence how we specify how to do this in the spec. Clearly this is not ideal and should be the other way round! Watch the JSDL space and let us know how you do this in the end. Cheers and take care, Ali On Tue, 5 Apr 2005, Rajic, Hrabri wrote: > Hi Ali, > > Placeholders would be special variables like home directory or job id ($ > sign prefixed below), which DRMAA could substitute in job attributes > like the output path: > > drmaa_set_attribute( $jt, DRMAA_OUTPUT_PATH, > $DRMAA_HOME_DIR/MY_APP/my_job_output.$DRMAA_JOB_ID.out" > > > > -Hrabri > > >-----Original Message----- > >From: Ali Anjomshoaa [mailto:ali@epcc.ed.ac.uk] > >Sent: Tuesday, April 05, 2005 10:59 AM > >To: Rajic, Hrabri > >Cc: Andreas Haas; Tim Harsch; DRMAA Working Group > >Subject: RE: [drmaa-wg] making a custom file path for output > > > > > >Hi Guys, > > > >If you could give me a quick synopsis of what you consider placeholders > >and in what context, perhaps I can help from the JSDL perspective. > > > >Cheers, > > > >Ali > > > > > >On Tue, 5 Apr 2005, Rajic, Hrabri wrote: > > > >> Andreas, > >> > >> I would suggest we approach the question as two issues: > >> > >> 1) what are the global placeholders in DRMAA > >> 2) in which (string typed) job attributes the placeholders could be > used > >> > >> > >> Here are few questions that go with the above: > >> > >> How is JDSL handling placeholders - nice to know? > >> Do all major DRM systems support placeholders well? > >> Do we need a new error code indicating failure in resolving a > >> placeholder? Any other errors needed? > >> > >> -Hrabri > >> > >> > >> >-----Original Message----- > >> >From: owner-drmaa-wg(a)ggf.org [mailto:owner-drmaa-wg@ggf.org] On > Behalf > >> Of > >> >Andreas Haas > >> >Sent: Tuesday, April 05, 2005 7:37 AM > >> >To: Tim Harsch > >> >Cc: DRMAA Working Group > >> >Subject: Re: [drmaa-wg] making a custom file path for output > >> > > >> >Hi Tim, > >> > > >> >I fully agree there is a need for this. Please see > >> > > >> > http://forge.gridforum.org/tracker/?aid=805 > >> > > >> >I raised it's priority to P2. I believe that issue must > >> >be solved with resubitted DRMAA spec. > >> > > >> >Regards, > >> >Andreas > >> > > >> > > >> >On Mon, 4 Apr 2005, Tim Harsch wrote: > >> > > >> >> I'm sorry I wasn't clear. I guess the heart of my question is: > How > >> can > >> >> create an arbitrarty template for the file and path name? > >> Specifically > >> >what > >> >> constant tells the DRM to insert job id into that filename > template. > >> >> > >> >> Said in Perl psuedo form: > >> >> > >> >> drmaa_set_attribute( $jt, > >> >> $DRMAA_OUTPUT_PATH + > ":/tmp/my_job_output.$DRMAA_JOB_ID.out" > >> ); > >> >> > >> >> # with that code job outputs look like > >> >> /tmp/my_job_output.1.out > >> >> /tmp/my_job_output.2.out > >> >> /tmp/my_job_output.3.out > >> >> > >> >> # without that code job outputs look like > >> >> /home/harsch/script_name.o1 > >> >> /home/harsch/script_name.o2 > >> >> /home/harsch/script_name.o3 > >> >> > >> >> Is there a DRMAA_JOB_ID constant? > >> >> > >> >> ----- Original Message ----- > >> >> From: "Rajic, Hrabri" <hrabri.rajic(a)intel.com> > >> >> To: "DRMAA Working Group" <drmaa-wg(a)gridforum.org> > >> >> Sent: Monday, April 04, 2005 12:02 PM > >> >> Subject: RE: [drmaa-wg] making a custom file path for output > >> >> > >> >> > >> >> There is a simple solution, provided it works for you: > >> >> to rely on the shell to supply the home directory path instead of > >> DRMAA. > >> >> > >> >> -Hrabri > >> >> > >> >> > >> >> >-----Original Message----- > >> >> >From: owner-drmaa-wg(a)ggf.org [mailto:owner-drmaa-wg@ggf.org] On > >> Behalf > >> >> Of > >> >> >Tim Harsch > >> >> >Sent: Monday, April 04, 2005 1:19 PM > >> >> >To: DRMAA Working Group > >> >> >Subject: [drmaa-wg] making a custom file path for output > >> >> > > >> >> >Hi all, > >> >> > I have a requirement for the app I'm writing ( using > >> >> Schedule::DRMAAc ) > >> >> >which I'm trying to implement. > >> >> > > >> >> >Normally the output files from jobs go to home directory and are > of > >> the > >> >> >form > >> >> ><jobname>.o<jobid> > >> >> > > >> >> >I want to move the jobs to a subdir in /tmp, but retain the > <jobid> > >> in > >> >> the > >> >> >filename. I tried this: > >> >> > > >> >> > ( $job_error, $diagnosis ) = drmaa_set_attribute( $jt, > >> >> > $DRMAA_OUTPUT_PATH, > >> >> > ":" . $tmpdir . '/' . > >> >> > __PACKAGE__ . '.' . > >> >> > $DRMAA_PLACEHOLDER_INCR . > >> >> > ".out" ); > >> >> > > >> >> >... but PLACEHOLDER is only for parametric jobs, I've been > scanning > >> the > >> >> >docs > >> >> >and have not found a placeholder for job id. Any ideas on how I > >> could > >> >> >accomplish this task? > >> >> > > >> >> >A similar task would be to move the job output path without > changing > >> >> the > >> >> >filename format at all. How could I do that? > >> >> > >> >> > >> > >> > >> > > > >-- > > > > ---------------------------------------------------- |epcc| - > > Ali Anjomshoaa > > EPCC, University of Edinburgh > > James Clerk Maxwell Building > > Mayfield Road E-mail: ali(a)epcc.ed.ac.uk > > Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 > > United Kingdom Fax: + 44 (0) 131 650 6555 > > ------------------------------------------------------------- > > > -- ---------------------------------------------------- |epcc| - Ali Anjomshoaa EPCC, University of Edinburgh James Clerk Maxwell Building Mayfield Road E-mail: ali(a)epcc.ed.ac.uk Edinburgh EH9 3JZ Phone: + 44 (0) 131 651 3388 United Kingdom Fax: + 44 (0) 131 650 6555 -------------------------------------------------------------

3 3

phone conf today
by Michel Drescher 06 Apr '05

06 Apr '05

Guys, was anybody on the phone conference today? I tried, but I've been the only one attending. After some time I quit - Later the meeting "has ended" as the computer operator told me. What was going on? Bill, can you shed some light on this? Cheers, Michel

3 2

JSDL Meeting
by Bill Harris 05 Apr '05

05 Apr '05

You have been invited to attend a Microsoft Office Live Meeting. At the time of the meeting, click on the following link to attend. URL: http://www.placeware.com/cc/cadence/join?id=RWZ334&role=attend Subject: JSDL Meeting When: Tuesday, Apr 5, 2005 8:00 AM (MDT) Scheduled to Occur: Recurring Duration: 1 Hour 30 Minutes Audio Information: 1-800-933-3594 ext 3000 pin: 6444 Alternate Attend Instructions: Go to: http://www.placeware.com/cc/cadence/join Your Name: (enter your name) Meeting ID: RWZ334 William Harris Program Architect 150 West Civic Center Dr. Sandy, Utah 84070 Oryx follow me - 800 281 6462 Inside cadence 88 202 6462 Cell Phone 801 301 2744

1 0