- jsdl-wg - lists.ogf.org

OGF PGI-JSDL : For resource requirements, proposal to use benchmarks as measurement units
by Etienne URBAH 22 Sep '11

22 Sep '11

Oxana, During OGF PGI Session 2 at OGF 33 on Wednesday 21 September 11:00-12:30, for JSDL resource requirements, we agreed to present a proposal for using benchmarks as measurement units. This relates to PGI requirement 115 traced at http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJD20 Practical Problem ----------------- For a given input data set, a user knows, by summing the CPU usage of individual cores, that his application needs 3 hours of CPU in total on a reference machine where the 'SPEC CFP 2006' floating point benchmark rates each CPU at 30. The user wants to send a Job running the same application on the same input data set. At Job submission time, the user is NOT able to know the 'SPEC CFP 2006' rating of the machine which will eventually run the Job. Still, the user wants to express in the Job Description document that the total CPU time limit for his Job must be large enough. Solution in English ------------------- The user does NOT know how well the applications of the 'SPEC CFP 2006' benchmark use multithreading. So, by security, this user rates each of the cores of his reference machine at maximum 30 'SPEC CFP 2006'. So, the total number of 'SPEC CFP 2006' operations needed by the Job is at most : 3 * 3600 * 30 = 324000 (s * 'SPEC CFP 2006') The user then wants to express following resource requirement in the Job Description document : Using the GLUE2 model, the 'MaxTotalCPUTime' attribute of the 'ComputingShare' entity must be at least 324000 divided by the 'Value' attribute of the 'Benchmark' whose 'Type' is 'cfp2006'. Solution in short human-readable pseudo-code -------------------------------------------- Job Definition: Job Description: Resource Requirements: GLUE2.ComputingShare.MaxTotalCPUTime >= 324000 / (GLUE2.Benchmark.Value where GLUE2.Benchmark.Type = 'cfp2006') Same solution in human readable pseudo-code with key=value statements --------------------------------------------------------------------- Job Definition: Job Description: Resource Requirements: Resource With Unit: Resource Reference = GLUE2.ComputingShare.MaxTotalCPUTime Lower Bounded Range = 324000 Resource Unit Reference = GLUE2.Benchmark.Type Resource Unit Name = 'cfp2006' Resource Unit Value = GLUE2.Benchmark.Value Simple solution in JSDL using a new 'jsdl+dx:BenchmarkType' element ------------------------------------------------------------------- This solution, which is simple for JSDL, requires that in the Execution Service, a reference to 'jsdl+dx:BenchmarkType' is hard-coded as a lookup from 'GLUE2.Benchmark.Type' to 'GLUE2.Benchmark.Value'. I also use the fact that : jsdl:TotalCPUTime == GLUE2.ComputingShare.MaxTotalCPUTime <jsdl:JobDefinition> <jsdl:JobDescription> ... <jsdl:Resources> <jsdl:TotalCPUTime> <jsdl:LowerBoundedRange> 324000.0 </jsdl:LowerBoundedRange> <jsdl+dx:BenchmarkType> cfp2006 </jsdl+dx:BenchmarkType> </jsdl:TotalCPUTime> </jsdl:Resources> ... </jsdl:JobDescription> </jsdl:JobDefinition> Complex solution in JSDL using a new 'jsdl+dx:ResourceWithUnit' element ----------------------------------------------------------------------- This solution, which is more complex for JSDL, provides the necessary information permitting the Execution Service to perform the lookup dynamically and prevents hard-coding. <jsdl:JobDefinition> <jsdl:JobDescription> ... <jsdl:Resources> <jsdl+dx:ResourceWithUnit> <jsdl+dx:ResourceReference> GLUE2.ComputingShare.MaxTotalCPUTime </jsdl+dx:ResourceReference> <jsdl:LowerBoundedRange> 324000.0 </jsdl:LowerBoundedRange> <jsdl+dx:ResourceUnitReference> GLUE2.Benchmark.Type </jsdl+dx:ResourceUnitReference> <jsdl+dx:ResourceUnitName> cfp2006 </jsdl+dx:ResourceUnitName> <jsdl+dx:ResourceUnitValue> GLUE2.Benchmark.Value </jsdl+dx:ResourceUnitValue> </jsdl+dx:ResourceWithUnit> </jsdl:Resources> ... </jsdl:JobDescription> </jsdl:JobDefinition> Thank you in advance for studying these proposals, and for providing comments and suggestions for improvement. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 0

GLUE 2.0 attributes for usage inside JSDL
by Etienne URBAH 21 Sep '11

21 Sep '11

Michael, Concerning the GLUE 2.0 attributes for usage inside JSDL, I suggest to look at following pages : http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJD3 http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do… Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 1

Re: [jsdl-wg] [gin] REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in
by Andre Merzky 20 Sep '11

20 Sep '11

On Tue, Sep 20, 2011 at 9:40 AM, Morris Riedel <m.riedel(a)fz-juelich.de> wrote: > > (a) > Focus: baby steps together - concept by concept (e.g. manual data-staging, pre-post processing, scalable-time, etc.), we should not try to standardize all requirements at once, there could be BES 1.1, BES1.2, BES1.3, etc. +1 > (c) > Next steps: experience tells us that we will not manage to achieve very much in the time available at the OGF event, also some progress will be made - but how we organize the next months: milestones, weekly phone calls, chairs?!, specification work with deadlines, etc. how about a F2F before next OGF? Cheers, Andre. -- Nothing is ever easy...

3 2

REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in
by Morris Riedel 20 Sep '11

20 Sep '11

All, after re-scheduling, we have the first of our sessions today [1]: Tuesday, September 20 2:00 pm - 3:30 pm PGI-OGSA-BES-JSDL Joint Session (90 mins) Andrew Grimshaw, Morris Riedel Location: Rhone 4 A high participation is appreciated especially since several of us have several parallel commitments in other sessions. Important will be three things (from my perspective): (a) Focus: baby steps together - concept by concept (e.g. manual data-staging, pre-post processing, scalable-time, etc.), we should not try to standardize all requirements at once, there could be BES 1.1, BES1.2, BES1.3, etc. (b) Working sessions: Skip introductions and general architecture discussions; focus on specification work concept by concept (XSD level?!) (c) Next steps: experience tells us that we will not manage to achieve very much in the time available at the OGF event, also some progress will be made - but how we organize the next months: milestones, weekly phone calls, chairs?!, specification work with deadlines, etc. Of course, I'm open to suggestions. Take care, Morris [1] http://www.ogf.org/gf/event_schedule/index.php?id=2392 ------------------------------------------------------------ Morris Riedel Deputy Division Leader Federated Systems and Data (FSD) Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/SharedDocs/Personen/IAS/JSC/EN/staff/riedel_m.html

1 0

OGF 33 and next telephone conference
by philipp.wieder＠tu-dortmund.de 16 Sep '11

16 Sep '11

Dear All, vacation time has passed and the next OGF is approaching. I would like to direct you attention to the following JSDL sessions: - JSDL general session on Monday, September 19, 09:00 to 10:30. - A combined PGI/BES/JSDL session on Wednesday, September 21 from 11:00 on the whole day. You find the complete schedule here [1]. I would like to prepare the sessions with a telephone conference next week. I suggest we use the day/time we used before which is Wednesday, September 14, 14:00 UTC. It translates to: Eastern Standard Time: 10:00 Japan Standard Time: 23:00 Central European Time: 16:00 Pacific Standard Time: 07:00 Since our telephone conferencing system only support German numbers I would prefer if someone else could host the telco. Who could do that? Best regards, Philipp. [1] http://ogf.org/gf/event_schedule/index.php?event_id=20

4 7

Fwd: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST)
by Etienne URBAH 15 Sep '11

15 Sep '11

Philipp and all, Concerning the preparation of the PGI / BES / JSDL sessions at OGF 33 in Lyon next week : I booked an EVO teleconference named 'OGF preparation of PGI / BES / JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and Oracle Java 1.7.0, so that you may have to use EVO beta at http://evo.caltech.edu/evoBeta/ which I tested successfully. Access to the EVO teleconference will be provided at http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s with password !!jsdlftw Thanks in advance to Philipp for providing the agenda. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr ----------------------------------------------------- -------- Original Message -------- Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) Date: Mon, 12 Sep 11 17:58:20 +0200 From: evosupport(a)vrvs.org To: urbah(a)lal.in2p3.fr Hello Etienne URBAH, You have BOOKED a meeting in EVO (http://evo.caltech.edu) Title: OGF preparation of PGI / BES / JSDL Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in Lyon next week Community: Universe Password: !!jsdlftw Meeting Access Information: - Meeting URL http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s - Password: !!jsdlftw - Phone Bridge ID: 399 7332 Password: 0162 Central European Summer Time (+0200) Start 2011-09-14 16:00 End 2011-09-14 18:00 Japan Standard Time (+0900) Start 2011-09-14 23:00 End 2011-09-15 01:00 Eastern Daylight Time (-0400) Start 2011-09-14 10:00 End 2011-09-14 12:00 Pacific Daylight Time (-0700) Start 2011-09-14 07:00 End 2011-09-14 09:00 EVO Phone Bridge Telephone Numbers: --------------- - USA (Caltech, Pasadena, CA) +1 626 395 2112 - Switzerland (CERN, Geneva) +41 22 76 71400 - Slovakia (UPJS, Kosice) +421 55 234 2420 - Italy (INFN, several cities) http://server10.infn.it/video/index.php?page=telephone_numbers Enter '4000' to access the EVO bridge - Germany (DESY, Hamburg) +49 40 8998 1340 - USA (BNL, Upton, NY) +1 631 344 6100 - United Kingdom (University of Manchester) +44 161 306 6802 - Australia (ARCS) +61 Adelaide 08 8463 1011 Brisbane 07 3139 0705 Canberra 02 6112 8742 Hobart 03 623 70281 Melbourne 03 8685 8362 Perth 08 6461 6718 Sydney 02 8212 4591 - Netherlands (Nikhef, Amsterdam) +31 20 7165293 Dial '2' at the prompt - Canada (TRIUMF, Vancouver) +1 604 222 7700 - Czech Republic (CESNET, Prague) +420 95 007 2386 - USA (MIT, Cambridge, MA) +1 617 715 4691 - France (RAP, Paris) +33 144 27 81 50 - Skype (tm) (World-wide) evo.phone See: http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html

4 6

OGF JSDL - Improved version of GLUE2 Attributes For Usage Inside JSDL
by Etienne URBAH 16 Jul '11

16 Jul '11

To JSDL stakeholders, I have saved an improved version of the 'GLUE2 Attributes For Usage Inside JSDL' document at http://forge.gridforum.org/sf/go/doc16186 Thank you in advance for reading and criticizing it. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 0

Genesis II suggested JSDL updates
by Andrew Grimshaw 16 Jul '11

16 Jul '11

1 0

JSDL session Saturday, 3:30 pm
by philipp.wieder＠tu-dortmund.de 15 Jul '11

15 Jul '11

Dear All, I would like to invite everybody interested in the future of JSDL to join us tomorrow at 3:30 pm in Snowbird. I think that it is high time that we put on the table what we have in terms of requirements and use cases and match it with the current status of the group. Furthermore, as you all know, there has been change in leadership during the previous months, something we have to react on. I had a discussion on a number of issues with Andreas already which I think provide a good starting point for tomorrow. We will have an open session tomorrow to discuss the various issues and I expect us to find a solution on how to continue with JSDL. I hope to see you tomorrow. Best regards, Philipp.

1 0

Combined JSDL Use Case Uploaded
by Mark M. Morgan 03 May '11

03 May '11

Folks, I have uploaded the combined JSDL Use Case document into grid forge.

1 0