[jsdl-wg] OGF PGI-JSDL : For resource requirements, proposal to use benchmarks as measurement units

22 Sep 2011

      Oxana,

During OGF PGI Session 2 at OGF 33 on Wednesday 21 September 
11:00-12:30, for JSDL resource requirements, we agreed to present a 
proposal for using benchmarks as measurement units.

This relates to PGI requirement 115 traced at 
http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJD20

Practical Problem
-----------------
For a given input data set, a user knows, by summing the CPU usage of 
individual cores, that his application needs 3 hours of CPU in total on 
a reference machine where the 'SPEC CFP 2006' floating point benchmark 
rates each CPU at 30.

The user wants to send a Job running the same application on the same 
input data set.
At Job submission time, the user is NOT able to know the 'SPEC CFP 2006' 
rating of the machine which will eventually run the Job.
Still, the user wants to express in the Job Description document that 
the total CPU time limit for his Job must be large enough.

Solution in English
-------------------
The user does NOT know how well the applications of the 'SPEC CFP 2006' 
benchmark use multithreading.  So, by security, this user rates each of 
the cores of his reference machine at maximum 30 'SPEC CFP 2006'.

So, the total number of 'SPEC CFP 2006' operations needed by the Job is 
at most :
3 * 3600 * 30  =  324000 (s * 'SPEC CFP 2006')

The user then wants to express following resource requirement in the Job 
Description document :
Using the GLUE2 model, the 'MaxTotalCPUTime' attribute of the 
'ComputingShare' entity must be at least 324000 divided by the 'Value' 
attribute of the 'Benchmark' whose 'Type' is 'cfp2006'.

Solution in short human-readable pseudo-code
--------------------------------------------
Job Definition:
   Job Description:
     Resource Requirements:
       GLUE2.ComputingShare.MaxTotalCPUTime >= 324000 / 
(GLUE2.Benchmark.Value where GLUE2.Benchmark.Type = 'cfp2006')

Same solution in human readable pseudo-code with key=value statements
---------------------------------------------------------------------
Job Definition:
   Job Description:
     Resource Requirements:
       Resource With Unit:
         Resource Reference      = GLUE2.ComputingShare.MaxTotalCPUTime
         Lower Bounded Range     = 324000
         Resource Unit Reference = GLUE2.Benchmark.Type
         Resource Unit Name      = 'cfp2006'
         Resource Unit Value     = GLUE2.Benchmark.Value

Simple solution in JSDL using a new 'jsdl+dx:BenchmarkType' element
-------------------------------------------------------------------
This solution, which is simple for JSDL, requires that in the Execution 
Service, a reference to 'jsdl+dx:BenchmarkType' is hard-coded as a 
lookup from 'GLUE2.Benchmark.Type' to 'GLUE2.Benchmark.Value'.

I also use the fact that :
jsdl:TotalCPUTime == GLUE2.ComputingShare.MaxTotalCPUTime

<jsdl:JobDefinition>
   <jsdl:JobDescription>
     ...
      <jsdl:Resources>
        <jsdl:TotalCPUTime>
          <jsdl:LowerBoundedRange> 324000.0 </jsdl:LowerBoundedRange>
          <jsdl+dx:BenchmarkType> cfp2006 </jsdl+dx:BenchmarkType>
        </jsdl:TotalCPUTime>
      </jsdl:Resources>
     ...
   </jsdl:JobDescription>
</jsdl:JobDefinition>

Complex solution in JSDL using a new 'jsdl+dx:ResourceWithUnit' element
-----------------------------------------------------------------------
This solution, which is more complex for JSDL, provides the necessary 
information permitting the Execution Service to perform the lookup 
dynamically and prevents hard-coding.

<jsdl:JobDefinition>
   <jsdl:JobDescription>
     ...
      <jsdl:Resources>
        <jsdl+dx:ResourceWithUnit>
          <jsdl+dx:ResourceReference>
            GLUE2.ComputingShare.MaxTotalCPUTime
          </jsdl+dx:ResourceReference>
          <jsdl:LowerBoundedRange>
            324000.0
          </jsdl:LowerBoundedRange>
          <jsdl+dx:ResourceUnitReference>
            GLUE2.Benchmark.Type
          </jsdl+dx:ResourceUnitReference>
          <jsdl+dx:ResourceUnitName>
            cfp2006
          </jsdl+dx:ResourceUnitName>
          <jsdl+dx:ResourceUnitValue>
            GLUE2.Benchmark.Value
          </jsdl+dx:ResourceUnitValue>
        </jsdl+dx:ResourceWithUnit>
      </jsdl:Resources>
     ...
   </jsdl:JobDescription>
</jsdl:JobDefinition>

Thank you in advance for studying these proposals, and for providing 
comments and suggestions for improvement.

Best regards.

-----------------------------------------------------
Etienne URBAH         LAL, Univ Paris-Sud, IN2P3/CNRS
                       Bat 200   91898 ORSAY    France
Tel: +33 1 64 46 84 87      Skype: etienne.urbah
Mob: +33 6 22 30 53 27      mailto:urbah@lal.in2p3.fr
-----------------------------------------------------