OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion
Balazs, Morris, Luigi, Johannes and all, Concerning the capture and discussion of the requirements : I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner. In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality. This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore. Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet). For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements : - Apparently, it will increase the total number of requirements, - But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR. As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' : - JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results. I will split it into the 4 following requirements : - Description of the needed functionality : JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run - Description of the conceptual Client interface : JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run - First option for a precise description of the Client interface : JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness. - Second option for a precise description of the Client interface : JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one). In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them : - IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID - IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one. - LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID - LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one. Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ? - IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
Hi Etienne, I have a feeling that there is sometimes a tendency to overcomplicate simple things. You are right in saying that many requirements are formulated in a confusing manner. In fact, it is very difficult to formulate a requirement or a definition. It is like describing a face. You often feel like "I recognize it when I see it, but can't describe". With unclear "requirements" one has to understand what *is* the requirement. And to do this, one has to understand the use case. Take your example of "scalable time". The use case is trivial: it makes *no sense whatsoever* to specify either CPU time or wallclock time in a heterogeneous system. Because there are different hardwares around. You *have* to characterize hardware on which this performance was achieved. Period. I hate the word "scalable" here because it really steels all attention, hinting at implementation details. Who said that execution time scales lineraly with benchmark values anyway? This is the real requirement: *** It must be possible to specify hardware characteristics (e.g. in terms of a benchmark value) corresponding to the indicated performance numbers *** And mind it, performance numbers in general are not limited to time. It might be also memory consumption from another foggy requirement of ours: it obviously will be different on 32- and 64-bit architectures. How will the tools use this information, is a different question. Cheers, Oxana 2010-04-30 17:24, Etienne URBAH пишет:
Balazs, Morris, Luigi, Johannes and all,
Concerning the capture and discussion of the requirements :
I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner.
In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality.
This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore.
Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet).
For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements :
- Apparently, it will increase the total number of requirements,
- But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR.
As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' :
- JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results.
I will split it into the 4 following requirements :
- Description of the needed functionality :
JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run
- Description of the conceptual Client interface :
JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run
- First option for a precise description of the Client interface :
JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness.
- Second option for a precise description of the Client interface :
JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft
I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one).
In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them :
- IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID
- IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one.
- LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID
- LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one.
Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ?
- IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service
Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
_______________________________________________ Pgi-wg mailing list Pgi-wg@ogf.org http://www.ogf.org/mailman/listinfo/pgi-wg
Oxana, Well said. I would add that I fear we may be trying too much to solve all problems the first time around - to "boil the ocean". To completely solve the whole problem is a daunting task indeed as there are so many issues. I personally believe we will make more progress if we solve the minimum problem first, e.g., securely run a simple job from infrastructure/sw-stack A on infrastructure/sw-stack B. "Infrastructure/sw-stack A" means a set of resources (e.g., true parallel-Jugene, clusters, sets of desktops) running a middleware stack (e.g., Unicore 6 or Arc) configured a particular way. In the European context this might mean an NGI such as D-Grid with Unicore 6 running a job on NorduGrid running Arc. (Please forgive me if I have the particulars of the NGIs wrong.) "Simple job" means a job that is typical, not special. This is not to say that its resource requirements are simple, it may have very particular requirements (cores per socket, interconnect, memory), rather I mean that the job processing required is simple: run w/o staging, simple staging, perhaps client interaction with the session directory pre, post, and during execution. Try to avoid complex job state models that will be hard to agree on, and difficult to implement in some environments. "Securely" means sufficient authentication information required at B is provided to B in a form it will accept from a policy perspective. Further, that we try as much as possible to avoid a delegation definition that extends inwards beyond the outer boundary of a particular infrastructure/sw-stack. (The last sentence is a bit awkward, I personally think that we will need to have two models of authentication and delegation - a legacy transport layer mechanism, and a message layer mechanism based on SAML, and that inside of a software stack we cannot expect sw-stacks to change their internal delegation mechanism.) I believe authentication/delegation is the most critical item: if we cannot get the authentication/delegation issues solved, the rest is moot with respect to a PRODUCTION environment. We may be able to do demo's and stunts while punting on authentication/delegation, but we will not integrate production systems.) A -----Original Message----- From: pgi-wg-bounces@ogf.org [mailto:pgi-wg-bounces@ogf.org] On Behalf Of Oxana Smirnova Sent: Friday, April 30, 2010 4:22 PM To: Etienne URBAH Cc: pgi-wg@ogf.org; David SNELLING; lodygens@lal.in2p3.fr Subject: Re: [Pgi-wg] OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion Hi Etienne, I have a feeling that there is sometimes a tendency to overcomplicate simple things. You are right in saying that many requirements are formulated in a confusing manner. In fact, it is very difficult to formulate a requirement or a definition. It is like describing a face. You often feel like "I recognize it when I see it, but can't describe". With unclear "requirements" one has to understand what *is* the requirement. And to do this, one has to understand the use case. Take your example of "scalable time". The use case is trivial: it makes *no sense whatsoever* to specify either CPU time or wallclock time in a heterogeneous system. Because there are different hardwares around. You *have* to characterize hardware on which this performance was achieved. Period. I hate the word "scalable" here because it really steels all attention, hinting at implementation details. Who said that execution time scales lineraly with benchmark values anyway? This is the real requirement: *** It must be possible to specify hardware characteristics (e.g. in terms of a benchmark value) corresponding to the indicated performance numbers *** And mind it, performance numbers in general are not limited to time. It might be also memory consumption from another foggy requirement of ours: it obviously will be different on 32- and 64-bit architectures. How will the tools use this information, is a different question. Cheers, Oxana 2010-04-30 17:24, Etienne URBAH пишет:
Balazs, Morris, Luigi, Johannes and all,
Concerning the capture and discussion of the requirements :
I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner.
In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality.
This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore.
Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet).
For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements :
- Apparently, it will increase the total number of requirements,
- But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR.
As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' :
- JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results.
I will split it into the 4 following requirements :
- Description of the needed functionality :
JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run
- Description of the conceptual Client interface :
JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run
- First option for a precise description of the Client interface :
JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness.
- Second option for a precise description of the Client interface :
JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft
I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one).
In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them :
- IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID
- IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one.
- LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID
- LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one.
Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ?
- IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service
Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
_______________________________________________ Pgi-wg mailing list Pgi-wg@ogf.org http://www.ogf.org/mailman/listinfo/pgi-wg
Oxana and all, Concerning OGF PGI Requirement JD20 (115) 'Client requirements independent of the speed of the Computing Resource' : WLCG is not only discussing this issue, but is really working on it : Using the current GLUE 1.3, WLCG provides a practical solution, and is currently improving its robustness permitting full operational deployment. This solution, which matches my JD20.3 proposal below, is described at http://indico.cern.ch/getFile.py/access?contribId=6&sessionId=0&resId=1&materialId=slides&confId=72057 I will write down this information inside the OGF PGI detailed Wiki page for JD20. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr ----------------------------------------------------- On Fri, 30/04/2010 22:22, Oxana Smirnova wrote:
Hi Etienne,
I have a feeling that there is sometimes a tendency to overcomplicate simple things.
You are right in saying that many requirements are formulated in a confusing manner.
In fact, it is very difficult to formulate a requirement or a definition. It is like describing a face. You often feel like "I recognize it when I see it, but can't describe".
With unclear "requirements" one has to understand what *is* the requirement. And to do this, one has to understand the use case.
Take your example of "scalable time". The use case is trivial: it makes *no sense whatsoever* to specify either CPU time or wallclock time in a heterogeneous system. Because there are different hardwares around. You *have* to characterize hardware on which this performance was achieved. Period. I hate the word "scalable" here because it really steels all attention, hinting at implementation details. Who said that execution time scales lineraly with benchmark values anyway? This is the real requirement:
*** It must be possible to specify hardware characteristics (e.g. in terms of a benchmark value) corresponding to the indicated performance numbers ***
And mind it, performance numbers in general are not limited to time. It might be also memory consumption from another foggy requirement of ours: it obviously will be different on 32- and 64-bit architectures.
How will the tools use this information, is a different question.
Cheers, Oxana
2010-04-30 17:24, Etienne URBAH пишет:
Balazs, Morris, Luigi, Johannes and all,
Concerning the capture and discussion of the requirements :
I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner.
In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality.
This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore.
Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet).
For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements :
- Apparently, it will increase the total number of requirements,
- But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR.
As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' :
- JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results.
I will split it into the 4 following requirements :
- Description of the needed functionality :
JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run
- Description of the conceptual Client interface :
JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run
- First option for a precise description of the Client interface :
JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness.
- Second option for a precise description of the Client interface :
JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft
I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one).
In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them :
- IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID
- IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one.
- LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID
- LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one.
Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ?
- IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service
Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
_______________________________________________ Pgi-wg mailing list Pgi-wg@ogf.org http://www.ogf.org/mailman/listinfo/pgi-wg
Hello Etienne, this is of course a related activity, but not a generic solution to the problem. WLCG used the SPECINT2000 (SI00) benchmark to advertise resource performance, until this benchmark became obsolete. Now a HEPiX working group developed a dedicated benchmark for High Energy Physics applications, called HEPSPEC2006 (HS06) (SPEC don't like them re-using the name, by the way). Quite obviously, HS06 is not necessarily suitable for non-HEP applications; it does not characterise a computing site: it characterises performance of the site for a particular application. After all, this is what benchmarks are all about. Now, Steve Traylen proposes ways of twisting the old and largely irrelevant Glue1.3 schema to accommodate various benchmarks. This is not what we need in PGI. Firstly, we have Glue2 which has all the problems addressed in the "solution" you quote already solved. Secondly, we'd like to go one step forward and ensure that we can characterize computing resources in any benchmark we find useful, or maybe even a set of them - for different applications to use. Such that Steve Traylen won't have to write 10 slides on a simple issue that can be solved by adding one more line to the configuration file. Cheers, Oxana 2010-06-10 13:32, Etienne URBAH пишет:
Oxana and all,
Concerning OGF PGI Requirement JD20 (115) 'Client requirements independent of the speed of the Computing Resource' :
WLCG is not only discussing this issue, but is really working on it :
Using the current GLUE 1.3, WLCG provides a practical solution, and is currently improving its robustness permitting full operational deployment.
This solution, which matches my JD20.3 proposal below, is described at http://indico.cern.ch/getFile.py/access?contribId=6&sessionId=0&resId=1&materialId=slides&confId=72057
I will write down this information inside the OGF PGI detailed Wiki page for JD20.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
On Fri, 30/04/2010 22:22, Oxana Smirnova wrote:
Hi Etienne,
I have a feeling that there is sometimes a tendency to overcomplicate simple things.
You are right in saying that many requirements are formulated in a confusing manner.
In fact, it is very difficult to formulate a requirement or a definition. It is like describing a face. You often feel like "I recognize it when I see it, but can't describe".
With unclear "requirements" one has to understand what *is* the requirement. And to do this, one has to understand the use case.
Take your example of "scalable time". The use case is trivial: it makes *no sense whatsoever* to specify either CPU time or wallclock time in a heterogeneous system. Because there are different hardwares around. You *have* to characterize hardware on which this performance was achieved. Period. I hate the word "scalable" here because it really steels all attention, hinting at implementation details. Who said that execution time scales lineraly with benchmark values anyway? This is the real requirement:
*** It must be possible to specify hardware characteristics (e.g. in terms of a benchmark value) corresponding to the indicated performance numbers ***
And mind it, performance numbers in general are not limited to time. It might be also memory consumption from another foggy requirement of ours: it obviously will be different on 32- and 64-bit architectures.
How will the tools use this information, is a different question.
Cheers, Oxana
2010-04-30 17:24, Etienne URBAH пишет:
Balazs, Morris, Luigi, Johannes and all,
Concerning the capture and discussion of the requirements :
I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner.
In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality.
This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore.
Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet).
For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements :
- Apparently, it will increase the total number of requirements,
- But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR.
As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' :
- JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results.
I will split it into the 4 following requirements :
- Description of the needed functionality :
JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run
- Description of the conceptual Client interface :
JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run
- First option for a precise description of the Client interface :
JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness.
- Second option for a precise description of the Client interface :
JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft
I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one).
In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them :
- IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID
- IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one.
- LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID
- LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one.
Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ?
- IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service
Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
_______________________________________________ Pgi-wg mailing list Pgi-wg@ogf.org http://www.ogf.org/mailman/listinfo/pgi-wg
Oxana and all, Concerning OGF PGI Requirement JD20 (115) 'Client requirements independent of the speed of the Computing Resource' : I thank Oxana for her response, and I apologize for not having answered sooner. Oxana is mainly right, the solution for which I gave a link : - does NOT covers many benchmarks, but only SPECINT2000 (and also HEPSpec06, see page 5), - does NOT new GLUE 2.0, but only old GLUE 1.3, - does NOT use JSDL, but only JDL with ClassAds. But this is still an example of practical solution of the practical problem. For me, the important point is that this practical solution did NOT invent a new concept of 'scalable time', but is based on the concept of 'Quantity of Work = Time * Power', which is well known in physics for being meaningful. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr ----------------------------------------------------- On Mon, 14/06/2010 17:19, Oxana Smirnova wrote:
Hello Etienne,
this is of course a related activity, but not a generic solution to the problem. WLCG used the SPECINT2000 (SI00) benchmark to advertise resource performance, until this benchmark became obsolete. Now a HEPiX working group developed a dedicated benchmark for High Energy Physics applications, called HEPSPEC2006 (HS06) (SPEC don't like them re-using the name, by the way). Quite obviously, HS06 is not necessarily suitable for non-HEP applications; it does not characterise a computing site: it characterises performance of the site for a particular application. After all, this is what benchmarks are all about. Now, Steve Traylen proposes ways of twisting the old and largely irrelevant Glue1.3 schema to accommodate various benchmarks.
This is not what we need in PGI. Firstly, we have Glue2 which has all the problems addressed in the "solution" you quote already solved. Secondly, we'd like to go one step forward and ensure that we can characterize computing resources in any benchmark we find useful, or maybe even a set of them - for different applications to use. Such that Steve Traylen won't have to write 10 slides on a simple issue that can be solved by adding one more line to the configuration file.
Cheers, Oxana
2010-06-10 13:32, Etienne URBAH пишет:
Oxana and all,
Concerning OGF PGI Requirement JD20 (115) 'Client requirements independent of the speed of the Computing Resource' :
WLCG is not only discussing this issue, but is really working on it :
Using the current GLUE 1.3, WLCG provides a practical solution, and is currently improving its robustness permitting full operational deployment.
This solution, which matches my JD20.3 proposal below, is described at http://indico.cern.ch/getFile.py/access?contribId=6&sessionId=0&resId=1&materialId=slides&confId=72057
I will write down this information inside the OGF PGI detailed Wiki page for JD20.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
On Fri, 30/04/2010 22:22, Oxana Smirnova wrote:
Hi Etienne,
I have a feeling that there is sometimes a tendency to overcomplicate simple things.
You are right in saying that many requirements are formulated in a confusing manner.
In fact, it is very difficult to formulate a requirement or a definition. It is like describing a face. You often feel like "I recognize it when I see it, but can't describe".
With unclear "requirements" one has to understand what *is* the requirement. And to do this, one has to understand the use case.
Take your example of "scalable time". The use case is trivial: it makes *no sense whatsoever* to specify either CPU time or wallclock time in a heterogeneous system. Because there are different hardwares around. You *have* to characterize hardware on which this performance was achieved. Period. I hate the word "scalable" here because it really steels all attention, hinting at implementation details. Who said that execution time scales lineraly with benchmark values anyway? This is the real requirement:
*** It must be possible to specify hardware characteristics (e.g. in terms of a benchmark value) corresponding to the indicated performance numbers ***
And mind it, performance numbers in general are not limited to time. It might be also memory consumption from another foggy requirement of ours: it obviously will be different on 32- and 64-bit architectures.
How will the tools use this information, is a different question.
Cheers, Oxana
2010-04-30 17:24, Etienne URBAH пишет:
Balazs, Morris, Luigi, Johannes and all,
Concerning the capture and discussion of the requirements :
I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner.
In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality.
This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore.
Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet).
For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements :
- Apparently, it will increase the total number of requirements,
- But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR.
As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' :
- JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results.
I will split it into the 4 following requirements :
- Description of the needed functionality :
JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run
- Description of the conceptual Client interface :
JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run
- First option for a precise description of the Client interface :
JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness.
- Second option for a precise description of the Client interface :
JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft
I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one).
In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them :
- IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID
- IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one.
- LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID
- LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one.
Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ?
- IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service
Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity.
Best regards.
----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------
participants (3)
-
Andrew Grimshaw -
Etienne URBAH -
Oxana Smirnova