April 2010 - pgi-wg - lists.ogf.org

OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion
by Etienne URBAH 19 Jul '10

19 Jul '10

Balazs, Morris, Luigi, Johannes and all, Concerning the capture and discussion of the requirements : I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner. In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality. This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore. Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet). For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements : - Apparently, it will increase the total number of requirements, - But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR. As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' : - JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results. I will split it into the 4 following requirements : - Description of the needed functionality : JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run - Description of the conceptual Client interface : JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run - First option for a precise description of the Client interface : JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness. - Second option for a precise description of the Client interface : JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one). In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them : - IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID - IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one. - LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID - LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one. Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ? - IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

3 5

Re: [Pgi-wg] [gin] PGI F2F Meeting in Amsterdam
by Morris Riedel 28 Apr '10

28 Apr '10

Hi, serious WIFI problems here - only minor connection for a few people here - we try to get a wired connection. Sorry we didn't expect such a problem... Take care, Morris >-- -----Ursprüngliche Nachricht----- >-- Von: gin-bounces(a)ogf.org [mailto:gin-bounces@ogf.org] Im Auftrag von Morris >-- Riedel >-- Gesendet: Dienstag, 27. April 2010 14:31 >-- An: pgi-wg(a)ogf.org >-- Cc: gin(a)ogf.org >-- Betreff: [gin] PGI F2F Meeting in Amsterdam >-- >-- Hi folks, >-- >-- I currently collect&prepare all sorts of material for tomorrow and send >-- several details during the next hours. >-- >-- Take care, >-- Morris >-- >-- ------------------------------------------------------------ >-- Morris Riedel >-- Distributed Systems and Grid Computing Division >-- >-- Jülich Supercomputing Centre (JSC) >-- Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel >--

1 0

Re: [Pgi-wg] Telcon & Screen-Sharing Information for Amsterdam
by Oxana Smirnova 28 Apr '10

28 Apr '10

Hi all, just called Morris - he says there are traffic problems in Amsterdam and the meeting will start ~10 minutes late (~9:40 local time) Cheers, Oxana 27.04.2010 14:52, Morris Riedel пишет: > Hi team, > > here some pieces of information about the telcon and screen-sharing tools > for Amsterdam: > > > (1) > TelCon System using HiDef (Attention: New Number! Not the old PGI one) > > Free of charge when calling from Skype: > +9900827045746500 > > International Numbers: > United States +1 (201) 793-9022 > Canada +1 (201) 793-9022 > Austria +43 (0) 82040115470 > Belgium +32 (0) 7 0357134 > France +33 (0) 826109071 > Germany +49 01805009527 > Ireland +353 (0) 818270968 > Italy +39 848390177 > Spain +34 (9) 02885791 > Switzerland +41 (0) 8 48560397 > United Kingdom +44 (0) 8454018081 > > Room number: 5746500 > > > The majority if participants in PGI felt in the past that this system is the > best system to use. > > No costs from Skype and nothing to install on the computer. > > > > (2) > Screen-Sharing > > http://webconf.vc.dfn.de/ogf-pgi-wg/ > > No login is required. Just enter as 'guest' with your real world name. > > > > (3) > We might use a video-conferencing facility that is available in Amsterdam > but this needs to be checked tomorrow morning. > Details on this will be send out when we decided that it makes sense to use > it after local investigation. > > > Your co-chair, > Morris > > ------------------------------------------------------------ > Morris Riedel > Distributed Systems and Grid Computing Division > > Jülich Supercomputing Centre (JSC) > Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel > > > > > > > _______________________________________________ > Pgi-wg mailing list > Pgi-wg(a)ogf.org > http://www.ogf.org/mailman/listinfo/pgi-wg

1 0

F2F Meeting in Amsterdam: Location information
by Morris Riedel 27 Apr '10

27 Apr '10

Hi team, as a final point of today just a short reminder about the location: Travel information for reaching the venue (NIKHEF building, Room H331) can be obtained here: https://www.nikhef.nl/en/about-nikhef/contact/how-to-reach-nikhef/ When you enter the building you will find a desk where you should be able to get a badge when you ask for the PGI F2F Meeting. Information about hotels in the area are listed here: http://www.egi.eu/about/events/hotels.html Registration Page: https://www.egi.eu/indico/conferenceDisplay.py?confId=25 I wish all a good trip and see you tomorrow! Your co-chair, Morris P.S. I will move out of office now being not able to read e-mails until night. Ciao! ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

Presence in Person in Amsterdam F2F Meeting
by Morris Riedel 27 Apr '10

27 Apr '10

Hi team, Thanks for registration and I have the registered people now as a list, but as I understand only a minor fraction is actually attending the F2F in person. The information of who is present in person is necessary for badges (and WIFI). I provide you with my current pieces of information and please send me an update in case of a change of your situation: # ACTIVELY PRESENT TOMORROW (NEED OF A BADGE): (1) KONYA, Balazs balazs.konya(a)hep.lu.se Lund University, NorduGrid (2) Mr. RIEDEL, MORRIS m.riedel(a)fz-juelich.de JUELICH SUPERCOMPUTING CENTRE (3) Mr. URBAH, Etienne urbah(a)lal.in2p3.fr LAL, Univ Paris-Sud, IN2P3/CNRS (4) Mr. WATZL, Johannes watzl(a)nm.ifi.lmu.de LMU Munich (5) Dr. ZANGRANDO, Luigi luigi.zangrando(a)pd.infn.it INFN - PADOVA # NOT PRESENT TOMORROW (NO BADGE) - But join per phone (1) Mr. BREWER, Stephen s.brewer(a)ecs.soton.ac.uk OMII-UK, Southampton University (2) Mr. MORGAN, Mark mmm2a(a)virginia.edu University of Virginia (3) Mr. PAISIOS, Emmanouil Emmanouil.Paisios(a)lrz.de LRZ (4) SMIRNOVA, Oxana oxana.smirnova(a)hep.lu.se NDGF (5) Dr. WALLOM, David david.wallom(a)oerc.ox.ac.uk University of Oxford/UK NGS/OGF (6) Andrew Grimshaw - University of Virginia Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

Agenda for Amsterdam
by Morris Riedel 27 Apr '10

27 Apr '10

Hi team, the expected major outcome of this meeting is an agreed and well formulated list of requirements that can be used to prepare and create specifications. The agenda will be fairly flexible since we will have the most of the time discussions around the requirements in order to understand each other. Nevertheless, I would like to provide an initial agenda with some constraints (i.e. STRICT TIME) implied by external folks that would like to join per phone for some specific slots: ### 2010-04-28 9:00 CET - 9:30 CET Logistics and check of video-conferencing facilities 9:30 CET - 10:30 CET (STRICT TIME) Process Status and Update 10:30 CET - 10:45 CET Short E-Mail and Coffee Break There is a large cafeteria on the second floor in the adjacent building of NIKHEF. 10:45 CET - 12:00 CET Requirement Discussions 12:00 CET - 13:00 CET Lunch break There is a restaurant across the street (http://www.cafe-restaurantpolder.nl/menu_en) 13:00 CET - 14:30 CET Requirement Discussions 14:30 CET - 15:00 CET E-Mails and Coffee break 15:00 CET - 16:30 CET Requirement Discussions 16:30 CET - 17:00 CET E-Mail, Snacks, and Coffee Break 17:00 CET - 18:30 CET Requirement Discussions 18:30 CET - 20:00 CET Getting to Hotels, etc. ~ 20:00 CET Dinner at some Restaurant in Amsterdam ### 2010-04-29 9:00 CET - 9:30 CET Status Update of yesterday and Progress Debrief 9:30 CET - 10:30 CET Requirements Discussions 10:30 CET - 10:45 CET E-Mails and Coffee break 10:45 CET - 12:00 CET Requirements Discussions 12:00 CET - 13:00 CET Lunch break 13:00 CET - 14:30 CET Requirement Discussions 14:30 CET - 15:00 CET E-Mails and Coffee break 15:00 CET - 16:30 CET Requirement Discussions 16:30 CET - 17:00 F2F Meeting Debrief and next steps 17:00 CET - ... Starting travel... Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

Telcon & Screen-Sharing Information for Amsterdam
by Morris Riedel 27 Apr '10

27 Apr '10

Hi team, here some pieces of information about the telcon and screen-sharing tools for Amsterdam: (1) TelCon System using HiDef (Attention: New Number! Not the old PGI one) Free of charge when calling from Skype: +9900827045746500 International Numbers: United States +1 (201) 793-9022 Canada +1 (201) 793-9022 Austria +43 (0) 82040115470 Belgium +32 (0) 7 0357134 France +33 (0) 826109071 Germany +49 01805009527 Ireland +353 (0) 818270968 Italy +39 848390177 Spain +34 (9) 02885791 Switzerland +41 (0) 8 48560397 United Kingdom +44 (0) 8454018081 Room number: 5746500 The majority if participants in PGI felt in the past that this system is the best system to use. No costs from Skype and nothing to install on the computer. (2) Screen-Sharing http://webconf.vc.dfn.de/ogf-pgi-wg/ No login is required. Just enter as 'guest' with your real world name. (3) We might use a video-conferencing facility that is available in Amsterdam but this needs to be checked tomorrow morning. Details on this will be send out when we decided that it makes sense to use it after local investigation. Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

Condensed Requirments List Version 3 in Grid Forge (ready for Amsterdam)
by Morris Riedel 27 Apr '10

27 Apr '10

Hi team, you can find the currently condensed requirements list (version 3) in Grid Forge (~ 6 MB): http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/doc man.root.drafts.condensed_requirement_lists/doc15968 An excel-export version of it can be found at: http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/doc man.root.drafts.condensed_requirement_lists/doc15969 Balazs, Etienne and myself spent the morning to finalize the list having the wiki as input. All requirements that have been re-proposed in the meanwhile will be discussed after the aforementioned list in Amsterdam in the next two days. @All: Please go through the wiki and the list and check whether we dropped requirements that you find substantial for PGI and re-propose them via e-mail. Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

PGI F2F Meeting in Amsterdam
by Morris Riedel 27 Apr '10

27 Apr '10

Hi folks, I currently collect&prepare all sorts of material for tomorrow and send several details during the next hours. Take care, Morris ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

OGF PGI : Requirements Wiki - Re-proposed and brand new requirements - Requirements needing clarification
by Etienne URBAH 26 Apr '10

26 Apr '10

Balazs, Morris, Luigi, Johannes and all, Inside the 'Requirements Wiki' containing Requirements tables at http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/Require… : As of now, re-proposed and brand new requirements requirements are : - IS2 Each Service MUST publish its properties (Name, Type, Endpoints, ...) and the properties of its Endpoints in conformance with GLUE recommendations - IS8 For not yet finished Activities, the Information functionality MUST accept requests querying the Activity Status, and MUST return this Status with different levels of verbosity (basic, detailed, more detailed) - IS9 For already finished Activities, the Information functionality SHOULD accept requests querying the Activity Status, but MAY return an error - IS13 For requests querying Activity Status, the Information functionality SHOULD support query languages - IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service - Ac0 In order to permit efficient Accounting, a grid infrastructure MUST provide an Accounting functionality (without real time requirements) which consolidates Accounting records at the grid level from various systems, securely keeps Accounting records even after Activities have finished, implements complex accounting queries, and supports the corresponding CPU load - Ac1 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Accounting functionality MUST be separated from the Execution Service - JM6.11 For all requests concerning Activities, the Execution Service functionality MUST accept, as input parameter, 1 Activity ID - JM11 The Execution Service MAY provide its functionalities to Clients through various protocols (RPC, Corba, ActiveMQ, ...), but MUST provide them to Clients through a Web Service As of now, requirements needing clarification are : - NF8 Clean specifications and interface definitions, don't keep obsolete and never used elements (by whom?) - NF15 Need to support high availability and reliability - NF16 Single global namespace for resources of all kind (storage devices, compute device), ... - NF17 Easier usage to get more users involved. How does it effects PGI?! A general rule like global namespaces (for example: path) - NF18 Potential re-use of PGI portTypes by other services (for example: information portType) Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 0