- pgi-wg - lists.ogf.org

Re: [Pgi-wg] Gridiron, or standardization gone backwards
by Steven Newhouse 20 Jul '10

20 Jul '10

> If anybody has a use case where these different systems must be brought > together, I'd like to have *this* use case, described in details, what exactly is > needed, what exactly has to be unified and standardised. EGI. Different NGIs wish to deploy different software and for their user communities to be able to access resources in different NGIs without worrying the software stack that is being run on the remote resources. This applies as much within Europe as between Europe and the USA and elsewhere in the world. Standards provides the solution. Steven -- Steven Newhouse EGI-InSPIRE Project Director EGI.eu Director Tel: +31 20 592 2206 Mob: +31 6 3037 2845

2 1

Re: [Pgi-wg] Gridiron, or standardization gone backwards
by Oxana Smirnova 20 Jul '10

20 Jul '10

Hi Morris, all, I definitely don't give up - otherwise I wouldn't have been writing these mails ;-) I try to understand what made the initially rapid process stale. And I do believe that substantial differences between different technologies do create obstacles. If the technologies would have been sufficiently similar, like soccer played in Scotland and England, it would have been straightforward to create common standards. Let's sit back and have a look at the crowd flocked to PGI, and ask ourselves, what exactly do we all want to achieve? Morris, I know you like low-hanging fruits :-) Next time you're in Lund, I'll show you an odd pear tree in the neighborhood: it was not pruned for hundred years or so, and is as tall as a pine now. Imagine where are low-hanging pears ;-) PGI keeps growing, fruits are getting out of reach. A tall pear tree certainly will create a large impact when felled, but is pretty much useless fruit-wise. It is easy to understand why so many technology providers are in PGI: the original PGI proclaimed that we'll create *the* reference standards, and everybody got worried that their favorite requirement won't be taken into account. No wonder we collected 237 or so *contradictory* requirements. The high-level use case we tried to address 2 years ago was: "A computing infrastructure is created using services provided by gLite, ARC and UNICORE" Then, going into details of what is a "computing infrastructure", we came with requirements that: 1. job description must be common (common JSDL flavour) 2. resource discovery and monitoring must be common (common GLUE2 schema) 3. execution interfaces must be common (common BES) Of course, authorisation and data management interfaces also must be common, but they were already largely unified thanks to GSI and SRM, so the task was easy and the goal was clearly in sight. Too bad it was never achieved - even though the impact wold have been limited, still quite many users would have benefited. Now, what is the new PGI use case? "Create a worldwide computing infrastructure using all possible technologies"? I hope not. It will be a bit like creating a common NFL+FIFA championship ;-) Cheers, Oxana

1 0

Gridiron, or standardization gone backwards
by Oxana Smirnova 20 Jul '10

20 Jul '10

Hi all, I intended to comment on use cases, but feel like commenting on the very fact of their appearance. When the PGI founders first met in September 2008 (yes, 2008), they produced a very advanced draft, almost a ready-made specification. The only reason they could not call it a specification was that they were nobody - that is, OGF did not know of them. They didn't have to waste time on formalizing use cases and requirements, because all these were in their heads. In football terms, they played by similar rules, and didn't have to explain to each other the gory details. They were driven by the desire to produce common rules of the game for themselves, such that they can play in the same league. Then OGF kindly adopted the team, but at the cost of putting forward formal requirements. No forward movement happened since. First step backwards was to trim the specs to a "strawman". Second step backwards was to drop the strawman and collect requirements. The third step backwards was to go back to use cases. I dread to think what will be the next PGI decision? To create itself? In September 2008 we thought that by December same year we'll have the core specs. Two years later we are discussing what is the best template for use cases and which teleconferencing tool to use. This is, well, unbelievable. And Ithink I know the reason. We try to compare incomparable things. Imagine an international football federation that brings together association football, American football, rugby, Australian rules football and all such things. And imagine this federation introducing common rules. What would this rule be? Right, "the game is played on a large field by two teams". Is there any practical use of this rule? No, every league will have to keep own "extensions". Despite often looking brain-damaged, footballers are clever enough not to invent common football rules. They realize that the term is overloaded, and they manage to disambiguate it. Grids brought together by PGI are as different as gridiron is different from soccer. Let's face it. They still can be played on the same pitch - meaning, they can use same hardware - but attempts to device common rules/specifications so far lead nowhere. It is as if gridiron guys would be keeping insisting that soccer has to be played with a ball that doesn't even look like a ball, and rugby folks would be agreeing, and soccer guys would be scratching their heads and meekly saying that their use case is actually to kick it with feet, not carry in armpits. The analogy is probably not exactly accurate, but I am quite frustrated, as I can see no progress whatsoever. Of the original PGI "creators" only three are still attending the meetings - Johannes, Aleksandr and myself. Any suggestions are welcomed. Cheers, Oxana

2 1

OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion
by Etienne URBAH 19 Jul '10

19 Jul '10

Balazs, Morris, Luigi, Johannes and all, Concerning the capture and discussion of the requirements : I think that the main problem is NOT the large number of requirements, but the fact that many original requirements were formulated in a very confusing manner. In particular, many original requirements simultaneously present : A) mainly, the WAY the Execution Service would perform a desired functionality, B) sometimes, the INTERFACE which a Client would use to access this functionality, C) implicitly, the NEED for this functionality. This causes us to lose much time in useless discussions, because this requires to use following process : - First, focus on C) to precisely define the functionality, and assess its usefulness, - Second, focus on B) to define the Client interface, - Third, verify that there is no garbage from A) left anymore. Working on the original explicit, implicit or hidden requirements found inside the documents, this is exactly the process which I personally followed, splitting complex confusing requirements into simpler and clearer ones. This permitted that requirements which I personally wrote down could be often quickly understood, discussed, and even sometimes quickly labeled YES (positively agreed) or NO (No agreement yet). For the many requirements which are still unclear, I therefore urge you to accept that I split them into simpler smaller requirements : - Apparently, it will increase the total number of requirements, - But in fact, after an easy work, that will increase the percentage of requirements labeled YES and diminish the percentage of requirements labeled UNCLEAR. As example, please study following requirement, labeled 'Still UNCLEAR after 10 mn discussion' : - JD20 (115) Job Description document should offer a new re-usable structure to define 'scalabletime' requests optionally depending on benchmark results. I will split it into the 4 following requirements : - Description of the needed functionality : JD20.1 For 'Wall clock time' and 'CPU time' limits, the Execution Service SHOULD accept Client requirements independently of the speed of the Computing Resource on which the Activity will run - Description of the conceptual Client interface : JD20.2 For 'Wall clock time' and 'CPU time' limits, the Job Description document SHOULD offer a new re-usable structure permitting to describe requirements independently of the speed of the Computing Resource on which the Activity will run - First option for a precise description of the Client interface : JD20.3 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be the ordered pair (Value, Unit) where : - 'Value' is the max number of operations allowed for the Activity, - 'Unit' is as unit suitable for a total number of operations, from a closed list to be refined (such as 'Specint2006 * hours' or 'Billions of floating point operations') This method is already used by BOINC and has proved it robustness. - Second option for a precise description of the Client interface : JD20.4 For 'Wall clock time' and 'CPU time' limits, the re-usable structure permitting to describe requirements independently of the speed of the computing resource on which the Activity will run SHOULD be a 'scalabletime' specified by a (Value, BenchmarkType, BenchmarkValue) triplet as described in chapter 7.2 of the Specification Draft I am absolutely certain that these 4 simple requirements will be quickly understood. I am quite sure that we will immediately obtain a YES for JD20.1 and JD20.2, and perhaps quickly a YES to either JD20.3 or JD20.4 (and a NO to the other one). In the same way of thinking, I will re-propose following dropped requirements. With sometimes an improved title, they describe the NEED for a precise Client interface, WITHOUT specifying the precise component which will implement them : - IS10 For requests querying Status of finished Activity, the Information functionality MUST accept, as input parameter, 1 Activity ID - IS12 For requests querying Activity Status, the Information functionality MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE2 model without explicit specification of the Activity IDs. The Information functionality MAY support multiple query languages in addition to the mandatory one. - LB4 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, 1 Activity ID - LB6 For all requests, the Query functionality for Activity Logs MUST accept, as input parameter, complex queries on Activities expressed according to the GLUE model without explicit specification of the Activity IDs. The Logging and Bookkeeping functionality MAY support multiple query languages in addition to the mandatory one. Besides, I just detected that we forgot to study following requirement, which was mentioned in my mail dated 26 April 2010. Is it possible to associate spreadsheet ID 172 to it ? - IS14 In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service Thank you in advance for accepting that, when starting with original confusion : - rigidity prevents improvement and favors endless confusion, - flexibility favors improvement and then clarity. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

3 5

OGF PGI Use Cases - EDGI
by Etienne URBAH 16 Jul '10

16 Jul '10

Johannes, Morris and all, Concerning PGI Use Cases : I have improved and reformatted the 2 EDGI Use Cases with the 'PGI Software Engineering Use Case' template available at http://forge.gridforum.org/sf/go/doc16024?nav=1 : - 'Enforce Security of a Production Service Grid' is available at http://forge.gridforum.org/sf/go/doc16010?nav=1 - 'Marshal Activities from Service Grid to Desktop Grid' is available at http://forge.gridforum.org/sf/go/doc16022?nav=1 Comments and suggestions for improvement are welcome. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 1

PGI Call today Agenda, Dial in information
by Johannes Watzl 15 Jul '10

15 Jul '10

PGI call today, Thursday 15 July 2010, 15:30 (CET): Agenda: (a) Discuss Use Cases and Use Case Template (b) Discussion of the whole document process : Which documents should OGF PGI produce, in which order, in which time frame, ... (c) Co-chair, Secretary positions (d) AOB Dial in information: https://www1.gotomeeting.com/join/890872960 Use your microphone and speakers (VoIP) - a headset is recommended. Or, call in using your telephone. Austria: +43 (0) 7 20881407 Belgium: +32 (0) 28 08 4297 Switzerland: +41 (0) 225 3314 55 Germany: +49 (0) 898 7806 6464 Denmark: +45 69 91 88 70 Spain: +34 955329911 Finland: +358 942 599 697 France: +33 (0) 182 880 463 United Kingdom: +44 (0) 203 318 4725 Ireland: +353 (0) 14 845 983 Italy: +39 0553989569 Netherlands: +31 (0) 208 080 385 Norway: +47 21 04 29 13 Sweden: +46 (0) 313 613 559 Access Code: 890-872-960 Audio PIN: Shown after joining the meeting Meeting ID: 890-872-960 GoToMeeting® Online Meetings Made Easy™ -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0

PGI call 2010-07-08 notes
by Johannes Watzl 12 Jul '10

12 Jul '10

Participants: Morris, Mark, Michael Saravo (new), Etienne, Johannes, Luigi, Michaela, Emmanouil notes by Etienne (16:00) and Johannes (15:30-16:00) Luigi: summarize the PGI progress -> added to the agenda (a) Johannes as new additional co-chair Morris: Secretary position vacant Requested a third co-chair (a.5) News from the OGF and workshop good progress on the requirements list somebody from gLite should look at the list because there was nobody present at OGF reorganization of glossary of terms on website (wiki) import from OGSA_BES and GLUE glossaries gone through the whole glossary on wiki and OGSA_BES action for Luigi: check requirements and glossary action for Morris: check if the document on gridforge isup to date Mark: what is the outcome? docuement of requirements? Morris: action: organize an output document (b) Debrief Use case Process and status Who send in use cases? Morris: according to usual process: we collected requirements collect use cases -> specification Actions ------- Chairs (Morris and Johannes): - Find a 3rd co-chair - Find a secretary - For the Requirements Spreadsheet : Check if GridForge contains the latest version. If not, then upload the latest version to GridForge - Give write access on GridForge to Emmanouil and Luigi so that they can upload documents - Next PGI telephone conference : Discussion not only on Use Cases, but also on the whole document process : Which documents should OGF PGI produce, in which order, in which time frame, ... Everyone can add comments to the template for Use Cases http://forge.gridforum.org/sf/go/doc16013?nav=1 - Etienne will add some comments Following people should upload their Use Cases following the template to GridForge http://forge.gridforum.org/sf/docman/do/listDocuments/projects.pgi-wg/docma… : - Emmanouil for Globus - Luigi for gLite - Morris for Unicore Following people should convert their Use Cases according to the provided template : - Etienne for EDGI - Aleksandr for ARC Next OGF PGI telephone conference on Thursday 2010-07-15 at 15h30 CET Review process for the Use Cases -------------------------------- Each member of OGF PGI may add comments to each Use Case. Options: # 1 - Compile one single draft document containing all Use Cases, everybody can comment this single document Vote YES by Morris, Mark Vote NO by Etienne : In order to enforce traceability when many people simultaneously add comments to the same document, only a simple text document under SVN is suitable, NOT a MS-WORD or OpenOffice document # 2 - For a while, keep 1 separate document for each Use Case in order to receive comments, later compile all of them into a single document Vote YES by Etienne, Aleksandr In order to manage modifications of a document by several people : - GridForge permits to put a separate lock on each document, - It is more efficient to manage a separate lock for each one of several separate documents than 1 single lock on 1 single big document. # 3 - Google Docs Vote NO by Etienne : That is NOT OGF groupware. Agreed process: Option # 2 Assume each Use Case document is on GridForge 1) Using GridForge, lock the document, then download it 2) Using a document editor (MS-Word, OpenOffice, ...), turn the 'track changes' option on, perform the changes, save the document 3) Using GridForge, upload the document, then unlock it 4) Inform the group via e-mail that there have been changes Deadlines : 2010-07-15 for everyone to make comments on each separate Use Case 2010-07-22 for the initial author to integrate comments on each separate Use Case -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0

Reminder & Agenda: PGI Telcon Today 2010-07-08; 3:30 pm CET
by Morris Riedel 08 Jul '10

08 Jul '10

Dear PGI team, today will be a telcon mostly debriefing the use case gathering process: (1) Telcon-System We still have no other options, so I suggest the DFN system today. It's part of the screen-sharing, please see (2) (2) Screen-Sharing (Voice Backup) http://webconf.vc.dfn.de/ogf-pgi-wg/ No login is required. Just enter as 'guest' with your real world name. (3) Proposed agenda: (a) Johannes as new additional co-chair (b) Debrief Use case Process and status (c) AOB Please feel free to add any other topic on the agenda. Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

PGI additional co-chair election process
by Riedel, Morris 08 Jul '10

08 Jul '10

Dear PGI members, as Balazs recently stepped down as a co-chair we have now two positions vacant for these roles to support me in chairing the activities of the group. Although I'm still keen in supporting this group as co-chair, some new roles within JUELICH (e.g. becoming deputy division head) will have potentially an impact in the time I can invest in this group and thus we need urgently a backup for me - the sooner the better. The election of the new co-chairs is an open process and the group decides, but keep in mind that we have a tough time schedule and these positions should be filled as soon as possible to maintain progress. Hence, to support a faster election process, I already discussed group setups with members of OGF and PGI folks, I would therefore like to take the opportunity to recommend the group setup as follows: (1) I recommend that Johannes Watzl (LMU, Munich, Germany) would be one of the additional co-chairs. Johannes is active in GIN since years and participated in PGI since it's beginning as being an effective secretary. Furthermore, his technology-neutral position makes him even more a suitable candidate for chairing sessions/workshops with delicate decisions among the technology providers that contribute to PGI. I already discussed this with Johannes and he is willing to take this role once selected by the group. (2) The selection of Johannes would leave the secretary position as vacant. Given the past and current efforts of Etienne Urbah (LAL, Paris, France) via posters/e-mails/wiki-pages/etc., I would recommend that he should be elected for the secretary role. I already discussed with Etienne his contributions and he is keen to support the group as well in a more formal position. (3) One further additional co-chair position is still open. I'm in discussion with GIN/PGI members from outside of Europe to promote PGI as an international activity! Until these discussions will come to a fruitful state with folks that are willing to give the time required, I'm still open to receive recommendations from the community. Finally, please send me any serious objections (privately) to the proposed setup. If I do not receive significant objections until the end of next week (2010-07-02), I see this setup as being agreed by the group. Afterwards, I will be once again in contact with our area director Andrew Grimshaw to officially confirm the above recommended setup. Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------

2 1

PGI use cases
by Johannes Watzl 07 Jul '10

07 Jul '10

Dear all, in today's meeting it was suggested to collect use cases of the currently participating middlewares to make the basis for the requirements list easier to understand. The following persons have been assigned initially: ARC use case: Aleksandr EDGI use case: Etienne GENESIS-II use case: Mark GLOBUS use case: Emmanouil gLite use case: Luigi RENKEI use case: Kazushige UNICORE use case: Morris Deadline for first draft Friday, 2 July 2010. (1 page max) The template for use cases will be sent out Friday this week. Every other use case is very welcomed. Best, Johannes -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

2 2