- pgi-wg - lists.ogf.org

Re: [Pgi-wg] PGI decision-making process - seeking input
by Oxana Smirnova 12 Nov '10

12 Nov '10

Hi Morris, this is not the final text yet, but a call for input, as it is said both in the subject and in the text. Please do not copy it into any Word file, I will send you one (and upload to the GridForge). Cheers, Oxana 12.11.2010 06:06, Morris Riedel пишет: > Hi, > > when I copy this into a word file using font size 11 this process is more than a half page as agreed. Could you please shorten it and make it more concrete to PGI. > > Many thanks. > > Take care, > Morris > >> -- -----Ursprüngliche Nachricht----- >> -- Von: pgi-wg-bounces(a)ogf.org [mailto:pgi-wg-bounces@ogf.org] Im Auftrag von Oxana Smirnova >> -- Gesendet: Donnerstag, 11. November 2010 23:27 >> -- An: pgi-wg(a)ogf.org >> -- Betreff: [Pgi-wg] PGI decision-making process - seeking input >> -- >> -- Hi all, >> -- >> -- some suggestions on decision-making process in PGI were already made, both on this list and during today's >> -- meeting (trust me ;-) ). I was appointed by the group co-chair to prepare a proposal for simple internal >> -- guidelines (this ought to be in the minutes). >> -- >> -- I therefore would like to solicit input from those of you who have opinion and did not express it yet. I'll try >> -- to outline the background: >> -- >> -- 1. PGI stands for "Production Grid Infrastructures", specifically: >> -- 1a. Stakeholders represent parties enabling production grid infrastructures: >> -- * administrators >> -- * middleware providers >> -- * users >> -- 2. (1) implies certain group specific, namely: >> -- 2a. Stakeholders enter with pre-existing deployed solutions, related interfaces etc >> -- 2b. There are substantial numbers of customers behind every stakeholder >> -- 2c. There is a substantial, and ever growing, number of stakeholders >> -- >> -- For reference, here's a list of distinct stakeholders represented in PGI which I deduced from currently active >> -- PGI participants, in no particular order: >> -- UNICORE, DEISA/PRACE, Globus/IGE, EDGI, NDGF, ARC, EGI, gLite, GENESIS, TeraGrid, SAGA, NAREGI/RENKEI, NGS (and >> -- I probably missed some). >> -- Several stakeholders are represented by more than 1 individual. Even if UNICORE, ARC and gLite will become a >> -- single EMI stakeholder, there will still be more than 10. >> -- >> -- In general, the group should follow the OGF guidelines and aim to achieve consensus. However, due to the large >> -- number of stakeholders and high stakes, consensus is often practically impossible. In such cases, decisions must >> -- be made following a consistent *open* procedure, and not left to the judgement of a single person: this is how >> -- it is done in modern democratic societies. >> -- >> -- A solution can be to: >> -- a) Limit number of decision-makers by applying criteria such as affiliation (one voice per stakeholder) and/or >> -- attendance (75% attendance of all group meetings, or similar), or >> -- b) Introduce voting procedures (quorum, majority, tie-breaking, veto), or >> -- c) Combine (a) and (b). >> -- >> -- Important decisions which involve complex documents and/or potentially affect functionality of the production >> -- Grid infrastructures, must be well prepared in advance: >> -- * proposal authors must provide sufficient material for the stakeholders such that they can use it to consult >> -- their user base if necessary >> -- * sufficient preparatory stage must be allowed, during which all the stakeholders will have the opportunity to >> -- study the proposals and consult their customers if necessary >> -- * materials must be made public on GridForge and advertised on the PGI mailing list, together with relevant >> -- deadlines >> -- * decisions achieved by consensus, and *especially* those achieved by voting, must be documented in respective >> -- public meeting notes for further reference >> -- >> -- >> -- Please send your thoughts either to the list or directly to me by next Wednesday, November 17. >> -- >> -- >> -- I anticipate irritated comments about overcomplicating the process and violating OGF rules. Believe me, there is >> -- nothing complicated, and this is a very simplified model of public decision-making process, successfully used in >> -- bodies from condominium boards to country parliaments. Ad-hoc procedures, absence of relevant documentation and >> -- reliance on a wise leader typically lead to stagnation and failures, even if there are individual success >> -- stories. I am confident OGF is not pursuing this latter track. >> -- >> -- >> -- Cheers, >> -- Oxana >

1 0

Re: [Pgi-wg] Requirements spreadsheet and comment period
by Oxana Smirnova 12 Nov '10

12 Nov '10

Hi Morris, you perhaps misunderstood my point: please circulate the spreadsheet with requirements, along with a clear call for comments and a clear deadline date for such. Or perhaps Johannes can do it - this is a most trivial task and I see no reason for delays. Many thanks again, Oxana

1 0

PGI decision-making process - seeking input
by Oxana Smirnova 12 Nov '10

12 Nov '10

Hi all, some suggestions on decision-making process in PGI were already made, both on this list and during today's meeting (trust me ;-) ). I was appointed by the group co-chair to prepare a proposal for simple internal guidelines (this ought to be in the minutes). I therefore would like to solicit input from those of you who have opinion and did not express it yet. I'll try to outline the background: 1. PGI stands for "Production Grid Infrastructures", specifically: 1a. Stakeholders represent parties enabling production grid infrastructures: * administrators * middleware providers * users 2. (1) implies certain group specific, namely: 2a. Stakeholders enter with pre-existing deployed solutions, related interfaces etc 2b. There are substantial numbers of customers behind every stakeholder 2c. There is a substantial, and ever growing, number of stakeholders For reference, here's a list of distinct stakeholders represented in PGI which I deduced from currently active PGI participants, in no particular order: UNICORE, DEISA/PRACE, Globus/IGE, EDGI, NDGF, ARC, EGI, gLite, GENESIS, TeraGrid, SAGA, NAREGI/RENKEI, NGS (and I probably missed some). Several stakeholders are represented by more than 1 individual. Even if UNICORE, ARC and gLite will become a single EMI stakeholder, there will still be more than 10. In general, the group should follow the OGF guidelines and aim to achieve consensus. However, due to the large number of stakeholders and high stakes, consensus is often practically impossible. In such cases, decisions must be made following a consistent *open* procedure, and not left to the judgement of a single person: this is how it is done in modern democratic societies. A solution can be to: a) Limit number of decision-makers by applying criteria such as affiliation (one voice per stakeholder) and/or attendance (75% attendance of all group meetings, or similar), or b) Introduce voting procedures (quorum, majority, tie-breaking, veto), or c) Combine (a) and (b). Important decisions which involve complex documents and/or potentially affect functionality of the production Grid infrastructures, must be well prepared in advance: * proposal authors must provide sufficient material for the stakeholders such that they can use it to consult their user base if necessary * sufficient preparatory stage must be allowed, during which all the stakeholders will have the opportunity to study the proposals and consult their customers if necessary * materials must be made public on GridForge and advertised on the PGI mailing list, together with relevant deadlines * decisions achieved by consensus, and *especially* those achieved by voting, must be documented in respective public meeting notes for further reference Please send your thoughts either to the list or directly to me by next Wednesday, November 17. I anticipate irritated comments about overcomplicating the process and violating OGF rules. Believe me, there is nothing complicated, and this is a very simplified model of public decision-making process, successfully used in bodies from condominium boards to country parliaments. Ad-hoc procedures, absence of relevant documentation and reliance on a wise leader typically lead to stagnation and failures, even if there are individual success stories. I am confident OGF is not pursuing this latter track. Cheers, Oxana

3 2

Re: [Pgi-wg] OGF PGI - Review of notes of OGF30 sessions on 26 October 2010 - Counting votes for requirements
by Oxana Smirnova 12 Nov '10

12 Nov '10

Hi Morris, all, I can't speak for Balazs, but I do agree that in order to implement the procedures nicely outlined by Andre, we need to define, for each and every decision: 1. what are the alternative options on which decisions have to be taken 2. who can vote (if voting is necessary) 3. which documents are available to support one or another options, and how much time is allotted to study the documents before the decision making So, nothing new - just business as usual, only in an orderly manner. Cheers, Oxana 2010-11-11 14:45, Morris Riedel пишет: > Dear Balazs, > > would you perhaps join the call today in order to explain more in detail what you mean? > > Thanks for your time. > > Take care, > Morris > >> -- -----Ursprüngliche Nachricht----- >> -- Von: pgi-wg-bounces(a)ogf.org [mailto:pgi-wg-bounces@ogf.org] Im Auftrag von Balazs Konya >> -- Gesendet: Donnerstag, 11. November 2010 13:46 >> -- An: pgi-wg(a)ogf.org >> -- Betreff: Re: [Pgi-wg] OGF PGI - Review of notes of OGF30 sessions on 26 October 2010 - Counting votes for >> -- requirements >> -- >> -- hi Andre, all, >> -- >> -- On 2010-11-07 19:01, Andre Merzky wrote: >> -- > For PGI, my very humble opinion is that a charter update is not needed >> -- > as long as the group is undecided on the explicit way forward -- and >> -- > that decision is long overdue. >> -- > >> -- > If a group is deadlocked like PGI (or rather if it is running circles >> -- > as PGI seems to do), it is the duty of the chairs to push the group >> -- > along. In the worst case, if full consensus cannot be reached, a vote >> -- > on the available options can lead to rough consensus, which ought to >> -- > be enough to get things going again. >> -- >> -- Thanks for this realistic "status report" on PGI. >> -- >> -- I agree, in order to move the group forward, a push is needed. >> -- >> -- So, as a first step, the group should understand the "available options". Then, >> -- after sufficient discussion of these options a vote should take place. >> -- >> -- For the vote, the group should define which group members have a voting right(s). >> -- >> -- And all this process should be done in a transparent relaxed manner, giving >> -- enough time to people to digest the thing they supposed to vote about. Otherwise >> -- the group will run into similar problems Etienne had just discovered. >> -- >> -- bye, >> -- Balazs >> -- _______________________________________________ >> -- Pgi-wg mailing list >> -- Pgi-wg(a)ogf.org >> -- http://www.ogf.org/mailman/listinfo/pgi-wg >> >> >> _______________________________________________ >> Pgi-wg mailing list >> Pgi-wg(a)ogf.org >> http://www.ogf.org/mailman/listinfo/pgi-wg

2 1

Re: [Pgi-wg] OGF PGI - Review of notes of OGF30 sessions on 26 October 2010 - Counting votes for requirements
by Oxana Smirnova 11 Nov '10

11 Nov '10

Hi, I'd like to point out that my "interesting thoughts" are directly based on the PGI group description here: http://forge.gridforum.org/sf/projects/pgi-wg This was the mandate of the group when it was approved by the OGF, and it explicitly contains the list of relevant standards and specifications, which we just re-discovered. It even contains SRM and GridFTP, well in line with the stated group's committment to deal with data management - something that was contested by the management in Brussels. Perhaps the group description needs to be updated, if management believes it contains controversial statements. What is the procedure for this? Cheers, Oxana 07.11.2010 18:01, Morris Riedel пишет: > Hi, > > Interesting thoughts. Indeed. > > > >-- Or will we start all the specifications from scratch? > > Depends on the rough consensus and majority decisions in the group step by step for each of the specification in question to be > profiled/produces by us. > > >> -- Maybe this is also something to clarify on Thursday. > > Perhaps, but the approach is clear and has been discussed - then with the 'rough consensus' no problem to move forward working on > the specifications. > > > Take care, > Morris > > > > >> -- -----Ursprüngliche Nachricht----- >> -- Von: pgi-wg-bounces(a)ogf.org [mailto:pgi-wg-bounces@ogf.org] Im Auftrag von Oxana Smirnova >> -- Gesendet: Sonntag, 7. November 2010 17:32 >> -- An: pgi-wg(a)ogf.org >> -- Betreff: Re: [Pgi-wg] OGF PGI - Review of notes of OGF30 sessions on 26 October 2010 - Counting votes for >> -- requirements >> -- >> -- Hi Morris, all, >> -- >> -- I came to think about the process: now that we have the use cases and have "derived" the requirements (exact set >> -- of which can be still argued and prioritised in various manners), is it time to come back to the specifications? >> -- The "strawman" and such? The high-level scheme on the photo is in no way different from what we had 2 years ago, >> -- after all (remember, the group was called "BES/JSDL/GLUE" in 2008), the circle is complete now. >> -- >> -- Or will we start all the specifications from scratch? >> -- >> -- Maybe this is also something to clarify on Thursday. >> -- >> -- Cheers, >> -- Oxana

4 6

Reminder & Agenda: PGI Telcon 2010-11-11; 3:30 pm CET
by Morris Riedel 11 Nov '10

11 Nov '10

Dear PGI team, just a gentle reminder about our usual telcon today: (1) Telcon-System and Screen-Sharing http://webconf.vc.dfn.de/ogf-pgi-wg/ No login is required. Just enter as 'guest' with your real world name. (2) Proposed agenda: (a) Check last minutes (b) Short Follow-up : public comments of our 2 documents in the pipeline (c) Requirement Discussions (re: Etienne; re: EMI) (d) Tasks about specification elements of PGI (e) AOB Please feel free to add any other topic on the agenda that I may have missed. Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Deputy Division Leader Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

OGF PGI : Notes of telephone conference held on 04 November 2010
by Etienne URBAH 09 Nov '10

09 Nov '10

Morris and Johannes, Thank you in advance for publishing quickly the notes of the OGF PGI telephone conference held on 04 November 2010. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

2 1

OGF PGI - Review of notes of OGF30 sessions on 26 October 2010 - Counting votes for requirements
by Etienne URBAH 08 Nov '10

08 Nov '10

Morris and Johannes, Thank you for the notes of the PGI sessions held at OGF30 in Brussels on 26 October 2010. Reviewing them and the associated requirement spreadsheet at http://forge.gridforum.org/sf/go/doc16080?nav=1 I have found several issues : A) Green color of spreadsheet cells for vote count (column T) ------------------------------------------------------------- At the end of the notes for session 3, it is mentioned that I 'globally approve all of the greens'. I confirm that I expressed this position. But inside the spreadsheet, the vote count cells (column T) which were green have turned yellow ! Then, my above position has no meaning anymore. B) Requirement supported by EDGI -------------------------------- Inside the requirement spreadsheet, you took into account only the 26 'Mandatory requirements' from my mail below. In fact, you should have taken into account the 'Complete list of requirements' specified at the end of the same mail. Therefore, I have updated the requirement spreadsheet with this complete list (70 requirements). C) Votes for requirements WITHOUT 'yes' status ---------------------------------------------- Inside the process of voting for requirements, we were supposed to vote ONLY for requirements having 'yes' status. Inside the requirement spreadsheet, it seemed to me that some votes were for requirements WITHOUT 'yes' status. In order to be sure, I have reactivated the display of the column displaying 'Agreement OGF29' (column F), and I have colored in yellow all cells NOT containing 'yes'. This clearly shows that MANY votes were for requirements WITHOUT 'yes' status. In particular, requirement 10 'SSL certificates of servers MUST be signed by a CA belonging to IGTF ' gathered 3 votes. This indicates that some requirements NOT having 'yes' status still have a great importance for several PGI members. Therefore, I propose to perform a new round of voting, allowing votes for all requirements WITHOUT consideration of their previous status. If this proposal is accepted, I would also vote for following requirements : 10, 12, 13, 14, 15, 18, 27, 38, 39, 41, 49, 51, 53, 75, 127, 161, 162, 164, 167 Conclusion ---------- Counting of votes has perhaps been performed too hastily. The attached 'PGI-Requirements-2010-10-26-Review-Etienne.xls' file contains the 'Complete list of requirements' of EDGI, and displays the 'Agreement OGF29' column (but I have NOT added any EDGI vote for any requirement NOT having 'yes' status). I suggest that each PGI member carefully checks the content of this spreadsheet. Anyway, I still agree with the priorities stated at OGF30 : 1) XML rendering for GLUE 2.0 2) Usage of GLUE 2.0 entities and attributes inside JSDL 3) Usage of GLUE 2.0 entities and attributes inside JSDL for SPMD and MPI 4) Usage of GLUE 2.0 entities and attributes inside JSDL to describe Applications 5) Usage of GLUE 2.0 entities and attributes inside BES Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr ----------------------------------------------------- -------- Original Message -------- Subject: Re: OGF PGI - Requirements triggered by the EDGI Use Cases Date: Tue, 19 Oct 2010 14:33:25 +0200 From: Etienne URBAH <urbah(a)lal.in2p3.fr> To: Morris RIEDEL <m.riedel(a)fz-juelich.de>, Johannes WATZL <watzl(a)nm.ifi.lmu.de>, pgi-wg(a)ogf.org CC: lodygens(a)lal.in2p3.fr, edgi-na2(a)mail.edgi-project.eu Morris, Johannes and all, Please find below the OGF PGI requirements triggered by the EDGI Use Cases : +-----------------------------+ | A) Mandatory requirements | +-----------------------------+ Following is the list of requirements with 'yes' status which are seen as mandatory for EDGI. Interoperation would be very difficult without them. Information functionality ------------------------- 1 : All grid entities (if possible) MUST be described using the GLUE model. If not possible, extensions for the GLUE model are necessary. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqIS1 5 : The Execution Service MUST NOT expose detailed information about the GLUE entities which the Execution Service does not manage (all that are not expressed by the computing part of GLUE). For example, Storage Element GLUE entity NOT exposed by Execution Service, NO details about Storage entity. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqIS4 165 : For already finished Activities, the Information functionality SHOULD accept requests querying the Activity Status, but MAY return an error http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqIS9 172 : In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Information functionality SHOULD be separated from the Execution Service http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqIS14 Security (Authentication, Authorization, Delegation) ---------------------------------------------------- 9 : If server authentication to clients, then it must be done with X.509 certificates on TLS (as mandatory option, allowing also other mechanisms additionally) http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqAA1 11 : Each Service MUST publish the Authentication and Authorization methods accepted by its Endpoints in conformance with GLUE recommendations http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqAA2 32 : There must be a mechanism which allows users to manage Activities submitted by other users (access control lists/methods/policies). In order to authorize (or not) an request on an Activity, each instance of the Execution Service MUST enforce a consistent authorization method. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqAA9 Application Repository ---------------------- 34 : An easy way of launching applications or pre-configured /pre-installed software w/o specifying location details. Installed/pre-configured ones should be exposed as well as part of the resource description. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqAR1 Accounting ---------- 36 : The Execution Service SHOULD provide Accounting records for each managed Activity. e.g. OGF Usage Records, for tracking resource usage. Most likely improving UR in terms of network and storage resource tracking and improvements of compute parts of multi-core-business. Grid-level attributes (for example: start-time on Grid vs. in batch-system). http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqAc2 173 : In order to prevent overloading the Execution Service (which has performance requirements for Activity Management), the Accounting functionality MUST be separated from the Execution Service http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqAc1 Logging and Bookkeeping ----------------------- 37 : WITH ITS ORIGINAL TITLE : In order to permit efficient Log analysis and Security audit, some Logging and Bookkeeping functionality MUST secure persistence at the grid level of Activity logs from various logging systems and MUST permit Client access to these Activity logs, even after Activities have finished. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqLB1 Activity Management ------------------- 45 : On creation of an Activity, the Execution Service MUST return to the Client an Activity ID permitting the Client to perform subsequent actions (Query, Cancel, ...) on this precise Activity http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJM4 62 : The Execution Service MUST log enough grid information inside logging systems (which MAY vary during the lifetime of the Activity), in order to permit efficient Log analysis and Security audit http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJM6 67 : If a Client queries for an already finished Activity, the Execution Service MAY response with an error http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJM6.8 74 : The execution service MUST NOT expose activity information when queried for resource information http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJM6.… 94 : Requirement to have a purge (maybe called wipe) operation. Removing all presence (except logs & usage records) of the activity (when it is not longer active or once finished). This functionality is only allowed on any final state according to the PGI state model. The functionality is not supposed to kill Activities, that is why its only allowed on final states. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJM20 Job Description --------------- 100 : The Job Description document MUST reference all grid entities in conformance to the GLUE specification http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJD3 101 : The Job Description document specification MUST permit the Client to request automatic data stage-in and stage-out http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJD4 107 : Job Description should be only used for Job Description not for any kind of feedback. Job status information should not be communicated with a 'changed JSDL' http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqJD11 State Model ----------- 119 : The Execution service MUST support a common state model http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqSM1 131 : The Execution service MUST perform the transition between Activity states requested by the Client ONLY if it makes sense. Otherwise, the Execution service MUST return an error to the Client. http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqSM10… Data Management --------------- NO requirement with 'yes' status is essential. Non functional -------------- 146 : Software components (Services and Clients) MUST ease traceability of the original author of a request http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqNF3 147 : Software components (Services and Clients) MUST generate adequate logs http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqNF4 148 : Software components (Services and Clients) MUST generate and propagate meaningful error messages, including context description http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqNF5 149 : Specifications SHOULD prevent the occurrence of SPOFs and bottlenecks http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqNF7 154 : Basic SW Engineering basic principles - implementation encapsulation, separation of policies, construction by composition, don't re-invent new mechanisms when some existing http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqNF12 160 : Execution Service MUST NOT try to provide ALL grid functionalities, but MUST have a well defined scope, and MUST work together with other grid services : Security, Accounting, Data (Storage, Movement, Catalog, ...), perhaps Information, Logging and Bookkeeping http://forge.gridforum.org/sf/wiki/do/viewPage/projects.pgi-wg/wiki/ReqNF13 +------------------------------------+ | B) Complete list of requirements | +------------------------------------+ Following is the complete list of requirements with 'yes' status which are seen as useful for EDGI. Information functionality : 1, 3, 4, 5, 6, 8, 165, 166, 172 Security (Authentication, Authorization, Delegation) : 9, 11, 17, 19, 20, 21, 22, 32 Application Repository : 34 Accounting : 36, 173 Logging and Bookkeeping : 37 WITH ITS ORIGINAL TITLE Activity Management : 43, 44, 45, 46, 55, 58, 59, 62, 63, 64, 65, 66, 67, 68, 74, 81, 82, 86, 90, 94 Job Description : 100, 101, 104, 107, 112, 114, 116, 118 State Model : 119, 120, 121, 122, 123, 124, 125, 128, 131, 134 Data Management : 143 Non functional : 145, 146, 147, 148, 149, 150, 154, 156, 159, 160 Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

4 4

Re: [Pgi-wg] WG: EMI participation in PGI
by Andre Merzky 07 Nov '10

07 Nov '10

Dear Alberto, just to clarify an OGF process point: OGF groups are *always* open for participation! In fact, joining the (public) mailing mailing list, or attending any official group meeting, is considered to suffice for joining a group. Having said that, the PGI group could certainly, IMHO, benefit from an explicit contribution from EMI, in particular as many within the group have been trying to cater (amongst others) to EMI's needs, but quite often reached an impasse due to disagreement on general strategies. I for one certainly hope that a direct EMI involvement can help to move the group forward, one way or the other! Best, Andre. On Thu, Nov 4, 2010 at 1:04 PM, Morris Riedel <m.riedel(a)fz-juelich.de> wrote: > All, > > a “blocked list e-mail” from EMI to the PGI list with statements to > consider and to be discussed in the call later today. > > Take care, > Morris > > Von: Alberto Di Meglio [mailto:Alberto.Di.Meglio@cern.ch] > Gesendet: Donnerstag, 4. November 2010 11:09 > An: pgi-wg(a)ogf.org > Cc: Riedel, Morris > Betreff: EMI participation in PGI > > Dear Morris, dear PGI members, > > One of the major objectives of the EMI project is to make sure that all EMI > services adopt agreed standards and implement them across all supported > middleware stacks in a coherent and consistent way. The work done in PGI is > for EMI very important and I would like to make sure that EMI can actively > contribute to this work and help producing widely agreed open standards. > Until now, EMI has not officially taken part in the PGI activities, despite > a number of current PGI members are in one way or another related to EMI. > > The existing representatives from ARC, gLite and UNICORE have traditionally > represented the long-term objectives of their respective collaborations. EMI > has introduced a different perspective and has requirements and interests > that are in some way larger than the sum of its parts. In this context, we > feel that asking any one of the existing PGI members to represent EMI would > not be correct, both for the members and for EMI itself. > > I would like to ask you to consider accepting an official EMI representative > in PGI to bring the voice of EMI into your activities without ambiguities > and with a clear mandate of actively participate to the definition of the > PGI specifications on one side and a clear commitment to push the adoption > of those specifications in the coming EMI releases. > > I’m aware that you have issued a call for comments on a list of requirements > on which the PGI members are asked to express their opinion. We are probably > late for this round, but we would still like the opportunity to contribute > to this phase in some practical way. It is of course up to you to decide how > this can be done. For the moment, EMI is selecting a person that will be > proposed to you as candidate EMI representative should you decide to accept > EMI within PGI. > > I look forward to working with you in the future. > > Best regards, > > Alberto -- Nothing is ever easy...

2 1

Thoughts on the airplane home (GIN, PGI, Cycle sharing)
by Andrew Grimshaw 07 Nov '10

07 Nov '10

All, I thought we had a very productive OGF. A number of interesting and exciting results in GIN, and solid progress on PGI requirements and a discussion of how to meet those requirements going forward. A couple of things stuck in my mind - so I thought I'd get them out there and get a response. First, with respect to GIN. I thought the demo's, particularly the use of the BES endpoints in SAGA, were very informative. Particularly when one keeps in mind that at least the Unicore/futuregrid resources were not even available until Friday afternoon (eastern time!). I agree with the sentiment that we should build on this very successful demo. In particular most groups have either done simple extensions to JSDL, or used existing mechanism in a particular way. Ditto with BES factory attributes. Several ideas came up (Morris, chime in I think you were keeping notes) Profiling we can do fairly easily: File system JSDL specifies how file systems can be defined - Mark correct me on this. We use one we call SCRATCH as a place to cache files that are large and constant, including often zip files that contain applications. We tell JSDL to mount it, we then use the "don't overwrite on stage-in" JSDL flag, and the "don't delete when you're done" flag. This is a huge help. Extra attributes We have found a need for several extra BES factory attributes and the use of those in JSDL matching. I'm guessing many of you have done this as well. These include whether an application is installed, whether static linking is required, I also think that while we are waiting for PGI to figure out how to deploy applications that we have a simple convention for applications, something like HAS_APPX_INSTALLED And then if the BES factory has that, then by convention that APPX_PATH and APPX_LIB_PATH are defined and can be used in some sort of prefix. File staging - demo of sites that support FSE at SC. We discussed that several implementations support FSE. Let's test that interop. It really changes the set of applications you can run if you can copy them onto the machine. If we can get a set of endpoints that support FSE we'll (GenesisII/XCG) set up a meta-scheduler that takes JSDL and manage them through the life cycle on the endpoints of the different implementations .This might also form the basis for some cycle trading ... Re: FutureGrid. I also want to repeat that if groups want to put up standards based implementations of services on FutureGrid (OGSA-BES, RNS, WS-Trust, ByteIO, gridFTP, SRM) we will work with them to make it so. One of the goals of FutureGrid is to provide PERSISTENT standards based endpoints with which to test. Second, with respect to PGI Application deployment/management. This came across as the number one requirement. This came as a surprise to me, though I agree it is an important next step to make execution services (OGSA-BES) more useful. There are at least two ways to handle this problem - one I'll call manual installation, configuration, and reification (MICR), and the other I'll call automatic deployment, configuration, and reification (ADCR). MICR This is definitely the easy case. First, an application is deployed at/on a site/compute resource. Appropriate metadata is added to the resource metadata or an information service is updated with information on the local bindings for the executable, paths, libraries, etc. Second, once a placement decision is made by a job manager (I'm using the OGSA-EMS term, here think meta-scheduler), the JSDL document that contains an abstract application name is reified (transformed) to have the site-appropriate values for the application. Third, the JSDL is sent to the compute resource. I believe this is basically how UNICORE 6 does this now. ADCR ADCR is similar to MICR except the deploying step of setting the application up and configuring the metadata is done automatically by a service. For simple applications - a single executable, or maybe a zip file that needs to be unzipped, this is easy. It can get complicated really fast. We at Genesis II defined a very simple deploying service combined with a reification service that worked for the easy case - executable and zip file. I can dig up the documentation on this if anyone is interested. I believe that the general case for this is a tarpit for a standards body. I have a PhD student working on it J. **** A note on reification >From Wikipedia Reification generally refers to bringing into being or turning concrete. Specifically, reification may refer to: * Reification (computer science) <http://en.wikipedia.org/wiki/Reification_%28computer_science%29> , making a data model for a previously abstract concept In this context I refer to taking an abstract JSDL that may give an abstract name of an application, e.g., <JSDL-abstract-application-name>/bin/biology/blastp-v5 </ JSDL-abstract-application-name> THIS IS MADE UP SYNTAX That might occur in a JSDL file, and then making it concrete for a particular endpoint by transforming the JSDL to have the actual local path name and defining any environment variables needed. We discussed this problem extensively in the OGSA, OGSA-EMS, and OGSA-BES working groups. A strawman interface was even defined. MPI Once again the importance of this came as a somewhat welcome surprise. I believe this is of considerable interest to UNICORE. Perhaps they could take the lead on this one. Is it simply a problem of the JSDL not being expressive enough to both select appropriate resources and pass topology information down to MPI? In other words, is this an easy problem? Or is there something else I am not seeing? Security The single greatest impediment to the GIN demonstrations was working out the security stuff. Andre was pretty clear about this. In other words, the functional aspects of the standards were not the problem - the certificates and other stuff were. It has been suggested that it may be time to dive back into this. We spent quite a bit of time on it 18 months ago. The result was a DRAFT profile document by Duane Merrill that bifurcated the profile into a transport level security profile based on delegated X.509 certificates and a message level security profile - primarily aimed at SAML (http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do cman.root.input_documents.security_material.comm_profile/doc15575) . I know that there was a lively debate about how to proceed. PGI profile This is the big question. We've gone through use cases, requirements, counting up the use cases that require each requirement, and two approaches to meeting those requirements (see slides). How are we to proceed? Should we ask each proposer to provide more detail? Should we solicit more proposals? Should we work through both paths? Should we vote? My own interest is in extending the existing specs and working with GIN to drive spec modifications. If we go with extending the existing specs - BES in particular, should we modify BES just a little and leave the basic factory and management pattern the same? Or go with a cleaner model more consistent with what Bernd presented (return an activity handle <EPR?> and then manage the activity directly). Either way we need make some decisions and move forward. Third, with respect to Cycle sharing At first I thought that the cycle sharing BOF did not lead to any interested in trying this out. As the week wore on my initial impression proved mistaken. Johannes followed up with several other groups and we may indeed have a sufficient critical mass to proceed. I will ask Joel to set up a source forge site and email list. Once that is set up I will email these two mailing lists with subscription information. My thoughts are to leverage the efforts of the GIN group and perhaps start with all agreements and SLA's being done by humans and verbal discussions - and see if we can actually use others cycles effectively using the existing technology before we leap into anything too complex.

3 3