August 2010 - pgi-wg - lists.ogf.org

Re: [Pgi-wg] [EGI Technical Forum 2010] Abstract 74: Standardization of Distributed Data Processing - Useful Standards and Interfaces
by Etienne URBAH 05 Aug '10

05 Aug '10

Steven, Concerning Abstract 74 for my poster proposed at the EGI Technical Forum 2010 : After logging in INDICO, I can read the content of my abstract, but the 'modify' button in NOT active (both on IE and Firefox). Therefore, I am providing the abstract in literary English below. This poster presents two UML diagrams concerning standards which may be useful for grid computing middleware, and interfaces which should be taken into account for proper design of grid middleware permitting operation in production. Selection and then widespread adoption of adequate standards would permit grid interoperability. The web page http://forge.gridforum.org/sf/go/doc15990?nav=1 contains a draft version of a first UML diagram presenting official and de facto standards which may be useful for the design of middleware permitting remote data processing (cloud computing) and distributed data processing (grid computing). The main point is that the GFD.147 recommendation of OGF, named GLUE Specification v. 2.0, permits to describe grid and cloud entities. This allows different GLUE compliant grids to discover and understand each other, so that this GLUE specification should be used as foundation for all other grid and cloud standards and recommendations for interoperability. The web page http://forge.gridforum.org/sf/go/doc15981?nav=1 contains a draft version of a second UML diagram presenting interfaces between Clients, Functionalities and Resources existing in remote data processing infrastructures (clouds) and in distributed data processing infrastructures (grids). This second diagram uses color coding, with grid and cloud functionalities and resources in green, interfaces permitting to access them in blue. Besides, official and de facto standards are inside square brackets. Final Users are scientists, which are interested only in 'payload' functionalities, like data, instruments, activities, and jobs, but grids and clouds require also 'support' functionalities, in particular information publication and discovery, security management, monitoring, logging and accounting. Therefore, interoperability requires to standardize all 'client interfaces' shown in blue, at the left. Grid and cloud functionalities are not independent of each other, but most functionalities require to access resources managed by other functionalities, in particular information items, security descriptors, logging records and accounting records. So, if we do not standardize 'backend' interfaces shown in blue, at the right, then different instances implementing the same functionality will not be able to correctly access required resources, and interoperability will not be achieved. Therefore, we have to standardize also these 'backend' interfaces. This task is huge, so we have assess, for each 'backend' interface, if standardization is really needed, and we have to prioritize, with following suggestions : Firstly, cloud and grid computing require information publication and discovery. For interoperability, we already have the GLUE specification as foundation, and we urgently need GLUE renderings, in particular for LDAP, XML, and SQL. Clients performing queries do not need that the information records are really kept in a relational database, but that queries may be expressed in a (perhaps limited) SQL syntax. Secondly, clouds and grids are valuable targets for criminal organizations, so that clouds and grids absolutely require high-grade security management, and interoperability between grids (an later clouds) require a common set of standards. We should confirm IGTF as Trust Anchor and SPG (formerly JSPG) for general security policies. For authentication, we should phase out GSI proxies in favor of RFC3820-compliant X509 proxies as soon as possible. For authorization, VOMS extensions of X509 proxies work correctly, but SAML assertions issued for example by VOMS servers also permit restrictions of authorization, and perhaps easier delegation. Thirdly, security audit absolutely requires log records for jobs and other activities which clients can submit. Interoperability between grids require that the format of the log records is standardized. We have no published standard yet, and we urgently need one. The JSDL working group of OGF is working on such a standard named 'Activity Instance Document Schema'. We should quickly assess if the draft available at http://forge.gridforum.org/sf/go/doc15395?nav=1 is appropriate. If yes, we should help the JSDL working group finalize it as an OGF recommendation, and implement it. Fourthly, accounting will become more and more important, as grid computing evolves from a research topic to a valuable commodity like cloud computing. The GFD.98 recommendation of OGF, named 'Usage Record - Format Recommendation', is already published. We should assess if this recommendation is appropriate, and if necessary, improve it, then implement it. By contrast, job management is used by very few other functionalities. As long as the four functionalities mentioned above are correctly implemented according to common standards, interoperation of job management between different clouds and grids could then be quickly achieved using bridges implementing an adequate adapter for each job management idiom, though perhaps with reduced scalability compared to full interoperability using the GDF.108 recommendation named 'OGSA® Basic Execution Service Version 1.0'. Now that I have finished, I thank Steven very much : This makes a good start for an academic paper. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr ----------------------------------------------------- On Thu, 05/08/2010 15:42, Steven Newhouse wrote: > Dear Etienne, > > We would like to accept your poster but feel the abstract needs to be > revised to better reflect the proposed content. Can you please submit a > better abstract - one where the text is easier to read rather than a > series of bullet points - immediately. > > Thanks, > > Steven

1 0

Re: [Pgi-wg] DEISA/PRACE and EGEE/EGI Interoperabiltiy Use case Uploaded
by David Wallom 05 Aug '10

05 Aug '10

I have done this now I believe David On 05/08/2010 13:12, "Morris Riedel" <m.riedel(a)fz-juelich.de> wrote: > Thanks for this David! > >> -- -----Ursprüngliche Nachricht----- >> -- Von: pgi-wg-bounces(a)ogf.org [mailto:pgi-wg-bounces@ogf.org] Im Auftrag von >> David Wallom >> -- Gesendet: Donnerstag, 5. August 2010 13:34 >> -- An: pgi-wg(a)ogf.org >> -- Betreff: Re: [Pgi-wg] DEISA/PRACE and EGEE/EGI Interoperabiltiy Use case >> Uploaded >> -- >> -- Hi Morris, >> -- >> -- I am going to start a review on your use case within gridforge. I have >> most >> -- comments already thought out so shouldn't take too long. >> -- >> -- David >> -- >> -- >> -- On 05/08/2010 11:24, "Morris Riedel" <m.riedel(a)fz-juelich.de> wrote: >> -- >> -- > Hi PGI team, >> -- > >> -- >> -- TBD: DEISA/UNICORE scientific use case (Morris) >> -- > >> -- > sorry for being a little bit late with this, but I tried to be as >> precise as >> -- > possible with this use case. >> -- > >> -- > It is available in GridForge: >> -- > >> http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do…>> a >> -- > n.root.input_documents.use_cases/doc16039 >> -- > >> -- > It would be nice if 1-2 could comment on this one. Thanks. >> -- > >> -- > Take care, >> -- > Morris >> -- > ------------------------------------------------------------ >> -- > Morris Riedel >> -- > Deputy Division Leader >> -- > Distributed Systems and Grid Computing Division >> -- > >> -- > Jülich Supercomputing Centre (JSC) >> -- > Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel >> -- > >> -- > >> -- > >> -- > _______________________________________________ >> -- > Pgi-wg mailing list >> -- > Pgi-wg(a)ogf.org >> -- > http://www.ogf.org/mailman/listinfo/pgi-wg >> -- >> -- -- >> -- =================================== >> -- Dr David Wallom >> -- Technical Manager >> -- Oxford e-Research Centre >> -- University of Oxford >> -- Rm 160, 7 Keble Road >> -- Oxford >> -- OX1 3QG >> -- >> -- +44(0)1865 610601 >> -- =================================== >> -- >> -- >> -- _______________________________________________ >> -- Pgi-wg mailing list >> -- Pgi-wg(a)ogf.org >> -- http://www.ogf.org/mailman/listinfo/pgi-wg -- =================================== Dr David Wallom Technical Manager Oxford e-Research Centre University of Oxford Rm 160, 7 Keble Road Oxford OX1 3QG +44(0)1865 610601 ===================================

2 1

Re: [Pgi-wg] ARC Use Cases
by Etienne URBAH 05 Aug '10

05 Aug '10

Aleksandr, Concerning the 'ARC Use cases' : I have just uploaded my comments as version 2 of the document at http://forge.gridforum.org/sf/go/doc16012?nav=1 Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr ----------------------------------------------------- On Wed, 04/08/2010 08:53, Morris Riedel wrote: > Dear PGI team, > > I just reviewed the ARC use cases: > > http://forge.ogf.org/sf/go/docr1185 > > > I would like to have a second reviewer - is there a volunteer? > > > Your co-chair, > Morris >> -- -----Ursprüngliche Nachricht----- >> -- Von: SourceForge Administrator [mailto:ogf-it@ogf.org] >> -- Gesendet: Mittwoch, 4. August 2010 08:49 >> -- An: Riedel, Morris >> -- Betreff: A document review has been submitted for Initial Review, ARC Use Cases >> -- >> -- >> -- Morris Riedel has submitted a review. >> -- >> -- The reviewer's comment: >> -- # use cases very general >> -- # often no specific use case applications are given >> -- # no production grid names are provided >> -- # very good matching to existing requirements from our list >> -- >> -- The reviewer has included an attachment. >> -- >> -- To view the submitted review, visit http://forge.ogf.org/sf/go/docr1185 >> >> >> _______________________________________________ >> Pgi-wg mailing list >> Pgi-wg(a)ogf.org >> http://www.ogf.org/mailman/listinfo/pgi-wg

1 0

Re: [Pgi-wg] DEISA/PRACE and EGEE/EGI Interoperabiltiy Use case Uploaded
by David Wallom 05 Aug '10

05 Aug '10

Hi Morris, I am going to start a review on your use case within gridforge. I have most comments already thought out so shouldn't take too long. David On 05/08/2010 11:24, "Morris Riedel" <m.riedel(a)fz-juelich.de> wrote: > Hi PGI team, > >> -- TBD: DEISA/UNICORE scientific use case (Morris) > > sorry for being a little bit late with this, but I tried to be as precise as > possible with this use case. > > It is available in GridForge: > http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do… > n.root.input_documents.use_cases/doc16039 > > It would be nice if 1-2 could comment on this one. Thanks. > > Take care, > Morris > ------------------------------------------------------------ > Morris Riedel > Deputy Division Leader > Distributed Systems and Grid Computing Division > > Jülich Supercomputing Centre (JSC) > Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel > > > > _______________________________________________ > Pgi-wg mailing list > Pgi-wg(a)ogf.org > http://www.ogf.org/mailman/listinfo/pgi-wg -- =================================== Dr David Wallom Technical Manager Oxford e-Research Centre University of Oxford Rm 160, 7 Keble Road Oxford OX1 3QG +44(0)1865 610601 ===================================

3 3

Reminder & Agenda: PGI Telcon Today 2010-08-05; 3:30 pm CET
by Morris Riedel 05 Aug '10

05 Aug '10

Dear PGI team, today will be our usual telcon: (1) Telcon-System and Screen-Sharing http://webconf.vc.dfn.de/ogf-pgi-wg/ No login is required. Just enter as 'guest' with your real world name. (2) Proposed agenda: (a) Debrief notes from the last meeting (b) Debrief use case process, status, and review coverage (c) Debrief glossary document process driven by Etienne (d) AOB Please feel free to add any other topic on the agenda that I may have missed. Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Deputy Division Leader Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

DEISA/PRACE and EGEE/EGI Interoperabiltiy Use case Uploaded
by Morris Riedel 05 Aug '10

05 Aug '10

Hi PGI team, >-- TBD: DEISA/UNICORE scientific use case (Morris) sorry for being a little bit late with this, but I tried to be as precise as possible with this use case. It is available in GridForge: http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do… It would be nice if 1-2 could comment on this one. Thanks. Take care, Morris ------------------------------------------------------------ Morris Riedel Deputy Division Leader Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel

1 0

OGF PGI Glossary of Terms document based on OGSA Glossary of Terms GFD.120
by Etienne URBAH 04 Aug '10

04 Aug '10

Morris, Johannes and all, Concerning the 'PGI Glossary of Terms' document based on 'OGSA Glossary of Terms' GFD.120 : - I needed to retrieve the source document of the GFD.120 At OGF, published documents are often simply PDF versions of documents stored by the corresponding Working Group inside GridForge. Therefore, I searched the source document inside the 'Documents' area of OGSA-WG in GridForge, and I found it at http://forge.gridforum.org/sf/go/doc13938?nav=1 - This permitted me to immediately create the first draft of the 'PGI Glossary of Terms' : - Chapter 2 containing the list of terms is mostly unchanged compared to OGSA. - Header, footer and all other chapters are already updated for PGI . I saved this first draft at http://forge.gridforum.org/sf/go/doc16036?nav=1 I will update chapter 2 using the PGI Vocabulary Wiki page. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah@lal.in2p3.fr -----------------------------------------------------

1 4

WG: A document review has been submitted for Initial Review, EGI Use Case
by Morris Riedel 04 Aug '10

04 Aug '10

Hi PGI team, I had a short view on the interesting EGI use case with minor comments: http://forge.ogf.org/sf/go/docr1186 Given that there are two authors (Oxana, Steven) and two reviewers (Etienne, Morris) I believe this document review process can be closed. Keep in mind that there will be a collective use case document later where comments will be again possible. Take care, Morris >-- -----Ursprüngliche Nachricht----- >-- Von: SourceForge Administrator [mailto:ogf-it@ogf.org] >-- Gesendet: Mittwoch, 4. August 2010 09:22 >-- An: Riedel, Morris >-- Betreff: A document review has been submitted for Initial Review, EGI Use Case >-- >-- >-- Morris Riedel has submitted a review. >-- >-- The reviewer's comment: >-- # very good use case >-- # not trivial to map to our requirements list >-- # A bit general and some application examples might be helpful >-- # nevertheless might become key use case for PGI >-- # minor comments in the document >-- >-- The reviewer has included an attachment. >-- >-- To view the submitted review, visit http://forge.ogf.org/sf/go/docr1186

1 0

WG: A document review has been submitted for Initial Review, ARC Use Cases
by Morris Riedel 04 Aug '10

04 Aug '10

Dear PGI team, I just reviewed the ARC use cases: http://forge.ogf.org/sf/go/docr1185 I would like to have a second reviewer - is there a volunteer? Your co-chair, Morris >-- -----Ursprüngliche Nachricht----- >-- Von: SourceForge Administrator [mailto:ogf-it@ogf.org] >-- Gesendet: Mittwoch, 4. August 2010 08:49 >-- An: Riedel, Morris >-- Betreff: A document review has been submitted for Initial Review, ARC Use Cases >-- >-- >-- Morris Riedel has submitted a review. >-- >-- The reviewer's comment: >-- # use cases very general >-- # often no specific use case applications are given >-- # no production grid names are provided >-- # very good matching to existing requirements from our list >-- >-- The reviewer has included an attachment. >-- >-- To view the submitted review, visit http://forge.ogf.org/sf/go/docr1185

1 0

EGI use case
by Oxana Smirnova 03 Aug '10

03 Aug '10

Hi all, since there were no other volunteers, I did my bestwriting up EGI use case: http://forge.gridforum.org/sf/go/doc16031 As I am not affiliated with EGI, I do not claim I got everything right, and will appreciate if EGI people can cross-check and correct whatever is wrong. Cheers, Oxana

4 5