January 2010 - pgi-wg - lists.ogf.org

PGI-call notes 2010-01-21
by Johannes Watzl 21 Jan '10

21 Jan '10

Dear all, please see below the notes of today's call. Best, Johannes Participants: Etienne, Morris, Johannes, Emannouil, Balazs, Oxana, Jules, Luigi Agenda: a) >-Debrief minutes of last PGI TelCon >- >-(b) >-Debrief PGI actions >- >-(c) >-Very short: continue discussion animation ppt of PGI operation & state model >-Re: Developer experience says MUST might be tough >- >-(d) >-Debrief Steven Newhouse comments on the list >- >-(e) >-Debrief Bernd Schuller comments on the list >- >-(f) >-Debrief Etienne Comments on the list >- >-(g) >-Next meeting >- >-(h) >-AOB Balazs: two more points for discussion shift meeting half an hour earlier agreed to shift the meeing on Thursday 15:30 (CET) how to edit the document make the document more readable notes in ppt - reference in document to ppt Balazs will be editor of the document remark changes Morris will keep track of ppt working slides identify tasks which should be done by certain people ad a) add action: nail down ppt lot of comments ad b) updated version of action will be sent out after the meeting ad c) going through the slides, editing the slides outcome of last weeks discussion: approach A will be chosen short overview of the "dropped" approaches Morris: questions: what do ARC people think about? Balazs: two operations mandatory? Morris: discuss now Balazs: requirement part in createActivity operation you should put a limit in your server for vector operations sees no problem Morris: one step approach here you get everything back maybe broken job ids or job ids point to broken jobs, killed jobs etc... Balazs: will fail because not validated both are vector operations spec requires validation Morris: only steps 1,2,3 are mandatory Balazs: what is the initial stat the job gets to Etienne: createActivity -> validated substate before createActivity previous substate has to be created submitted validating submitted validated Morris: state model with different starting points? Etienne: initial state without name inside the submitted state action for Balazs: integrate discussed points in document before next meeting Etienne: notification can be only used if supported by SOAP Morris: keep this first out of PGI space Balazs: how can session directory be handled? Morris: back to the client we have two options A use of GLUE elements in ResourceInfo portType B query job properties in the Execution portType Balazs: and we have 0 SOAP Response contains it next meeting: discuss comments by S. Newhouse and B. Schuller these comments are integrated int he ppt Morris: very good conclusions Balazs will check this with spec before next meeting - highlight points to be discussed Etienne: title of document change? Balazs: Etienne will edit document with track changes on Etienne: move chapter 5 before chapter 2 Balazs: will be done -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0

Re: [Pgi-wg] Reminder & Agenda: PGI Telcon Today 2010-01-21; 4 pm CET
by Morris Riedel 21 Jan '10

21 Jan '10

All, the new version of the slides that should support our telcon today are now on GridForge: http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do… Nevertheless, I will edit on the fly during using the screen-sharing. @Etienne: I wasn't able to catch your e-mail answer since it was too long. I believe it would be good if you support the telcon today with your answers when we discuss the corresponding issues. Thanks. Take care, Morris ------------------------------------------------------------ Morris Riedel Jülich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel "We work to better ourselves, and the rest of humanity" Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender) >------Original Message----- >-From: pgi-wg-bounces(a)ogf.org [mailto:pgi-wg-bounces@ogf.org] On Behalf Of Morris Riedel >-Sent: Thursday, January 21, 2010 10:52 AM >-To: pgi-wg(a)ogf.org >-Cc: gin(a)ogf.org >-Subject: [Pgi-wg] Reminder & Agenda: PGI Telcon Today 2010-01-21; 4 pm CET >- >-Dear PGI team, >- >- first of all thanks Johannes for the doodle poll. >- >- >-As doodle decided we have our call already today at 4 pm CET: >- >- >-(1) >-Numbers: >- >-To join the PGI teleconference, please use this calling information: >- >-via Skype call +9900827049931906 (free of charge), ordinary phone numbers >-(local rates) with the 9931906 conference number: >- >-Austria 0820 401 15470 >-Belgium 0703 57 134 >-France 0826 109 071 >-Germany +49 (0) 180 500 9527 >-Switzerland 0848 560 397 >- >- >- >-(2) >-Screen-Sharing: >- >-http://webconf.vc.dfn.de/ogf-pgi-wg/ >- >-No login is required. Just enter as 'guest' with your real world name. >- >- >- >-(3) >-Proposed agenda: >- >-(a) >-Debrief minutes of last PGI TelCon >- >-(b) >-Debrief PGI actions >- >-(c) >-Very short: continue discussion animation ppt of PGI operation & state model >-Re: Developer experience says MUST might be tough >- >-(d) >-Debrief Steven Newhouse comments on the list >- >-(e) >-Debrief Bernd Schuller comments on the list >- >-(f) >-Debrief Etienne Comments on the list >- >-(g) >-Next meeting >- >-(h) >-AOB >- >-Please feel free to add other items on the agenda as you like. >- >-A high participation from the PGI and GIN members would be appreciated. >- >-Your co-chair, >-Morris >- >------------------------------------------------------------- >-Morris Riedel >-Jülich Supercomputing Centre (JSC) >-More: http://www.fz-juelich.de/jsc/JSCPeople/riedel >- >-"We work to better ourselves, and the rest of humanity" >- >-Sitz der Gesellschaft: Jülich >-Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 >-Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe >-Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), >-Dr. Ulrich Krafft (stellv. Vorsitzender)

1 0

Update on PGI Action (Move spec to drafts)
by Riedel, Morris 21 Jan '10

21 Jan '10

Hi team, here another update on one action item: >- A - action (Morris): copy strawman rendering doc to drafts Done. The actual draft specification (version 0.37) is now accessible in GridForge via: Doucments - Root Folder - Drafts - Specification Direct link: http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do… So far, I kept a copy within the Input Document Section since I propose a renaming of the document within drafts to: "Draft PGI Specification" It is neither a strawman document nor a concrete rendering - it's just a draft specification from PGI. However, of course, I seek to discuss this tomorrow in the telcon with all of you. Your co-chair, Morris ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------

5 9

Re: [Pgi-wg] Reminder & Agenda: PGI Telcon Today 2010-01-21; 4 pm CET
by Morris Riedel 21 Jan '10

21 Jan '10

All, looking at the comments so far - I guess we should also put an additional agenda item such as: ToDo (All): Define a set of specification strategy: A Improvements (JSDL, OGSA-BES, etc.) B New Specs (Execution Environment Spec, Delegation Spec, Information Spec, etc.) C PGI Profiles (joint sets of them) I doubt that we have time today for this, but hopefully in one of the next telcons... Take care, Morris >------Original Message----- >-From: Morris Riedel [mailto:m.riedel@fz-juelich.de] >-Sent: Thursday, January 21, 2010 10:52 AM >-To: 'pgi-wg(a)ogf.org' >-Cc: 'gin(a)ogf.org' >-Subject: Reminder & Agenda: PGI Telcon Today 2010-01-21; 4 pm CET >- >-Dear PGI team, >- >- first of all thanks Johannes for the doodle poll. >- >- >-As doodle decided we have our call already today at 4 pm CET: >- >- >-(1) >-Numbers: >- >-To join the PGI teleconference, please use this calling information: >- >-via Skype call +9900827049931906 (free of charge), ordinary phone numbers >-(local rates) with the 9931906 conference number: >- >-Austria 0820 401 15470 >-Belgium 0703 57 134 >-France 0826 109 071 >-Germany +49 (0) 180 500 9527 >-Switzerland 0848 560 397 >- >- >- >-(2) >-Screen-Sharing: >- >-http://webconf.vc.dfn.de/ogf-pgi-wg/ >- >-No login is required. Just enter as 'guest' with your real world name. >- >- >- >-(3) >-Proposed agenda: >- >-(a) >-Debrief minutes of last PGI TelCon >- >-(b) >-Debrief PGI actions >- >-(c) >-Very short: continue discussion animation ppt of PGI operation & state model >-Re: Developer experience says MUST might be tough >- >-(d) >-Debrief Steven Newhouse comments on the list >- >-(e) >-Debrief Bernd Schuller comments on the list >- >-(f) >-Debrief Etienne Comments on the list >- >-(g) >-Next meeting >- >-(h) >-AOB >- >-Please feel free to add other items on the agenda as you like. >- >-A high participation from the PGI and GIN members would be appreciated. >- >-Your co-chair, >-Morris >- >------------------------------------------------------------- >-Morris Riedel >-Jülich Supercomputing Centre (JSC) >-More: http://www.fz-juelich.de/jsc/JSCPeople/riedel >- >-"We work to better ourselves, and the rest of humanity" >- >-Sitz der Gesellschaft: Jülich >-Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 >-Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe >-Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), >-Dr. Ulrich Krafft (stellv. Vorsitzender)

1 0

Reminder & Agenda: PGI Telcon Today 2010-01-21; 4 pm CET
by Morris Riedel 21 Jan '10

21 Jan '10

Dear PGI team, first of all thanks Johannes for the doodle poll. As doodle decided we have our call already today at 4 pm CET: (1) Numbers: To join the PGI teleconference, please use this calling information: via Skype call +9900827049931906 (free of charge), ordinary phone numbers (local rates) with the 9931906 conference number: Austria 0820 401 15470 Belgium 0703 57 134 France 0826 109 071 Germany +49 (0) 180 500 9527 Switzerland 0848 560 397 (2) Screen-Sharing: http://webconf.vc.dfn.de/ogf-pgi-wg/ No login is required. Just enter as 'guest' with your real world name. (3) Proposed agenda: (a) Debrief minutes of last PGI TelCon (b) Debrief PGI actions (c) Very short: continue discussion animation ppt of PGI operation & state model Re: Developer experience says MUST might be tough (d) Debrief Steven Newhouse comments on the list (e) Debrief Bernd Schuller comments on the list (f) Debrief Etienne Comments on the list (g) Next meeting (h) AOB Please feel free to add other items on the agenda as you like. A high participation from the PGI and GIN members would be appreciated. Your co-chair, Morris ------------------------------------------------------------ Morris Riedel Jülich Supercomputing Centre (JSC) More: http://www.fz-juelich.de/jsc/JSCPeople/riedel "We work to better ourselves, and the rest of humanity" Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender)

1 0

New day for PGI call: Thursday
by Johannes Watzl 20 Jan '10

20 Jan '10

Dear all, looking at the doodle, Thursday seems to be the most appropriate day for our weekly calls. All upcoming calls will be held on Thursdays, 16:00 (CET). The call this week will take place on Thursday, 21 Jan 2010, 16:00 (CET). Best, Johannes -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0

action list
by Johannes Watzl 18 Jan '10

18 Jan '10

----------------- New actions ----------------- action (Emmanouil: GIN list set of DEMOS and call for more demos action (Emmanouil): get in touch with Ilia action (Morris): ask Nils about the date of the interoperability day ----------------- Previous actions ----------------- COMPLETED action (Morris): session proposals -> call for sessions As we discussed the 3 PGI working sessions should be part of the interoperability day that was already agreed with the local chair man (Nils) and therefore I requested no official sessions yet. However, I contacted Nils to clarify things. ----------------- COMPLETED action (Morris): should arrange that someone from gLite is there ----------------- COMPLETED action (Morris): check if there is a BES session, discuss with Andrew ----------------- action (Aleksander): security aspects for credentials in data staging will be done by Aleksander ----------------- COMPLETED action (Morris): copy strawman rendering doc (latest version from Aleksander) to drafts The actual draft specification (version 0.37) is now accessible in GridForge via: Doucments - Root Folder - Drafts - Specification Direct link: http://forge.gridforum.org/sf/docman/do/downloadDocument/projects.pgi-wg/do… So far, I kept a copy within the Input Document Section since I propose a renaming of the document within drafts to: "Draft PGI Specification" It is neither a strawman document nor a concrete rendering - it's just a draft specification from PGI. ----------------- COMPLETED action (Morris, all): strengthen gLite participation ----------------- -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0

PGI call notes, 2010-01-15
by Johannes Watzl 18 Jan '10

18 Jan '10

Dear all, please see below the notes from our last meeting, Friday, 15 Jan 2010. Best, Johannes Meeting on 2010-01-15, 16:00 (CET) Notes Participants: Emmanouil, Etienne, Steven, Morris, Luigi, Johannes (i) Debrief minutes of last PGI TelCon nail down validation steps in ppt have to write it down and agree on it (ii) Debrief PGI actions Action: Johannes resend the action list (iii) Discuss regular change of telcon timeslot in 2010 instead of Friday afternoon action for Johannes: set up a doodle (iv) Continue discussion animation ppt of PGI operation & state model Focus: Waiting for a job-id until all validation steps have been performed is unrealistic from implementation view (e.g. 100 jobs in submit array) first spec: start with scientific gateway, no job id, no state create JSDL call createActivity shpping JSDL first state submitted maybe we should have valaidation process before most implementation logic parsing of JSDL going through XML, compliance checks GLUE link, optional validation check JSDL seems to be fine -> get job id new state submitted validated ->SOAP response when is submitted hold happening Etienne: submitted outgoing does not mean responding but sending the job to compute facility Morris: maybe change Etienne: because standard name Morris: we have SOAP response PGI benefit> provide job id and staging info staging info might not be there certainly take some time job id gets crucial if we have really an array and submit job with 100 JSDLs not really realistic create activity = 1 Next slide: lot of points but not a full list crucial points questions? no implementation issue if lot of JSDLs three identified approaches A. two operations in port type we really give the different activities a different symantics gives us a job id may take a long time second operation typical array, but does not guarantee next iteration: query unvalidated job ids B. one operation in port type give back job id before validating C. one operation in port type specify as part of JSDL problem related to job directory problem session directory: could be provided but must not be provided query BES container and get a complete list Steven: combinations of approaches WSRM provides a way ensuring getting message from one ep to another Etienne: strongly opposed to C you cannot guarantee that a client is unique enforced at the server level Morris: we should drop Option C. Etienne: yes Steven: client could provide an alias server could create a string which is unique for the server Etienne: server cannot reasonably believe that id is unique Morris: one server could be in workflow should not be allowed to create uuid -> drop C Etienne: B is not acceptable by ARC they want early staging poll for the job status and get session directory afterwards Morris: UNICORE same feature request Etienne: Strongly in favor of option A Morris: Approach A. Etienne: must have slightly differnet name Morris: createActivity createValidatedActivity Steven: many times this was discussed wanted to do we returned job ids without any expectation of validation createValidatedActivity client takes the risk of validating ppt edited - approach A Morris: anything missed? no metting close and need ARC guys for (v) Debrief Steven Newhouse comments on the list (if time) (vi) Debrief Bernd Schuller comments on the list (if time) (vii) Interop demonstration Setups for OGF28 (viii) Next meeting (ix) AOB Etienne: developing a BES client to be able to interoperate with ARC will ask where we are on this subject Emmanouil: monitoring service DMON Ilya Saverchenko could present this Action for Emanouel: GIN list et of DEMOS cll for more demos Action for Emanouel: get in touch with Ilia Next week - if not agreed earlier on the doodle - Friday -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0

PGI call notes 2009-12-18
by Johannes Watzl 18 Jan '10

18 Jan '10

Dear all, please see below the revised meeting notes from 18 Dec 2009. Best, Johannes Participants: Luigi, Balazs, Balazs, Morris, Aleksander, Etienne, Emmanouil Paisos, Oxana Emmanouil Paisos short introduction: working at LRZ in the group of Helmut Heller Globus, Grid services, ... debrief the minutes (updated version by Etienne) state model from Etienne - agreed submitted state - incoming, validated? Validation aspects preprocessing -> next year Luigi: please explain why we need the hold state? Etienne: Why do we need to add the hold substate? Inside the submitted state the ARC middleware can send back not only id but also staging information Balazs: in the draft the submitted hold is explained Luigi: If I put the job state in hold, I have to unlock to assume. Morris: Only if a hold point is set in AGU-JSDL document Balazs: for manual changes Morris: next point in the agenda go through action items some administrative things agreed on changing the title of the strawman document to PGI Execution Service Draft Specification filename: PGI_ExecutionService_Draft.doc joint session at OGF28? session request for interoperability day 3 PGI, 2 GIN open floor for presentations action tracking on the mailing list Luigi: discussion about the job lease in CREAM awaiting more email answers for discussion Morris: please resend OGF28 Morris: is there from gLite at OGF28? Luigi: can participate Morris: someone from ARC? Balazs: yes the interoperability day is this also at the OGF? Morris: at OGF action for Morris: ask Nils about the date of the interoperability day discuss the submitted state going through the animation "step-wise animation submitted" Balazs: only if the JSDL is correct, a jobid is created Morris: if JSDL description is wrong, then there is no job everybody agrees? -> yes Etienne: would like at this moment there is an error from the execution service to the scientific gateway Balazs: how long does it take to go through the createActivity operation? Morris: XML evaluation is very fast can throw back a fault as a SOAP response Balazs: vector operation? Etienne: some statistics about jobs sent 5 to 10 seconds for a job to be accepted by the execution service 1 min before the brokering Luigi: we can provide the same interface to the WMS Morris: can we do that? Etienne: the job is in the accepted status with its job id which is matched to the computing element Morris: should we/can we do a brokering interface here Etienne: brokering should be possible but is out of scope match requirements described in the JSDL to a computing element Balazs: we should try to keep this interface open enough that a broker is possible Morris: early enough? could be a long time thinking of 2-3 sec/JSDL Balazs: job id and data staging information Oxana: why worry about how long it takes? Morris: the only appearence after the validation steps -> chance to respond something adding responses Etienne: if we have to wait for an hour to get the job ids for 100 jobs we have to accept that or we have to reduce the number of JSDLs sent together or use completely other method (parameter sweep?) if there are a lot of jobs we have to wait long Luigi: if http connection breaks, SOAP breaks Morris: produces ghost JSDLs Aleksander: the service will detect, that the respose was not submitted to the client Etienne: this has a great impact - should take care of it Aleksander: responses are sent after JSDL processing no cancelling involved Morris: question if we can have here a job id creation before the validation steps Aleksander: job ide could be assigned by the client Luigi: no because it cannot be unique -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0

PGI call day - voting
by Johannes Watzl 17 Jan '10

17 Jan '10

Dear all, as discussed in our last meeting we want to shift the PGI call from Friday to another day. Please vote here: http://www.doodle.com/sdqsfaffpq4huuvt Best, Johannes -- _ _ _ _ _ _ Johannes Watzl |\/| |\ | |\/| Institut für Informatik / Dept. of CS | | | \| | | Ludwig-Maximilians-Universität München ======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany Room E 005, Phone +49-89-2180-9162 Munich Network Management Team Email: watzl(a)nm.ifi.lmu.de Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl

1 0