Re: [Pgi-wg] EPR issue and approach using WS-Enumeration
This problem turned up early in RNS. We looked into WS-Enum and made a different choice (along with RNS). I am attaching a document that describes why we chose to go with a statefull iterator rather than enumeration,as well as all of the related documents. A
-----Original Message----- From: pgi-wg-bounces@ogf.org [mailto:pgi-wg-bounces@ogf.org] On Behalf Of Morris Riedel Sent: Wednesday, April 08, 2009 11:32 AM To: pgi-wg@ogf.org Subject: [Pgi-wg] EPR issue and approach using WS-Enumeration
Hi PGI team,
is there any experience by the members of using WS-Enumeration for tackling with the emerging 'EPR pile' in the endpoint? (WS-Enumeration is like separate google hits pages)
I took a similar approach with Usage Records in OGF RUS a while ago since this problem is nearly the same as ours - having scalability problems.
I think this is a problem where each middleware has a limit currently, looking especially on emerging many task computing jobs (running rather short - but many (potentially different) of them).
(1) Submitting 100.000 jobs and two fail. Both have to have a JSDL for resubmit - but this means you have typically 100.000 instances/EPRs tracked in the container...
(2) Submitting 100.000 jobs and two fail using bulk submission or something like that with only one JSDL - you have difficulties to get the job description of these two.
Maybe there is already someone having experience with WS-Enumeration in this context here in the PGI space?
Thanks, Morris
------------------------------------------------------------ Morris Riedel SW - Engineer Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Forschungszentrum Juelich Wilhelm-Johnen-Str. 1 D - 52425 Juelich Germany
Email: m.riedel@fz-juelich.de Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel Phone: +49 2461 61 - 3651 Fax: +49 2461 61 - 6656
Skype: MorrisRiedel
"We work to better ourselves, and the rest of humanity"
Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender)
------Original Message----- -From: Andrew Grimshaw [mailto:grimshaw@virginia.edu] -Sent: Wednesday, April 08, 2009 6:13 PM -To: 'Morris Riedel'; pgi-wg@ogf.org -Subject: RE: [Pgi-wg] EPR issue and approach using WS-Enumeration - -This problem turned up early in RNS. We looked into WS-Enum and made a -different choice (along with RNS). I am attaching a document that describes why we -chose to go with a statefull iterator rather than enumeration,as well as all of the -related documents. -A - -> -----Original Message----- -> From: pgi-wg-bounces@ogf.org [mailto:pgi-wg-bounces@ogf.org] On Behalf -> Of Morris Riedel -> Sent: Wednesday, April 08, 2009 11:32 AM -> To: pgi-wg@ogf.org -> Subject: [Pgi-wg] EPR issue and approach using WS-Enumeration -> -> Hi PGI team, -> -> is there any experience by the members of using WS-Enumeration for -> tackling with the emerging 'EPR pile' in the endpoint? -> (WS-Enumeration is like separate google hits pages) -> -> I took a similar approach with Usage Records in OGF RUS a while ago -> since this problem is nearly the same as ours - having scalability
Thanks - I will have a look... ------------------------------------------------------------ Morris Riedel SW - Engineer Distributed Systems and Grid Computing Division Jülich Supercomputing Centre (JSC) Forschungszentrum Juelich Wilhelm-Johnen-Str. 1 D - 52425 Juelich Germany Email: m.riedel@fz-juelich.de Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel Phone: +49 2461 61 - 3651 Fax: +49 2461 61 - 6656 Skype: MorrisRiedel "We work to better ourselves, and the rest of humanity" Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender) problems.
-> -> -> I think this is a problem where each middleware has a limit currently, -> looking especially on emerging many task computing jobs (running -> rather short - but many (potentially different) of them). -> -> (1) -> Submitting 100.000 jobs and two fail. Both have to have a JSDL for -> resubmit -> - but this means you have typically 100.000 instances/EPRs tracked in -> the container... -> -> (2) -> Submitting 100.000 jobs and two fail using bulk submission or -> something like that with only one JSDL - you have difficulties to get -> the job description of these two. -> -> -> Maybe there is already someone having experience with WS-Enumeration -> in this context here in the PGI space? -> -> Thanks, -> Morris -> -> -> -> ------------------------------------------------------------ -> Morris Riedel -> SW - Engineer -> Distributed Systems and Grid Computing Division Jülich Supercomputing -> Centre (JSC) Forschungszentrum Juelich Wilhelm-Johnen-Str. 1 D - 52425 -> Juelich Germany -> -> Email: m.riedel@fz-juelich.de -> Info: http://www.fz-juelich.de/jsc/JSCPeople/riedel -> Phone: +49 2461 61 - 3651 -> Fax: +49 2461 61 - 6656 -> -> Skype: MorrisRiedel -> -> "We work to better ourselves, and the rest of humanity" -> -> Sitz der Gesellschaft: Jülich -> Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 -> Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe -> Vorstand: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft -> (stellv. Vorsitzender) ->
participants (2)
-
Andrew Grimshaw -
Morris Riedel