- gridrpc-wg - lists.ogf.org

DIET 2.10 is available
by Eddy Caron 29 Nov '17

29 Nov '17

Dear all, DIET ‘SC17' Release 2.10 is now available on the new DIET Web Site: http://graal.ens-lyon.fr/diet/?page_id=778 1. What's major improvements in DIET ‘SC17’ 2.10 ? ----------------------------------------------------------------------- Update workflow unit tests to take the results of the execution into account Workflow engine improvement Upgrade to support Cmake 3.3 and later Optional config file for dietForwarder Improve the GRPC support Cmake compilation improvement which prevented compilation with recent GCC versions Upgrade for Fedora packaging Update the DIET forwarder documentation Enjoy Eddy ---------------------------------------------------------------------------------------------- Eddy Caron. Mcf HDR ENS Lyon Inria CNRS UCBL ENS Lyon - LIP - Projet Avalon Chargé de missions LIP: Projets / Valorisation / International E-Mail : Eddy.Caron(a)ens-lyon.fr [ Tel : 04.72.72.86.83 ][ Web page : http://avalon.ens-lyon.fr/~ecaron ] ------------------------------------------------------------------------------------------------

1 0

DIET 2.9 is available
by Eddy Caron 28 May '15

28 May '15

Dear all, DIET ‘S4C' Release 2.9 is now available on the new DIET Web Site: http://graal.ens-lyon.fr/diet/?page_id=349 1. What's major improvements in DIET ‘S4C' 2.9 ? ----------------------------------------------------------------------- Security: SSL communications are now available. Cloud: some improvements Clients can automatically retry a diet_call certain number of times More information are provided by SeD to the LogService Agents and SeDs now listen on each IP address (instead of the first OmniORB finds) OAR binding improvement Some bugs and warning are fixed Compilation improvement QuickStart documentation is provided Upgrade of some parts of the documentation Enjoy Eddy ---------------------------------------------------------------------------------------------- Eddy Caron. Mcf HDR ENS Lyon Inria CNRS UCBL ENS Lyon - LIP - Projet Avalon 46 Allee d'Italie, 69364 Lyon Cedex 07, France E-Mail : Eddy.Caron(a)ens-lyon.fr [ Tel : 04.72.72.85.47 ][ Web page : http://graal.ens-lyon.fr/~ecaron ] ------------------------------------------------------------------------------------------------

1 0

Library for the GridRPC _and_ GridRPC Data Management standards available
by Yves Caniou 22 Oct '13

22 Oct '13

Dear All It has been a while that the mailing list has not been used, but there was work progressing underneath : Some time ago, we did a prototype for the GridRPC Data Management API, that we mostly used to check if things were missing to the API relying on practical cases. I'm glad to inform you that a new, very modular and already working library has been made available, which provides both the GridRPC _and_ most of the GridRPC Data Management API. This new implementation has already many modules to manage different communication protocols (hopefully more to come); it also allows a single GridRPC client program to call more than one Grid middleware for services and much more. Additionally everything has been made to ease module development (no name conflicts: If the Grid middleware is already GridRPC compliant, the module is more or less empty, if not, it is thus really possible to make it compliant). The different modules available are - Grid middleware modules for DIET and Ninf, and - Data Manager modules for Dagda, iRods, Owncloud, Rsync, Scp. The library uses cmake and can be configured and installed with the provided generator.sh script. Please check the install/testing/ repository from where you can launch a complete local deployment/execution/shutdown of DIET and Ninf by just calling the as-is furnished script. Please download, contribute to code or send your returns: https://sourceforge.net/projects/gridrpcdm/ Regards. .Y -------- Associate Professor at Université de Lyon Délégation CNRS at JFLI, University of Tokyo http://graal.ens-lyon.fr/~ycaniou

1 0

OGF36 presentation
by Yves Caniou 31 Oct '12

31 Oct '12

Dear all, I put the presentation online on the redmine forge of the OGF. You can access it here: https://redmine.ogf.org/projects/gridrpc-wg/documents Regards. .Y -- Yves Caniou Associate Professor at Université Lyon 1, Member of the team project INRIA GRAAL in the LIP ENS-Lyon, http://graal.ens-lyon.fr/~ycaniou/

2 2

Slides and GridRPC data management library
by Yves Caniou 30 Oct '12

30 Oct '12

Dear All, Message from Gaël who got some issues posting on the mailing list: I updated the library on the git repository. It is still experimental but it should be ready for experiments in a few weeks. There is still work to do on the transfers/data management modules, and on the documentation. Any help and comments appreciated. Besides, the DIET module works fine even there are still some memory leaks that should be fixed quickly. The URL for the library is https://forge.mis.u-picardie.fr/projects/gridrpcdm/ Regards, Gaël. -- Yves Caniou Associate Professor at Université Lyon 1, Member of the team project INRIA GRAAL in the LIP ENS-Lyon, http://graal.ens-lyon.fr/~ycaniou/

1 0

OGF36 -- GridRPC Data Management
by Yves Caniou 20 Sep '12

20 Sep '12

Dear All, We'll present the current work on the Data Management in the GridRPC, mainly the availibility of the library implemented the standard and its performance. Here is a sum-up of the talk. Please do not hesitate to come with use cases so that we can talk about how this work can be integrated, and the benefits that it may lead to. ***************************** Title: Transparent collaboration of GridRPC middleware using the OGF standardized GridRPC data management API In september 2011, the Open Grid Forum standardized the document "Data Management API within the GridRPC" which discribes an optional API that extends the GridRPC standard. Used in a GridRPC middleware, it provides a minimal set of functions to handle a large set of data operations among which: movements, replications, migrations, data prefetch and persistency. We'll present a library implementing the API that has been integrated in two different middleware, respectively DIET and NINF. We have conducted several experiments, showing very high benefits that a Grid user can expect 1) in terms of resource usage compared to the current GridRPC context since useless transfers are avoided; 2) in terms of reducing the completion time of an application to obtain results the soonest (data can be prefetched and replicated, hence letting calculus to be submitted really soon in a workflow analysis in addition to the possible overlap between computations and communications); 3) in terms of code portability, since we show with these examples that at last the same GridRPC code can be compiled and executed within two different GridRPC middleware which implements the GridRPC data management API; 4) finally we thus obtain middleware interoperability without any explicit glue as generally done: we show as a proof of concept that resources dispatched across different administrative domains can be used altogether without the underlying distributed data management systems having any knowledge of the workflow and/or computing resources: computational servers of DIET and NINF transparently collaborate to the same calculus by sharing GridRPC data! Last results has been published in [1,2] in which we explained the API and described the improvements using the implementation of this standard. Further work will go on the transparent management of protocols such as GridFTP, iRods, torrent, and the use of catalog for data, as well on security. [1] Yves Caniou, Eddy Caron, Gaël Le Mahec, and Hidemoto Nakada. Standardized Data Management in GridRPC Environments. In 6th International Conference on Computer Sciences and Convergence Information Technology, Jeju Island, Korea, Nov. 29 - Dec. 1 2011. [2] Yves Caniou, Eddy Caron, Gaël Le Mahec, and Hidemoto Nakada. Transparent Collaboration of GridRPC Middleware using the OGF Standardized GridRPC Data Management API. In The International Symposium on Grids and Clouds (ISGC), February 26 - March 2 2012. Proceedings of Science. -- Yves Caniou Associate Professor at Université Lyon 1, Member of the team project INRIA GRAAL in the LIP ENS-Lyon, http://graal.ens-lyon.fr/~ycaniou/

1 0

DIET 2.8.1 is available
by Eddy Caron 12 Jul '12

12 Jul '12

Dear all, The new release of DIET 2.8.1 is available DIET 'klair' Release 2.8.1 and tools are now available on the DIET Web Site: http://graal.ens-lyon.fr/DIET/download_DIET and in your favorite distribution (Debian or Fedora). 1. What's new in DIET 'klair' 2.8.1 ? ------------------------------------------------ * Features: - Windows port initial work - Number of simultaneous connexion is increased - Improve the automatic omniORB configuration - Improve clients connexion * Source code organization: - Removed dead code - A bit of code cleanup & indentation - Removed references to ackfile - Removed some compilation warnings - replace snprintf by strncpy where appropriate - Improved build system * List of available packages - Debian 'wheezy' - Fedora 16 - Fedora 17 * Bug Fix: - Fixed compilation on Mac OS X - Fixed tests binary locations - Fixed ORBMgr bug: bad initialization of ORBMgr::TheMgr in libDIET_SeD - Fixed a bug with the initialization of ORBMgr: CorbaCommon_SRCS was included in both the sources for DIET_core and DIET_SeD, thus leading to having two ORBs - Fixed compilation on AIX - Fixed a bug when no configuration file was found: boost::lexical_cast<unsigned int> throws an exception when it receives an empty string. - Fixed: unlock without lock in DIET_client.cc - Fixed: double free in diet_wait_any - Fixed: double free in diet_wait_or - replace getpid() by uuid to prevent multiple elements to have the same name - Fixed: concurrency issue in DAGDA - Fixed: DAGDA storage directory creation failure - Forwarders now unregister themselves from omniNames when they are shut down properly. - Forwarders can be launched after any DIET element - To avoid problems when a file managed by DAGDA is modified outside DAGDA, we know compute the files sizes before any transfer. - Let CMake manage rpath. Thus, you need to explicitly set your LD_LIBRARY_PATH or DYLD_LIBRARY_PATH variable. Enjoy Eddy ---------------------------------------------------------------------------------------------- Eddy Caron. Mcf HDR ENS Lyon ENS Lyon - LIP - Projet Avalon 46 Allee d'Italie, 69364 Lyon Cedex 07, France E-Mail : Eddy.Caron(a)ens-lyon.fr [ Tel : 04.72.72.80.04 ][ Web page : http://graal.ens-lyon.fr/~ecaron ] ------------------------------------------------------------------------------------------------

1 0

Webminar WG GridRPC
by Eddy Caron 13 Mar '12

13 Mar '12

Dear all, Today we will have the GridRPC Working Group session. We will try to offer a webcast of this session through the support of the SysFera company. I hope that the wireless will be enough to do that. Cross fingers. Join us. Hello , SysFera Company invites you to attend this online meeting. Topic: GridRPC Working Group (OGF'34) Date: Tuesday 13 March 2012 Time: 11:00, GMT Time (London, GMT) Meeting Number: 958 033 569 Meeting Password: GRPC ------------------------------------------------------- To join the online meeting (Now from mobile devices!) ------------------------------------------------------- 1. Go to https://sysfera.webex.com/sysfera/j.php?ED=5690967&UID=29612917&PW=NOWEwODY… 2. If requested, enter your name and email address. 3. If a password is required, enter the meeting password: GRPC 4. Click "Join". To view in other time zones or languages, please click the link: https://sysfera.webex.com/sysfera/j.php?ED=5690967&UID=29612917&PW=NOWEwODY… ------------------------------------------------------- For assistance ------------------------------------------------------- 1. Go to https://sysfera.webex.com/sysfera/mc 2. On the left navigation bar, click "Support". You can contact me at: meetings(a)sysfera.com To add this meeting to your calendar program (for example Microsoft Outlook), click this link: https://sysfera.webex.com/sysfera/j.php?ED=5690967&UID=29612917&ICS=MI&LD=1… The playback of UCF (Universal Communications Format) rich media files requires appropriate players. To view this type of rich media files in the meeting, please check whether you have the players installed on your computer by going to https://sysfera.webex.com/sysfera/systemdiagnosis.php. Sign up for a free trial of WebEx http://www.webex.com/go/mcemfreetrial http://www.webex.com IMPORTANT NOTICE: This WebEx service includes a feature that allows audio and any documents and other materials exchanged or viewed during the session to be recorded. By joining this session, you automatically consent to such recordings. If you do not consent to the recording, discuss your concerns with the meeting host prior to the start of the recording or do not join the session. Please note that any such recordings may be subject to discovery in the event of litigation. Best regards, Eddy ---------------------------------------------------------------------------------------------- Eddy Caron. Mcf HDR ENS Lyon ENS Lyon - LIP - Projet GRAAL/Avalon 46 Allee d'Italie, 69364 Lyon Cedex 07, France E-Mail : Eddy.Caron(a)ens-lyon.fr [ Tel : 04.72.72.80.04 ][ Web page : http://graal.ens-lyon.fr/~ecaron ] ------------------------------------------------------------------------------------------------

1 0

Fwd: OGF applications group page
by Hidemoto Nakada 23 Feb '12

23 Feb '12

Joel and Guy, thank you for your hard work for OGF34. we (GridRPC WG) thought that we requested GridRPC group session on Tuesday by the attached e-mail, but it did not appear in the schedule. Could you please add it on the schedule? Fortunately, Tuesday morning, which is convenient for us, seminar room seems to be available. thank you in advance. -- HIDEMOTO NAKADA, GridRPC Working Group. ---------- Forwarded message ---------- From: Eddy Caron <Eddy.Caron(a)ens-lyon.fr> Date: Wed, Jan 25, 2012 at 6:27 AM Subject: Re: OGF applications group page To: "Daniel S. Katz" <dsk(a)ci.uchicago.edu> Cc: Peter Tröger <peter(a)troeger.eu>, Andre Merzky <andre(a)merzky.net>, Wolfgang Ziegler <wolfgang.ziegler(a)scai.fraunhofer.de>, Joel Replogle <replogle(a)ogf.org>, Hidemoto Nakada <hide-nakada(a)aist.go.jp> Dear Daniel and Joel, The current text of the Grid-RPC is outdated. I will sent a some upgrade as soon as possible. About the next session it could be fruitful to give more information. Is it possible to have a session on Tuesday ? Text for the next Grid-RPC session : ------------------------------------------------ Join the next Grid-RPC session and come with your data management use case ... In september 2011, the Open Grid Forum standardized the document "Data Management API within the GridRPC" [1] which discribes an optional API that extends the GridRPC standard [2]. Used in a GridRPC middleware, it provides a minimal set of functions to handle a large set of data operations among which: movements, replications, migrations, data prefetch and persistency. After a few remarks on the GridRPC DM API, we'll present a basic implementation that we have integrated in two different middleware, respectively DIET [3] and NINF [4]. We have conducted several experiments, showing very high benefits that a Grid user can expect 1) in terms of resource usage compared to the current GridRPC context since useless transfers are avoided; 2) in terms of reducing the completion time of an application to obtain results the soonest (data can be prefetched and replicated, hence letting calculus to be submitted really soon in a workflow analysis in addition to the possible overlap between computations and communications); 3) in terms of code portability, since we show with these examples that at last the same GridRPC code can be compiled and executed within two different GridRPC middleware which implements the GridRPC data management API; 4) finally we thus obtain middleware interoperability without any explicit glue as generally done like in [5]: we show as a proof of concept that resources dispatched across different administrative domains can be used altogether without the underlying distributed data management systems having any knowledge of the workflow and/or computing resources: computational servers of DIET and NINF transparently collaborate to the same calculus by sharing GridRPC data! [1] Y. Caniou, E. Caron, G. Le Mahec and Hidemoto Nakada, "Data Management API within the GridRPC", June 2011, Open Grid Forum standard GFD-R-P.186. [2] H. Nakada, S. Matsuoka, K. Seymour, J.J. Dongarra, C. Lee and H. Casanova, "A GridRPC Model and API for End-User Applications", June 2007, Open Grid Forum standard GFD-R.052. [3] E. Caron and F. Desprez, "DIET: A Scalable Toolbox to Build Network Enabled Servers on the Grid", International Journal of High Performance Computing Applications, 2006. [4] Y. Tanaka, H. Nakada, S. Sekiguchi, T. Suzumura and S. Matsuoka, "Ninf-G: A Reference Implementation of {RPC}-based Programming Middleware for Grid Computing", Journal of Grid Computing, 2003. [5] Y. Suzuki, N. Kushida, N. Teshima, K. Nakajima, A. Nishida and N. Nakajima, "Interoperation between Atomic Energy Grid Infrastructure (AEGIS) and Other Grids", in High Performance Computing on Vector Systems 2008, Chap. 3, Roller, S.; Benkert, K.; Galle, M.; Bez, W.; Kobayashi, H.; Hirayama, T. (Eds.), 2009. Best Regards, ---------------------------------------------------------------------------------------------- Eddy Caron. Mcf HDR ENS Lyon ENS Lyon - LIP - Projet GRAAL/Avalon 46 Allee d'Italie, 69364 Lyon Cedex 07, France E-Mail : Eddy.Caron(a)ens-lyon.fr [ Tel : 04.72.72.80.04 ][ Web page : http://graal.ens-lyon.fr/~ecaron ] ------------------------------------------------------------------------------------------------ Le 24 janv. 2012 à 17:52, Daniel S. Katz a écrit : Hi Peter, Eddy, and Andre, Can you take a quick look at http://www.ogf.org/gf/group_info/areasgroups.php?area_id=5 and see what is out of date and what needs to be changed? Please email Joel the changes that should be made, and copy Wolfgang and me. Thanks, Dan -- Daniel S. Katz University of Chicago (773) 834-7186 (voice) (773) 834-6818 (fax) d.katz(a)ieee.org or dsk(a)ci.uchicago.edu http://www.ci.uchicago.edu/~dsk/

1 1

OGF applications group page
by Eddy Caron 25 Jan '12

25 Jan '12

Dear all, The OGF members asked the update of the outdated information on this page http://www.ogf.org/gf/group_info/areasgroups.php?area_id=5 old version: Group Chair(s): Craig Lee, Hidemoto Nakada, Keith Seymour Group Secretary(s): Yoshio Tanaka, Yusuke Tanimura Group Email: gridrpc-wg(a)ogf.org (Subscribe) (Archive by date) new version (based on the current activity) Group Chair(s): Eddy Caron, Hidemoto Nakada Group Secretary(s): Yves Caniou, Yusuke Tanimura Group Email: gridrpc-wg(a)ogf.org (Subscribe) (Archive by date) It could be great to have someone from GridSolve or more relevant from SmartGriRPC ? Anybody from UTK ? Then Keith are you ok to continue to be invilved in the Group Chair ? Yoshio Tanaka is ok to continue to be involved in the Group Secretary ? Yves are you ok to become an official member of the Group Secretary, because you did that unofficially since a long time The Group Description and Group Focus and Scope must be updated too. Any inputs are welcome. I will add some information about the recent development around data management. Regards, Eddy ---------------------------------------------------------------------------------------------- Eddy Caron. Mcf HDR ENS Lyon ENS Lyon - LIP - Projet GRAAL/Avalon 46 Allee d'Italie, 69364 Lyon Cedex 07, France E-Mail : Eddy.Caron(a)ens-lyon.fr [ Tel : 04.72.72.80.04 ][ Web page : http://graal.ens-lyon.fr/~ecaron ] ------------------------------------------------------------------------------------------------

2 1