gridrpc-wg

Grid Remote Procedure Call WG

Applications

This Working Group will produce an OGF Recommendation for a grid-enabled, remote procedure call (RPC) mechanism. The first document entitled A GridRPC Model and API for End-User Applications has been completed and published as GFD-R.52. Currently we are concentrating on the second recommendation document, which defines GridRPC API for middleware developers that extends the model and GridRPC API for end-users, and also focuses on data management mechanism within the GridRPC model. The GridRPC middleware tools developed will further lower the barrier to acceptance for grid use by hiding the tremendous amount of infrastructure necessary to make grids work, while providing even higher-level abstractions for domain-specific middleware.

The top-level objective of defining a proposed middleware recommendation can be decomposed into (at least) the following specific objectives:

* Definition of a specific data structure to be used for GridRPC arguments in middleware.

* Definition of a "grpc_arg" data type to be used in conjunction with the argument data structure.

* Definition of the argument data structure creation, destruction, lifetime and copy semantics.

* Definition of mechanisms for handling persistent data, e.g., definition and use of a concept such as "data handles" (which might be the same as or similar to a grpc_arg data type). This may also involve concepts such as lazy copy semantics, and data leases or time-outs.

* Definition of API mechanisms to enable workflow management.

* Evaluate the compatibility and interoperability with other systems, e.g., WSRF.

* Desirable Properties -- the Proposed Recommendation will not necessarily specify any properties, such as thread safety, security, and fault tolerance, but it should not be incompatible with any such useful properties.

* Demonstrate implementability of all parts of the API.

* Demonstrate and evaluate at least two implementations of the complete GridRPC middleware recommendation.

We specifically exclude the following topics as non-objectives:

* Generalized Workflow -- the GridRPC WG does not wish to define or develop general workflow management tools, rather we will endeavor to make it possible for end-users and middleware developers to interact with workflow tools developed elsewhere through the GridRPC API.

* Services names -- the Middleware GridRPC Proposed Recommendation will not define the Naming Service for service discovery but will have requirements for its use.

* Wire protocol interoperability between implementations -- achieving full interoperability between implementations requires defining, in essence, a wire protocol. This is actually orthogonal to the problem of defining the user-level model and API and, hence, can be done later as part of a separate effort.

Since the proposed GridRPC Working Group has such a clear focus, the exit strategy is straightforward: producing a middleware Proposed Recommendation document. In the case that any of the technical issues identified above become too difficult to resolve within the allotted schedule, then the working group may have to re-charter to focus on that key issue. Even if the Proposed Recommendation document is produced on time, it is anticipated that a number of related issues will remain to be resolved that were considered to be outside of the scope of the proposed document, e.g., naming services. Hence, one or more "follow-on" working groups may be proposed at that time.

Title: A GridRPC Model and API for End-User Applications
Title: Data Management API within the GridRPC
Title: Interoperability Testing for The GridRPC API Specification

1. Is the scope of the proposed group sufficiently focused?
Yes. This proposed working group is focused specifically on producing an OGF proposed recommendation document for a GridRPC tool suitable for middleware development. Much work has already been done in at least four independent research groups around the world on the end-user model and API that is the starting point for this work. The specific issues to be resolved as part of this group work have already been identified in the GridRPC Working Groups efforts to date. The proposed working group is focused on resolving these specific issues and producing a second OGF Proposed Recommendation document.

2. Are the topics that the group plans to address clear and relevant for the Grid research, development, industrial, implementation, and/or application user community?
Yes. Since RPC is an established programming model, a GridRPC tool will provide a much lower "barrier to acceptance" for application groups that are already using an RPC-style. The fact that GridRPC will conveniently package so much underlying grid technology means that it is essential that this work be done within the context of GGF to properly leverage the infrastructure development and implementation work on which GridRPC will rely. A middleware API will enable systems to be built that embody even higher-levels of abstraction for domain-specific tools.

3. Will the formation of the group foster (consensus-based) work that would not be done otherwise?
Yes. Since there are four different research groups from three countries on three continents already pursuing this work, the formation of this group is essential to coordinate their activities, eliminate redundant work, and establish a common model and API that represents the best practices of the user community. (See "Base of Interested Consumers" below.)

4. Do the group's activities overlap inappropriately with those of another OGF group or to a group active in another organization such as IETF or W3C?
No. In fact, the proposed groups activities dovetail with other work being done in OGF and W3C. A GridRPC Proposed Recommendation will focus on the user-level API, i.e., a programming tool for humans. By contrast, the Open Grid Services Architecture (OGSA) and the Web Services Definition Language (WSDL) are focusing on the interface between program modules. Hence, from an implementation perspective, a GridRPC Proposed Recommendation could leverage the request-response style that is possible in WSDL and OGSA to something that is more suitable for routine human use. With regards to workflow, the GridRPC WG wishes to dovetail with other efforts and make such workflow management tools even more available.

5. Are there sufficient interest and expertise in the group's topic, with at least several people willing to expend the effort that is likely to produce significant results over time?
Yes. As already mentioned, there are four research groups with funded projects already producing implementations (NetSolve, Ninf-G, DIET, and OmniRPC). Representatives from three of these groups are members of the proposed working group organization. Collectively they represent years of experience in this area, have funding to support further development, and are connected to a user community that will drive the development of the Proposed Recommendation. (See next question.)

6. Does a base of interested consumers (e.g., application developers, Grid system implementers, industry partners, end-users) appear to exist for the planned work?
Yes. GridRPC will address the needs of an established and growing user base for network-enabled services such as NetSolve, Ninf or DIET. Of the four GridRPC-like tools available today, NetSolve is the most widely deployed and used. For the last three years, the NetSolve website has logged the following number of downloads per year: The number of downloads is 627 in 2000, 831 in 2001 and 1106 in 2002. A recent download log file (containing only 676 downloads) revealed that downloads came from 46 countries on 6 continents (from every continent except Antartica). Each download session also had to indicate an intended use. These uses include generic "experimental", "research", and "educational" uses, but also specifically mentioned simulation, image processing, signal processing, MRI data processing, finite element analysis, acoustics, computational fluid dynamics, bioengineering codes, neurophysiological codes, electron microscope tomography, mechanical engineering, chemical engineering, hydrology, radar pulse fingerprinting, synthetic aperture radar, FFTs, and running MCELL (a Monte Carlo simulator for cellular microphysiology). These are certainly just a few of the actual applications. We note that the original NetSolve tool is being expanded and generalized to GridSolve. The Ninf-G group reports the cumulative number of downloads since the first release of Ninf-G: 1325 in Ninf-G2, 1132 in Ninf-G4, and 21 in Ninf-G5 at the end of April in 2008. These downloads originated from countries on 5 continents: UK, Germany, Spain, France, Greek, Poland, Portugal, Austria, Switzerland, Japan, Korea, Singapore, Thailand, Taiwan, USA, Brazil, South Africa and so on. Known applications of Ninf-G include hybrid QM/MD simulations, fragment molecular orbital simulations, combinatorial optimization using genetic algorithms and branch-and-bound methods. The DIET group (which already has recently made a public release available) lists many applications as simulation of electronic devices, Digital Elevation Model computation, simulation of molecular and atomic systems and Grid Protein Pattern Scanning, Cosmological simulation, robotics application. In addition, DIET was chosen to be the Grid middleware of the Decrypthon Grid, a joint initiative of AFM (French association against muscular dystrophy), CNRS, and IBM, to help genomics and proteomics research. DIET is also involved with the JAEA (Japan Atomic Energy Agency) and other french research teams (IRIT/ENSEEIHT at Toulouse and LaBRI at Bordeaux) in the building of an international expert system for sparse linear algebra on an international grid computing environment. The number of download is 412 (since 2004 to 2008 with annual average close to 100). These downloads originated from 34 countries: ten major countries in amount of download are France, Algeria, USA, Corea, Canada, Romania, Ireland, Japan, Armenia and China. The OmniRPC group (which is close to having a version for public release) lists two major applications: (1) computational drug discovery using molecular mechanics to optimize the 3D conformations of a molecule, and (2) computing radiation transfer in the early phase of the universe on the GRAPE-6 machine. Clearly the network-enabled RPC concept is established and has a growing user base. A GridRPC recommendation would certainly facilitate the work of these and many other potential grid users, besides providing a standard to which grid computing products can be built.

7. Does the OGF have a reasonable role to play in the determination of the technology?
Yes. Not only does the OGF have a reasonable role to play in the determination of this technology, it is essential that this work takes place in the context of OGF since the proposed GridRPC Working Group will have to coordinate with and leverage the work of other groups. Specifically some important capabilities, such as Naming Services or Workflow treatment, will be integral to the GridRPC Proposed Recommendation but will not be developed as part of it. Hence, coordination and collaboration is essential with other such groups include Data Transport, Scheduling and Reservations, DRMAA, and Grid Services Workflow.

Active

This Working Group will produce an OGF Recommendation for a grid-enabled, remote procedure call (RPC) mechanism. The first document entitled A GridRPC Model and API for End-User Applications has been completed and published as GFD-R.52. Currently we are concentrating on the second recommendation document, which defines GridRPC API for middleware developers that extends the model and GridRPC API for end-users, and also focuses on data management mechanism within the GridRPC model. The GridRPC middleware tools developed will further lower the barrier to acceptance for grid use by hiding the tremendous amount of infrastructure necessary to make grids work, while providing even higher-level abstractions for domain-specific middleware.