gridrpc-wg

Grid Remote Procedure Call WG

Applications

The GridRPC Working Group was originally chartered to produce a GGF Proposed Recommendation for a grid-enabled, remote procedure call (RPC) mechanism. In the course of this work, it became clear that there was a natural division between a GridRPC mechanism for end-users and for middleware developers. On one hand, an end-user API would enable a wide range of applications and would be straightforward to define. On the other hand, a middleware API would allow powerful middleware tools to be developed but require additional complexity. Hence, the GridRPC Working Group chose to divide its work into two documents, one for end-users and a second for middleware developers. The first document is entitled A GridRPC Model and API for End-User Applications. This has been completed and published as GFD-R.52. The second document will be A GridRPC Model and API for Middleware Developers. This rechartering document captures the specific requirements, goals, milestones and deliverables for producing this second document.

This second Proposed Recommendation will primarily consist of an Application Programming Interface (API), and associated programming model, for middleware developers that extends the model and API for end-users. The GridRPC middleware tools developed will further lower the barrier to acceptance for grid use by hiding the tremendous amount of infrastructure necessary to make grids work, while providing even higher-level abstractions for domain-specific middleware.

The top-level objective of defining a proposed middleware recommendation can be decomposed into (at least) the following specific objectives:

* Definition of a specific data structure to be used for GridRPC arguments in middleware, e.g., a stack, a vector, etc.

* Definition of a "grpc_arg" data type, if necessary, to be used in conjunction with the argument data structure.

* Definition of the argument data structure creation, destruction, lifetime and copy semantics.

* Definition of possible introspection capabilities for call arguments and attributes.

* Definition of mechanisms for handling persistent data, e.g., definition and use of a concept such as "data handles" (which might be the same as or similar to a grpc_arg data type). This may also involve concepts such as lazy copy semantics, and data leases or time-outs.

* Definition of API mechanisms to enable workflow management.

* Evaluate the compatibility and interoperability with other systems, e.g., WSRF.

* Desirable Properties -- the Proposed Recommendation will not necessarily specify any properties, such as thread safety, security, and fault tolerance, but it should not be incompatible with any such useful properties.

* Demonstrate implementability of all parts of the API.

* Demonstrate and evaluate at least two implementations of the complete GridRPC middleware recommendation.

We specifically exclude the following topics as non-objectives:

* Generalized Workflow -- the GridRPC WG does not wish to define or develop general workflow management tools, rather we will endeavor to make it possible for end-users and middleware developers to interact with workflow tools developed elsewhere through the GridRPC API.

* Services names -- the Middleware GridRPC Proposed Recommendation will not define the Naming Service for service discovery but will have requirements for its use.

* Interoperability between implementations -- achieving interoperability between implementations requires defining, in essence, a wire protocol. This is actually orthogonal to the problem of defining the user-level model and API and, hence, can be done later as part of a separate effort.

Since the proposed GridRPC Working Group has such a clear focus, the exit strategy is straightforward: producing a middleware Proposed Recommendation document. In the case that any of the technical issues identified above become too difficult to resolve within the allotted schedule, then the working group may have to re-charter to focus on that key issue. Even if the Proposed Recommendation document is produced on time, it is anticipated that a number of related issues will remain to be resolved that were considered to be outside of the scope of the proposed document, e.g., naming services. Hence, one or more "follow-on" working groups may be proposed at that time.

Title: A GridRPC Model and API for End-User Applications
Title: A GridRPC Model and API for Middleware Developers
Title: Interoperability Testing for The GridRPC API Specification

1. Is the scope of the proposed group sufficiently focused?
Yes. This proposed working group is focused specifically on producing a OGF proposed recommendation document for a GridRPC tool suitable for middleware development. Much work has already been done in at least four independent research groups around the world on the end-user model and API that is the starting point for this work. The specific issues to be resolved as part of this group���s work have already been identified in the GridRPC Working Group���s efforts to date. The proposed working group is focused on resolving these specific issues and producing a second OGF Proposed Recommendation document.

2. Are the topics that the group plans to address clear and relevant for the Grid research, development, industrial, implementation, and/or application user community?
Yes. Since RPC is an established programming model, a GridRPC tool will provide a much lower "barrier to acceptance" for application groups that are already using an RPC-style. The fact that GridRPC will conveniently package so much underlying grid technology means that it is essential that this work be done within the context of GGF to properly leverage the infrastructure development and implementation work on which GridRPC will rely. A middleware API will enable systems to be built that embody even higher-levels of abstraction for domain-specific tools.

3. Will the formation of the group foster (consensus-based) work that would not be done otherwise?
Yes. Since there are four different research groups from three countries on three continents already pursuing this work, the formation of this group is essential to coordinate their activities, eliminate redundant work, and establish a common model and API that represents the best practices of the user community. (See "Base of Interested Consumers" below.)

4. Do the group's activities overlap inappropriately with those of another OGF group or to a group active in another organization such as IETF or W3C?
No. In fact, the proposed group���s activities dovetail with other work being done in OGF and W3C. A GridRPC Proposed Recommendation will focus on the user-level API, i.e., a programming tool for humans. By contrast, the Open Grid Services Architecture (OGSA) and the Web Services Definition Language (WSDL) are focusing on the interface between program modules. Hence, from an implementation perspective, a GridRPC Proposed Recommendation could leverage the request-response style that is possible in WSDL and OGSA to something that is more suitable for routine human use. With regards to workflow, the GridRPC WG wishes to dovetail with other efforts and make such workflow management tools even more available.

5. Are there sufficient interest and expertise in the group's topic, with at least several people willing to expend the effort that is likely to produce significant results over time?
Yes. As already mentioned, there are four research groups with funded projects already producing implementations (NetSolve, Ninf-G, DIET, and OmniRPC). Representatives from three of these groups are members of the proposed working group organization. Collectively they represent years of experience in this area, have funding to support further development, and are connected to a user community that will drive the development of the Proposed Recommendation. (See next question.)

6. Does a base of interested consumers (e.g., application developers, Grid system implementers, industry partners, end-users) appear to exist for the planned work?
Yes. GridRPC will address the needs of an established and growing user base for network-enabled services such as NetSolve and Ninf. Of the four GridRPC-like tools available today, NetSolve is the most widely deployed and used. For the last three years, the NetSolve website has logged the following number of downloads per year: The number of downloads is 627 in 2000, 831 in 2001 and 1106 in 2002. A recent download log file (containing only 676 downloads) revealed that downloads came from 46 countries on 6 continents (from every continent except Antartica). Each download session also had to indicate an intended use. These uses include generic "experimental", "research", and "educational" uses, but also specifically mentioned simulation, image processing, signal processing, MRI data processing, finite element analysis, acoustics, computational fluid dynamics, bioengineering codes, neurophysiological codes, electron microscope tomography, mechanical engineering, chemical engineering, hydrology, radar pulse fingerprinting, synthetic aperture radar, FFTs, and running MCELL (a Monte Carlo simulator for cellular microphysiology). These are certainly just a few of the actual applications. We note that the original NetSolve tool is being expanded and generalized to GridSolve. The Ninf-G group reports 514 downloads from 203 unique hosts in a 4 month period starting November, 2002, when Ninf-G was released. (While there may have been a burst of downloads following the release of Ninf-G, it is possible that the yearly number of downloads for Ninf-G may equal or exceed that of NetSolve.) These downloads originated from countries on 5 continents: UK, Germany, Spain, France, Greek, Poland, Portugal, Austria, Switzerland, Japan, Korea, Singapore, Thailand, Taiwan, USA, Brazil, and South Africa. Known applications of Ninf-G include molecular simulations, combinatorial optimization using genetic algorithms and branch-and-bound methods. The DIET group (which already has recently made a public release available) lists four major applications (1) simulation of electronic devices, (2) Digital Elevation Model computation, (3) simulation of molecular and atomic systems and (4) Grid Protein Pattern Scanning. The OmniRPC group (which is close to having a version for public release) lists two major applications: (1) computational drug discovery using molecular mechanics to optimize the 3D ���conformations��� of a molecule, and (2) computing radiation transfer in the early phase of the universe on the GRAPE-6 machine. Clearly the network-enabled RPC concept is established and has a growing user base. A GridRPC recommendation would certainly facilitate the work of these and many other potential grid users, besides providing a standard to which grid computing products can be built.

7. Does the OGF have a reasonable role to play in the determination of the technology?
Yes. Not only does the OGF have a reasonable role to play in the determination of this technology, it is essential that this work take place in the context of OGF since the proposed GridRPC Working Group will have to coordinate with and leverage the work of other groups. Specifically some important capabilities, such as Naming Services, will be integral to the GridRPC Proposed Recommendation but will not be developed as part of it. Hence, coordination and collaboration is essential with other such groups include Data Transport, Scheduling and Reservations, DRMAA, and Grid Services Workflow.

Active

The GridRPC Working Group was originally chartered to produce a GGF Proposed Recommendation for a grid-enabled, remote procedure call (RPC) mechanism. In the course of this work, it became clear that there was a natural division between a GridRPC mechanism for end-users and for middleware developers. On one hand, an end-user API would enable a wide range of applications and would be straightforward to define. On the other hand, a middleware API would allow powerful middleware tools to be developed but require additional complexity. Hence, the GridRPC Working Group chose to divide its work into two documents, one for end-users and a second for middleware developers. The first document is entitled A GridRPC Model and API for End-User Applications. This has been completed and published as GFD-R.52. The second document will be A GridRPC Model and API for Middleware Developers. This rechartering document captures the specific requirements, goals, milestones and deliverables for producing this second document. This second Proposed Recommendation will primarily consist of an Application Programming Interface (API), and associated programming model, for middleware developers that extends the model and API for end-users. The GridRPC middleware tools developed will further lower the barrier to acceptance for grid use by hiding the tremendous amount of infrastructure necessary to make grids work, while providing even higher-level abstractions for domain-specific middleware.