Attached. Gridforge: https://forge.gridforum.org/sf/go/doc14102?nav=1 -- Andreas Savva Fujitsu Laboratories Ltd
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs. Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI). Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange. To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies. I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities. See you at OGF19. Abdeslem DJAOUI
You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg
-- : : Geoffrey Fox gcf@indiana.edu FAX 8128567972 http://www.infomall.org : Phones Cell 812-219-4643 Home 8123239196 Lab 8128567977 : SkypeIn 812-669-0772 with voicemail, International cell 8123910207
Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:> In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons). I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others. Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no. Abdeslem -----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg
-- : : Geoffrey Fox gcf@indiana.edu FAX 8128567972 http://www.infomall.org : Phones Cell 812-219-4643 Home 8123239196 Lab 8128567977 : SkypeIn 812-669-0772 with voicemail, International cell 8123910207
In the MPI Workflow-MPI connection, I recommend looking carefully at Microsoft's CCR and DSS which are part of their recently released Robotics Studio. http://msdn.microsoft.com/robotics/. Robotics naturally uses a workflow model with services for its different functions. You find that this service(thread) model gets about 4 microsecond latency on MPI communication patterns (on Dell with two Xeon processors each of 4 cores) and 30-50 microseconds on service operations. (early results on 2-core Opteron/Xeon at end of http://grids.ucs.indiana.edu/ptliupages/presentations/qiupresentation.ppt) This performance is an order of magnitude better than say NaradaBrokering even using vast improvement of threading in Java1.5 GMSEC from NASA Goddard is a good Mule-like system applied to Grid-like space applications As you say, Mule puts yet another interoperability layer n top of existing containers including Web Services. I complained at an Israeli Grid meeting that this seemed odd as I thought Web Services were the interoperability layer. But of of course the real world doesn't believe this as will be seen in Web 2.0 mashups (aka workflow - see OGF19 Monday meeting!) where REST interactions dominate Djaoui, A (Abdeslem) wrote:
Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:>
In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons).
I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others.
Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no.
Abdeslem
-----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors
Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg
-- : : Geoffrey Fox gcf@indiana.edu FAX 8128567972 http://www.infomall.org : Phones Cell 812-219-4643 Home 8123239196 Lab 8128567977 : SkypeIn 812-669-0772 with voicemail, International cell 8123910207
Yes at first read DSS seems similar to OGSI/WSRF (although REST friendly), but CCR is very interesting for the type of asynchronous messaging I had in mind. Apart from the Licencing restrictions, I will try and find out more about how "open" CCR is and what it does. Also, the references to orchestration probably mean a central process coordinating and synchronizing others (ie as in WS-BPEL workflow). I guess that's what you need from a robot! It is not clear to me if CCR can allow coupling of independent workflows, something needed in some envisaged Grid applications. The performance figures compared to Naradabrokering are interesting, but it is not clear to me if this is the result of use of threads or some underlying binary protocol. Going back to OGSA message broker, the intention is to define the interface and semantics so that messages can be transported on different wire protocols, including more efficient protocols such as openais, http://developer.osdl.org/dev/openais As for REST versus web services/SOAP I don't see a real problem there. SOAP as the web services interoperability layer is an extensible message/data model (ie headers and body) and processing rules, not an application protocol. It can be used by any application protocol (including REST). Amazon WS Simple Message Queue for example can be accessed either a REST API or a non-REST API. Of course for the non-REST API additional infrastructure that deals with SOAP headers is needed. Abdeslem ________________________________ From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Wednesday, January 24, 2007 12:34 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service In the MPI Workflow-MPI connection, I recommend looking carefully at Microsoft's CCR and DSS which are part of their recently released Robotics Studio. http://msdn.microsoft.com/robotics/. Robotics naturally uses a workflow model with services for its different functions. You find that this service(thread) model gets about 4 microsecond latency on MPI communication patterns (on Dell with two Xeon processors each of 4 cores) and 30-50 microseconds on service operations. (early results on 2-core Opteron/Xeon at end of http://grids.ucs.indiana.edu/ptliupages/presentations/qiupresentation.ppt) This performance is an order of magnitude better than say NaradaBrokering even using vast improvement of threading in Java1.5 GMSEC from NASA Goddard is a good Mule-like system applied to Grid-like space applications As you say, Mule puts yet another interoperability layer n top of existing containers including Web Services. I complained at an Israeli Grid meeting that this seemed odd as I thought Web Services were the interoperability layer. But of of course the real world doesn't believe this as will be seen in Web 2.0 mashups (aka workflow - see OGF19 Monday meeting!) where REST interactions dominate Djaoui, A (Abdeslem) wrote: Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:> In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons). I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others. Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no. Abdeslem -----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org <http://mule.codehaus.org> by Israeli association of Grid Technologies http:www.grid.org.il <http://www.grid.org.il> With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors Djaoui, A (Abdeslem) wrote: When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs. Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI). Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange. To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies. I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities. See you at OGF19. Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg -- : : Geoffrey Fox gcf@indiana.edu FAX 8128567972 http://www.infomall.org : Phones Cell 812-219-4643 Home 8123239196 Lab 8128567977 : SkypeIn 812-669-0772 with voicemail, International cell 8123910207
Hi, for what its worth: the SAGA groups are in the process of defining a message exchange API, which allows a range of modes: reliable/unreliable, message bus/point to point, correctness verified/unverified, and ordered/unordered. The target use case for this API is the exchange of _large_ and _unstructured_ messages (think megabytes or gigabytes), so thats probably a very different scenario than OGSA thinks about (?). Anyway, it might be useful to syncronize in terms of terminology and paradigms. I'd be happy to give you some details of the SAGA message API at OGF19, if you have interest. The API is in flux, but is expected to stabilize during the next 3 weeks or so. Cheers, Andre, Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: gcf@indiana.edu Cc: Mailing List for OGSA-WG <ogsa-wg@ogf.org> Subject: Re: [ogsa-wg] OGSA message broker service
Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:>
In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons).
I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others.
Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no.
Abdeslem
-----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors
Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg -- "So much time, so little to do..." -- Garfield
Hi That sounds relevant and I would very much like to see the details. For an OGSA message broker it will also be necessary to define A broker/broker interface, but for the client/broker interface, there could be a lot of common ground. See you at OGF19. Abdeslem -----Original Message----- From: Andre Merzky [mailto:andre@merzky.net] Sent: Wednesday, January 24, 2007 2:09 PM To: Djaoui, A (Abdeslem) Cc: gcf@indiana.edu; Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service Hi, for what its worth: the SAGA groups are in the process of defining a message exchange API, which allows a range of modes: reliable/unreliable, message bus/point to point, correctness verified/unverified, and ordered/unordered. The target use case for this API is the exchange of _large_ and _unstructured_ messages (think megabytes or gigabytes), so thats probably a very different scenario than OGSA thinks about (?). Anyway, it might be useful to syncronize in terms of terminology and paradigms. I'd be happy to give you some details of the SAGA message API at OGF19, if you have interest. The API is in flux, but is expected to stabilize during the next 3 weeks or so. Cheers, Andre, Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: gcf@indiana.edu Cc: Mailing List for OGSA-WG <ogsa-wg@ogf.org> Subject: Re: [ogsa-wg] OGSA message broker service
Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:>
In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons).
I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others.
Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no.
Abdeslem
-----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors
Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg -- "So much time, so little to do..." -- Garfield
The SAGA calendar is filling rapidly - would it be possible to reserve a time slot in advance for this? Would sometime Monday evening be an option for you? Thanks, best regards, Andre. Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
Subject: RE: [ogsa-wg] OGSA message broker service From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: Andre Merzky <andre@merzky.net> Cc: gcf@indiana.edu, Mailing List for OGSA-WG <ogsa-wg@ogf.org>
Hi That sounds relevant and I would very much like to see the details. For an OGSA message broker it will also be necessary to define A broker/broker interface, but for the client/broker interface, there could be a lot of common ground.
See you at OGF19.
Abdeslem
-----Original Message----- From: Andre Merzky [mailto:andre@merzky.net] Sent: Wednesday, January 24, 2007 2:09 PM To: Djaoui, A (Abdeslem) Cc: gcf@indiana.edu; Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
Hi,
for what its worth: the SAGA groups are in the process of defining a message exchange API, which allows a range of modes: reliable/unreliable, message bus/point to point, correctness verified/unverified, and ordered/unordered.
The target use case for this API is the exchange of _large_ and _unstructured_ messages (think megabytes or gigabytes), so thats probably a very different scenario than OGSA thinks about (?). Anyway, it might be useful to syncronize in terms of terminology and paradigms.
I'd be happy to give you some details of the SAGA message API at OGF19, if you have interest. The API is in flux, but is expected to stabilize during the next 3 weeks or so.
Cheers, Andre,
Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: gcf@indiana.edu Cc: Mailing List for OGSA-WG <ogsa-wg@ogf.org> Subject: Re: [ogsa-wg] OGSA message broker service
Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:>
In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons).
I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others.
Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no.
Abdeslem
-----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors
Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg -- "So much time, so little to do..." -- Garfield
Hi Abdeslem, Your OGSA message broker proposal attracts many interest. Do you want to have a open discussion on this at one of OGSA-WG's session? Does Tuesday, January 30, 9-10:30 am "OGSA EMS session" work for you? If not, which session work for you? https://forge.gridforum.org/sf/go/wiki1678 Thanks, ---- Hiro Kishimoto Andre Merzky wrote:
The SAGA calendar is filling rapidly - would it be possible to reserve a time slot in advance for this? Would sometime Monday evening be an option for you?
Thanks, best regards,
Andre.
Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
Subject: RE: [ogsa-wg] OGSA message broker service From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: Andre Merzky <andre@merzky.net> Cc: gcf@indiana.edu, Mailing List for OGSA-WG <ogsa-wg@ogf.org>
Hi That sounds relevant and I would very much like to see the details. For an OGSA message broker it will also be necessary to define A broker/broker interface, but for the client/broker interface, there could be a lot of common ground.
See you at OGF19.
Abdeslem
-----Original Message----- From: Andre Merzky [mailto:andre@merzky.net] Sent: Wednesday, January 24, 2007 2:09 PM To: Djaoui, A (Abdeslem) Cc: gcf@indiana.edu; Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
Hi,
for what its worth: the SAGA groups are in the process of defining a message exchange API, which allows a range of modes: reliable/unreliable, message bus/point to point, correctness verified/unverified, and ordered/unordered.
The target use case for this API is the exchange of _large_ and _unstructured_ messages (think megabytes or gigabytes), so thats probably a very different scenario than OGSA thinks about (?). Anyway, it might be useful to syncronize in terms of terminology and paradigms.
I'd be happy to give you some details of the SAGA message API at OGF19, if you have interest. The API is in flux, but is expected to stabilize during the next 3 weeks or so.
Cheers, Andre,
Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: gcf@indiana.edu Cc: Mailing List for OGSA-WG <ogsa-wg@ogf.org> Subject: Re: [ogsa-wg] OGSA message broker service
Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:>
In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons).
I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others.
Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no.
Abdeslem
-----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors
Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg
Sure Hiro, I will prepare some slides to to start the discussion. Abdeslem -----Original Message----- From: Hiro Kishimoto [mailto:hiro.kishimoto@jp.fujitsu.com] Sent: Sun 1/28/2007 10:19 PM To: Andre Merzky Cc: Djaoui, A (Abdeslem); Shantenu Jha; gcf@indiana.edu; Thilo Kielmann; Hartmut Kaiser; Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service Hi Abdeslem, Your OGSA message broker proposal attracts many interest. Do you want to have a open discussion on this at one of OGSA-WG's session? Does Tuesday, January 30, 9-10:30 am "OGSA EMS session" work for you? If not, which session work for you? https://forge.gridforum.org/sf/go/wiki1678 Thanks, ---- Hiro Kishimoto Andre Merzky wrote:
The SAGA calendar is filling rapidly - would it be possible to reserve a time slot in advance for this? Would sometime Monday evening be an option for you?
Thanks, best regards,
Andre.
Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
Subject: RE: [ogsa-wg] OGSA message broker service From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: Andre Merzky <andre@merzky.net> Cc: gcf@indiana.edu, Mailing List for OGSA-WG <ogsa-wg@ogf.org>
Hi That sounds relevant and I would very much like to see the details. For an OGSA message broker it will also be necessary to define A broker/broker interface, but for the client/broker interface, there could be a lot of common ground.
See you at OGF19.
Abdeslem
-----Original Message----- From: Andre Merzky [mailto:andre@merzky.net] Sent: Wednesday, January 24, 2007 2:09 PM To: Djaoui, A (Abdeslem) Cc: gcf@indiana.edu; Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
Hi,
for what its worth: the SAGA groups are in the process of defining a message exchange API, which allows a range of modes: reliable/unreliable, message bus/point to point, correctness verified/unverified, and ordered/unordered.
The target use case for this API is the exchange of _large_ and _unstructured_ messages (think megabytes or gigabytes), so thats probably a very different scenario than OGSA thinks about (?). Anyway, it might be useful to syncronize in terms of terminology and paradigms.
I'd be happy to give you some details of the SAGA message API at OGF19, if you have interest. The API is in flux, but is expected to stabilize during the next 3 weeks or so.
Cheers, Andre,
Quoting [Djaoui, A (Abdeslem)] (Jan 24 2007):
From: "Djaoui, A (Abdeslem)" <A.Djaoui@rl.ac.uk> To: gcf@indiana.edu Cc: Mailing List for OGSA-WG <ogsa-wg@ogf.org> Subject: Re: [ogsa-wg] OGSA message broker service
Well, the multitude of messaging systems out there is exactly one reason I believe OGSA needs to define "standard" APIs for messaging. Of course for Pub/sub it will leverage existing standards. It is interesting to read from the overview of Mule, <quote:> "But there are already frameworks out there. It is often misconstrued that Mule aims to be a replacement for existing application frameworks, but this is not the case. In fact Mule leverages many open source projects such as Axis, Spring, ActiveMQ, Plexus and PicoContainer. Mule fills a void in enterprise java development ... Mule makes light work of wiring these systems together ..." </End quote:>
In the same spirit, OGSA message broker should fill the void in Web services messaging that distributed application that need to share state across different security realms will require. OGSA defines the interfaces and semantics only. Implementations could be based on existing messaging systems. I was involved for quite a while with the INFOD-WG and instrumental in coordinating that activity with OGSA information services. Currently INFOD deals with defining an interface to a registry but potentially it could expand to deal with other messaging patterns. For scalability reasons, an OGSA broker is different in that a central registry is not used, but federations of brokers will need to be defined, using WS-federation for sharing identities across different authentication and authorization domains. So the focus in this activity is to build on the existing web services specification (not java specific) and to leverage other work in OGSA (for compatibility reasons).
I am aware of the existence of NaradaBrokering (but only know a little about it), and I believe something similar needs to be standardized in OGSA. You mention its use for monitoring (real time sensors), but what I had in mind for an OGSA broker, was the capability to allow MPI-like message exhange patterns between distributed activities(web services). I gave the example of coupling two WS-BPEL processes, but there are others.
Anyway an OGSA message broker interface could end up the same or similar to Naradbrokerrring, Mule, Infod version 2 or 3, or JMS,... The question in my mind is: Do the existing systems allow composition of web services out of the box, in a standard way. At least for some of the systems I know better, the answer is no.
Abdeslem
-----Original Message----- From: Geoffrey Fox [mailto:gcf@grids.ucs.indiana.edu] Sent: Tuesday, January 23, 2007 1:10 PM To: Djaoui, A (Abdeslem) Cc: Mailing List for OGSA-WG Subject: Re: [ogsa-wg] OGSA message broker service
You might want to look at Enterprise Grid work using Mule http:mule.codehaus.org by Israeli association of Grid Technologies http:www.grid.org.il With OGF there is long running activity called INFOD and of course there is WS-Eventing/Notification and their merger Apache Synapse project addresses some relevant issues We use publish-subscribe http://www.naradabrokering.org in nearly all projects we have -- especially for real-time sensors
Djaoui, A (Abdeslem) wrote:
When building Grid applications, coordinating ativities may be necessary as pointed out in a recent Email about workflow http://www.ogf.org/pipermail/ogsa-wg/2006-November/002324.html Usually, in traditional Enterprise Application Integration (EAI), the problem is solved using message oriented middleware (MOM) such as JMS. These (MOMs) provide asynchronous point-to-point communication as well as pub/sub capabilities, but Interoperability is difficult to achieve with the existing multitude of proprietary MOMs.
Although messaging is briefly described in section 3.9.4.2 (message delivery) of the OGSA specification 1.5, no activity in this area was deemed necessary so far. As OGSA moves beyond the basic service capabilities into service composition, it is necessary to specify a standard message queuing and routing interface, if OGSA is to enable new kinds of useful distributed applications (similar to EAI).
Amazon Web Services (AWS) http://aws.amazon.com Recognize the power of message queuing and provide a Simple Queue Service for their customers. In AWS, distributed application components performing different tasks can exchange messages without requiring the receiving party to be ready to receive the message of even be available at the time of exchange.
To enable, interoperable message queuing and routing OGSA will need to define similar interfaces (and semantics) for client applications to publish and consume messages to/from OGSA broker services. Broker services accept messages and route them to other brokers or to different consumers. A variety of message patterns will need to be supported (e.g. point-to-point, one-to-many, notification, ...). For OGSA service composition/coordination, it is inevitable for some to draw a parallel with classical massively parallel computer where services playing the role of computer nodes and messaging playing the role of the Message Passing Interface (MPI). Of course there are limitations to this parallel. One is the performance, reliability and latency of messaging system compared to MPI. Another is the need to federate broker services from different security realms having potentially different policies.
I would like to start exploring how an OGSA message broker might be used by client applications. An example use case for the discussion could be two (or more) independent BPEL processes (exposed as web services) that need to share/exchange (internal) state information. This currently not possible in WS-BPEL, but is needed for coordinating/coupling distributed activities.
See you at OGF19.
Abdeslem DJAOUI -- ogsa-wg mailing list ogsa-wg@ogf.org http://www.ogf.org/mailman/listinfo/ogsa-wg
participants (5)
-
Andre Merzky -
Andreas Savva -
Djaoui, A (Abdeslem) -
Geoffrey Fox -
Hiro Kishimoto