Hi Michal, The NSI WG is well aware of the fact that the asynchronous behavior of the protocol isn't working for those behind a firewall or NAT. A couple of organizations that are connected to SURFnet started to implement NSI to provision paths through our network and they all run into this problem. To help them out SURFnet offered to come up with a `user NSI CS protocol' that is firewall and NAT save. You are invited to help define how this should look like :) HansT. On 10/31/12 11:24 AM, michalb wrote:

Hello,

This is for reservation query - will there be a synchronous version of that operation (basic query at least)? It is not always possible to listen on query responses and polling would help under such circumstances.

br michal

-----Oryginalna wiadomość----- From: Jeroen van der Ham Sent: Tuesday, September 18, 2012 4:37 PM To: nsi-wg@ogf.org Group Subject: [Nsi-wg] Query request

Hi,

While thinking about a demonstration scenario for SC, we came upon an interesting use-case for the query command. We are trying to plan workflows on the infrastructure, depending on availability. For that it would actually be useful to know not only if a certain resource is in use, but also when it could become available again.

Would it be possible to add capabilities for the query command to provide something like that?

Jeroen.

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg _______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Hi Michal, On 10/31/12 1:25 PM, michalb wrote:

I wanted to point out a little bit of protocol inconsistance. In the NSI Connection Service Protocol_20_v1 doc one can read (regarding 'replyTo' field):

Do you know where I can find the latest version of the NSI CS 2.0 document?

"If no endpoint value is provided in a operation request, then it is assumed the requester is not interested in a response and will use alternative mechanism to determine the result."

So for some GUI purposes which surely will be behind firewall/NAT, I reserve, provison, etc with no 'replyTo' provided and now I'd like to "use alternative mechanism to determine the result" (which is query) and because query is async only the whole thing gets defeated. Would be nice to have a simple query that returns basic query result so the feature of not setting 'replyTo' would have more sense.

Very good point, this leaves users behind firewalls and NATs completely in the dark about the status of there reservation. This could easily be solved because a summary query that is supposed to only return information local to that NSA there is no need to send the result asynchronous. HansT.

-----Oryginalna wiadomość----- From: Hans Trompert Sent: Wednesday, October 31, 2012 1:01 PM To: michalb Cc: nsi-wg@ogf.org Subject: Re: [Nsi-wg] Query request

Hi Michal,

The NSI WG is well aware of the fact that the asynchronous behavior of the protocol isn't working for those behind a firewall or NAT. A couple of organizations that are connected to SURFnet started to implement NSI to provision paths through our network and they all run into this problem. To help them out SURFnet offered to come up with a `user NSI CS protocol' that is firewall and NAT save. You are invited to help define how this should look like :)

HansT.

On 10/31/12 11:24 AM, michalb wrote:

...

Hello,

This is for reservation query - will there be a synchronous version of that operation (basic query at least)? It is not always possible to listen on query responses and polling would help under such circumstances.

br michal

-----Oryginalna wiadomość----- From: Jeroen van der Ham Sent: Tuesday, September 18, 2012 4:37 PM To: nsi-wg@ogf.org Group Subject: [Nsi-wg] Query request

Hi,

While thinking about a demonstration scenario for SC, we came upon an interesting use-case for the query command. We are trying to plan workflows on the infrastructure, depending on availability. For that it would actually be useful to know not only if a certain resource is in use, but also when it could become available again.

Would it be possible to add capabilities for the query command to provide something like that?

Jeroen.

Hmm, this version is from June this year, there have been two OGF meetings since then and a lot has changed, isn't there a newer version of this document (Guy can probably answer this)? HansT. On 10/31/12 1:58 PM, michalb wrote:

Hans,

I found it here https://forge.ogf.org/sf/go/doc16446?nav=1

ReplyTo can be found on 12/13 page.

br michal

-----Oryginalna wiadomość----- From: Hans Trompert Sent: Wednesday, October 31, 2012 1:54 PM To: michalb Cc: nsi-wg@ogf.org Subject: Re: [Nsi-wg] Query request

Hi Michal,

On 10/31/12 1:25 PM, michalb wrote:

...

I wanted to point out a little bit of protocol inconsistance. In the NSI Connection Service Protocol_20_v1 doc one can read (regarding 'replyTo' field):

Do you know where I can find the latest version of the NSI CS 2.0 document?

...

"If no endpoint value is provided in a operation request, then it is assumed the requester is not interested in a response and will use alternative mechanism to determine the result."

So for some GUI purposes which surely will be behind firewall/NAT, I reserve, provison, etc with no 'replyTo' provided and now I'd like to "use alternative mechanism to determine the result" (which is query) and because query is async only the whole thing gets defeated. Would be nice to have a simple query that returns basic query result so the feature of not setting 'replyTo' would have more sense.

Very good point, this leaves users behind firewalls and NATs completely in the dark about the status of there reservation. This could easily be solved because a summary query that is supposed to only return information local to that NSA there is no need to send the result asynchronous.

HansT.

...

-----Oryginalna wiadomość----- From: Hans Trompert Sent: Wednesday, October 31, 2012 1:01 PM To: michalb Cc: nsi-wg@ogf.org Subject: Re: [Nsi-wg] Query request

Hi Michal,

The NSI WG is well aware of the fact that the asynchronous behavior of the protocol isn't working for those behind a firewall or NAT. A couple of organizations that are connected to SURFnet started to implement NSI to provision paths through our network and they all run into this problem. To help them out SURFnet offered to come up with a `user NSI CS protocol' that is firewall and NAT save. You are invited to help define how this should look like :)

HansT.

On 10/31/12 11:24 AM, michalb wrote:

...

Hello,

This is for reservation query - will there be a synchronous version of that operation (basic query at least)? It is not always possible to listen on query responses and polling would help under such circumstances.

br michal

-----Oryginalna wiadomość----- From: Jeroen van der Ham Sent: Tuesday, September 18, 2012 4:37 PM To: nsi-wg@ogf.org Group Subject: [Nsi-wg] Query request

Hi,

While thinking about a demonstration scenario for SC, we came upon an interesting use-case for the query command. We are trying to plan workflows on the infrastructure, depending on availability. For that it would actually be useful to know not only if a certain resource is in use, but also when it could become available again.

Would it be possible to add capabilities for the query command to provide something like that?

Jeroen.

On Wed, 31 Oct 2012, michalb wrote:

"If no endpoint value is provided in a operation request, then it is assumed the requester is not interested in a response and will use alternative mechanism to determine the result."

So for some GUI purposes which surely will be behind firewall/NAT, I reserve, provison, etc with no 'replyTo' provided and now I'd like to "use alternative mechanism to determine the result" (which is query) and because query is async only the whole thing gets defeated. Would be nice to have a simple query that returns basic query result so the feature of not setting 'replyTo' would have more sense.

Wauv; that is silly. Very good find though. +1 for making summary query synchronous as John suggested (this would of course mean that we are making the protocol more practical to use, which is a somewhat new thing is this group :=) ). Best regards, Henrik Henrik Thostrup Jensen <htj at nordu.net> Software Developer, NORDUnet

Thought I might bump this back up since we never closed on the idea of a synchronous response for the query summary operation. Begin forwarded message:

From: John MacAuley <john.macauley@surfnet.nl> Subject: Re: [Nsi-wg] Query request Date: 1 November, 2012 8:31:36 AM EDT To: Henrik Thostrup Jensen <htj@nordu.net> Cc: NSI Working Group <nsi-wg@ogf.org>

The reason we put in the clause was for those applications that did not want to wait around for a v1 provision response. Obviously, it could be used for all other operations except query. Still a useful feature, just doesn't provide 100% coverage :-)

So the new query behaviour would be if (query.rq && query.operation == Summary && header.replyTo == null) then perform a synchronous request, otherwise, perform an asynchronous request.

John.

On 2012-11-01, at 6:22 AM, Henrik Thostrup Jensen <htj@nordu.net> wrote:

...

On Wed, 31 Oct 2012, michalb wrote:

...

"If no endpoint value is provided in a operation request, then it is assumed the requester is not interested in a response and will use alternative mechanism to determine the result."

So for some GUI purposes which surely will be behind firewall/NAT, I reserve, provison, etc with no 'replyTo' provided and now I'd like to "use alternative mechanism to determine the result" (which is query) and because query is async only the whole thing gets defeated. Would be nice to have a simple query that returns basic query result so the feature of not setting 'replyTo' would have more sense.

Wauv; that is silly. Very good find though.

+1 for making summary query synchronous as John suggested (this would of course mean that we are making the protocol more practical to use, which is a somewhat new thing is this group :=) ).

Best regards, Henrik

Henrik Thostrup Jensen <htj at nordu.net> Software Developer, NORDUnet

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Hi, It was still on my list to investigate the need for a separate User API that is firewall and NAT save. Over the last couple of months I realized that the only real way of solving this is to make the whole NSI synchronous instead of asynchronous. I do think that it is possible to make everything synchronous but there are also a couple of very good reasons to leave it asynchronous. (please note that I do not want to provoke any kind of discussion here, or at least not now) I think that we can get away by leaving the protocol as is. If users can not or do not want to receive the asynchronous messages then that is fine, the aim is to have enough resources available so hardly any request for bandwidth will be turned down anyway. If it is important to the user to know the status of his request, or he is just curious, he can use the a NSI query request to find out the actual state of his request. There is only one but here, this means that the query request must be synchronous! As an extra measure we are implementing the possibility to have status changes sent to the user by email. This is in no way meant as an replacement for the asynchronous messages, and of course only works if the request was made by one of our own users because otherwise we do not have an email address we can use. Almost all users have indicated that they trust us to provide them with the bandwidth when they need it, they only really want to be notified one way or the other when things go wrong. To most email is a very acceptable way of doing so. HansT. On 1/9/13 10:21 AM, Henrik Thostrup Jensen wrote:

On Tue, 8 Jan 2013, John MacAuley wrote:

...

Thought I might bump this back up since we never closed on the idea of a synchronous response for the query summary operation.

I actually wondered about this sometime. I'm still for it. It makes us marginally more firewall friendly, but in an important way.

Would it be a good idea have two seperate requests for this instead of the one? Possibly calling one query, and the other queryRecursive or similar. The current names are somewhat misleading.

Best regards, Henrik

Henrik Thostrup Jensen <htj at nordu.net> Software Developer, NORDUnet _______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

I'm very much in favor of having the NSI CS 2.0 summary query implemented as a synchronous call as is in NSI CS 1.0. HansT. On 10/31/12 1:37 PM, John MacAuley wrote:

I would be more than happy to move the summary query to a synchronous call and have the detailed query asynchronous. Would definitely help with clients behind firewalls.

Sent from my iPhone

On 2012-10-31, at 6:24 AM, "michalb" <michalb@man.poznan.pl> wrote:

...

Hello,

This is for reservation query - will there be a synchronous version of that operation (basic query at least)? It is not always possible to listen on query responses and polling would help under such circumstances.

br michal

-----Oryginalna wiadomość----- From: Jeroen van der Ham Sent: Tuesday, September 18, 2012 4:37 PM To: nsi-wg@ogf.org Group Subject: [Nsi-wg] Query request

Hi,

While thinking about a demonstration scenario for SC, we came upon an interesting use-case for the query command. We are trying to plan workflows on the infrastructure, depending on availability. For that it would actually be useful to know not only if a certain resource is in use, but also when it could become available again.

Would it be possible to add capabilities for the query command to provide something like that?

Jeroen.

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg _______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

---

nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Very 1970s of you ;-) On 2012-11-01, at 12:00 PM, Jerry Sobieski <jerry@nordu.net> wrote:

Ok. This is on my list of NSI version 3 issues: Fixing the MTL I didn't want to get into it yet either, but I had this on my short list waiting for after SC...

The NSI protocol layer should not have to deal with issues associated with the session layer. NSI just wants to send and receive messages between NSAs, and have some assurance that the message was successfully delivered, or not. This has nothing to do with which message is being sent - it applies equally to all/any messaging between NSAs. This goes back to the MTL specification... We should not be projecting these lower layer issues into the NSI layer.

The problem is, admittedly, that NATs and Firewalls get in the way. The MTL should hide all of this from the NSI messaging. At the NSI layer we should not be worried about the replyTo fields as these are not part of NSI layer or even the MTL layer. For version3 we should formalize an MTL that delivers messages between NSAs - period. ...regardless of where those NSAs reside or which messages are being sent. The basic issue is to construct a session between two NSAs that can carry messages either way, and at any time, as long as the session is up. If the session is down, we queue the message until it is back up. If the message remains queued too long waiting for the session to re-appear, it times out...and a "MTL Timeout Error: Undeliverable Message" gets returned to the calling process. If the the message is delivered to the peer (receipt acknowledged), but the NSI layer does not receive a response to the NSI primitive in some period of time, then we get a different timeout error: "NSI Timeout Error: Unresponsive Peer".

This MTL allows either or both NSAs to establish the session. It could be configurable for each peer NSA. If I see a remote NSA in the topology and I want to send a Reserve msg, then I open a session and keep it open until one of us decide to drop it. If I know of a neighbor NSA that I would expect to interact with often, I can immediately bring up a session when I start and hold it open forever. For Others I bring up sessions only when needed, or maybe there is a an aggregator somewhere (not adjacent) that I use often, I can open and hold a session with that NSA, or open one as needed. But each NSA can configure how they wish to manage their sessions with other NSAs...indeed, this is in essence part of the control plane topology we introduced in v2.

This simplifies RAs for mere mortals since they can establish the session, send and/or receive messages, then drop the session until the wish to check again sometime later. If an agent wants to be notified immediately, it holds the session open indefinitely, or it sets itself outside the FW or NAT with a routable IP address and the notifying NSA can open a session when required. Or the user agent can simply periodically open a session and check,...

While this mechanism does not allow two MTLs to reach each other if both are behind a NAT or FW, this is situation that essentially prevents *any* sessions from being opened between these two hosts... and solving this problem is not an NSI or MTL issue - its a network engineering issue. A publically routable federating NSA could act as a control plane go between to work around the fortress walls between many NSAs behind FW and NATs (a classic peer-to-peer strategy)

The point is that NSI does not need to - and should not be burdened with - worrying about low level session issues... If the session is up, it can talk to the remote NSA and new or queued messages are sent in order. If it is down, messages get queued for a finite period of time. The MTL insures that the messages are "delivered" or "not delivered" and deals with the lower layer details. (This MTL notion is a common feature of many signaling protocols -e.g. ITU 2931 had the SCCP protocol that handled message delivery, routing, flow control, segmentation, etc. for delivering the actua signaling messages between the signaling nodes.)

This MTL can employ WS libraries or SOAP functions or other mechanisms, but NSI layer is not sensitive to them. Thus NSI becomes independent of those layers... and the MTL interface is all NSI needs to worry about in terms of sending or receiving messages. The MTL communications format can be advertised in the topology or out of band between trusted peers.

This MTL also makes it easy to send messages between any NSAs. And having the capability to send a message between two NSAs regardless of their role in the service tree is critical for some of the advanced tree processing functions we talked about for multi-point, monitoring, and fault processing, etc. We really REALLY should fix this and make NSI the master of how it sends messages - not the underlying code libraires.

But I agree with Inder, this is a topic for the spring when we begin making v3 features/requirements. Or a beer topic at SC :-)

Jerry

On 10/31/12 10:14 AM, Hans Trompert wrote:

...

I'm very much in favor of having the NSI CS 2.0 summary query implemented as a synchronous call as is in NSI CS 1.0.

HansT.

On 10/31/12 1:37 PM, John MacAuley wrote:

...

I would be more than happy to move the summary query to a synchronous call and have the detailed query asynchronous. Would definitely help with clients behind firewalls.

Sent from my iPhone

On 2012-10-31, at 6:24 AM, "michalb" <michalb@man.poznan.pl> wrote:

...

Hello,

This is for reservation query - will there be a synchronous version of that operation (basic query at least)? It is not always possible to listen on query responses and polling would help under such circumstances.

br michal

-----Oryginalna wiadomość----- From: Jeroen van der Ham Sent: Tuesday, September 18, 2012 4:37 PM To: nsi-wg@ogf.org Group Subject: [Nsi-wg] Query request

Hi,

While thinking about a demonstration scenario for SC, we came upon an interesting use-case for the query command. We are trying to plan workflows on the infrastructure, depending on availability. For that it would actually be useful to know not only if a certain resource is in use, but also when it could become available again.

Would it be possible to add capabilities for the query command to provide something like that?

Jeroen.

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg _______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

---

nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

---

nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Hi all, It seems that in the current document we have sections on the MDL and MTL. The MTL is deemed to be out of scope, and it is recommended to be "TCP/IP-HTTP-XML (such as SOAP)". Which I would scrap, and just put in SOAP, since NSI does not support any other TCP/IP-HTTP-XML stack. Given that TCP, HTTP and SOAP already provide more than enough delivery assurance, I have to wonder why we actually need that extra conceptual layer in the Message Delivery Layer. It already is basically outsourced to those layers anyway. And for the few cases where upper layer logic is required to make sure that a request is handled correctly, we turn to the State Machine in the form of acknowledgements. I know that the State Machine is currently still being improved. But did we also take the suggestion to make all short exchanges synchronous, and to allow optional acknowledgement for longer asynchronous exchanges? This would make it much simpler to use NSI in cases with limited connectivity (firewalls/clients/NAT etc.) Jeroen. Begin forwarded message:

From: Jeroen van der Ham <vdham@uva.nl> Subject: Re: [Nsi-wg] Query request Date: 2 November 2012 10:20:34 CET To: Jerry Sobieski <jerry@nordu.net> Cc: <nsi-wg@ogf.org>

Hi,

The NSI has chosen to use SOAP and WSDL as its message transport layer. This is a reasonably standard protocol, with support (more or less) in many different languages. For what we aim to do it is pretty good, and it has proven to work pretty well in the past few demos.

You're saying that we should not be projecting lower layer issues into the NSI layer. In my opinion this is exactly what has happened already. And it is also exactly the reason that we are having all these issues.

Reliable message transport layers are readily available off the shelf. TCP makes sure that a packet gets from the source to the destination. On top of that we're using HTTP, which also provides some additional guarantees that the message gets there, including some error codes regarding malformed messages, messages out of order, authentication errors, et cetera. We've used an additional SOAP layer on top, which provides you with even more guarantees.

Can someone please explain to me that in this scenario we really need to have separate confirmation messages? And even if we do, why do these messages have to be exchanged in a completely separate TCP-HTTP-SOAP session?

I understand that we have a state machine, with transition using confirmation messages. But can't we use any of the TCP/HTTP/SOAP status messages to achieve that?

Jeroen.

Hi, I don't understand this still. What does this add to the current mechanism of reliable message transport and the state machine which handles the error conditions? Jeroen. On 6 Dec 2012, at 11:43, Guy Roberts <Guy.Roberts@dante.net> wrote:

Jeroen,

The MTL layer looks after delivery between a pair of peering RA and PA. My understanding is that the MDL has different role, to confirm the correct delivery to all child PAs. This allows the logic associated with checking that all child PAs have responded correctly to be removed from the state machines. See the attached slide.

Guy

-----Original Message----- From: nsi-wg-bounces@ogf.org [mailto:nsi-wg-bounces@ogf.org] On Behalf Of Jeroen van der Ham Sent: 06 December 2012 10:18 To: NSI Working Group Subject: [Nsi-wg] Message Delivery Layer

Hi all,

It seems that in the current document we have sections on the MDL and MTL. The MTL is deemed to be out of scope, and it is recommended to be "TCP/IP-HTTP-XML (such as SOAP)". Which I would scrap, and just put in SOAP, since NSI does not support any other TCP/IP-HTTP-XML stack.

Given that TCP, HTTP and SOAP already provide more than enough delivery assurance, I have to wonder why we actually need that extra conceptual layer in the Message Delivery Layer. It already is basically outsourced to those layers anyway. And for the few cases where upper layer logic is required to make sure that a request is handled correctly, we turn to the State Machine in the form of acknowledgements.

I know that the State Machine is currently still being improved. But did we also take the suggestion to make all short exchanges synchronous, and to allow optional acknowledgement for longer asynchronous exchanges? This would make it much simpler to use NSI in cases with limited connectivity (firewalls/clients/NAT etc.)

Jeroen.

Begin forwarded message:

...

From: Jeroen van der Ham <vdham@uva.nl> Subject: Re: [Nsi-wg] Query request Date: 2 November 2012 10:20:34 CET To: Jerry Sobieski <jerry@nordu.net> Cc: <nsi-wg@ogf.org>

Hi,

The NSI has chosen to use SOAP and WSDL as its message transport layer. This is a reasonably standard protocol, with support (more or less) in many different languages. For what we aim to do it is pretty good, and it has proven to work pretty well in the past few demos.

You're saying that we should not be projecting lower layer issues into the NSI layer. In my opinion this is exactly what has happened already. And it is also exactly the reason that we are having all these issues.

Reliable message transport layers are readily available off the shelf. TCP makes sure that a packet gets from the source to the destination. On top of that we're using HTTP, which also provides some additional guarantees that the message gets there, including some error codes regarding malformed messages, messages out of order, authentication errors, et cetera. We've used an additional SOAP layer on top, which provides you with even more guarantees.

Can someone please explain to me that in this scenario we really need to have separate confirmation messages? And even if we do, why do these messages have to be exchanged in a completely separate TCP-HTTP-SOAP session?

I understand that we have a state machine, with transition using confirmation messages. But can't we use any of the TCP/HTTP/SOAP status messages to achieve that?

Jeroen.

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

<MDL.pptx>

Sorry to butt in.. first time poster ... If you have decided to use SOAP with HTTP binding, WS-RM (ReliableMessaging) will ensure message delivery. Perhaps this layer (MDL) is referring to that?  Vikas ________________________________ From: Inder Monga <imonga@es.net> To: Jeroen van der Ham <vdham@uva.nl> Cc: NSI Working Group <nsi-wg@ogf.org> Sent: Thursday, December 6, 2012 7:26 AM Subject: Re: [Nsi-wg] Message Delivery Layer Hi All, I haven't had a chance to read the entire thread on this.  The MDL layer essentially handles the case when MTL fails to deliver a message. The scenario being, whichever reliable message transport layer or multiple of them below, tries to deliver the message end to end. In case the message does not reach after multiple retries, the error is handed to MDL layer (and yes it is logical). The MDL layer can have a different timeout, choose to deliver the message again or try a differnet mechanism. That is upto the implementor. This prevents the NSI state machine having to deal with message timeouts and retries at the application layer. It simplifies the state machine . If there is more confusion lets have a skype call. Inder On Thu, Dec 6, 2012 at 2:57 AM, Jeroen van der Ham <vdham@uva.nl> wrote: Hi,

I don't understand this still. What does this add to the current mechanism of reliable message transport and the state machine which handles the error conditions?

Jeroen.

On 6 Dec 2012, at 11:43, Guy Roberts <Guy.Roberts@dante.net> wrote:

...

Jeroen,

The MTL layer looks after delivery between a pair of peering RA and PA.  My understanding is that the MDL has different role, to confirm the correct delivery to all child PAs. This allows the logic associated with checking that all child PAs have responded correctly to be removed from the state machines.  See the attached slide.

Guy

-----Original Message----- From: nsi-wg-bounces@ogf.org [mailto:nsi-wg-bounces@ogf.org] On Behalf Of Jeroen van der Ham Sent: 06 December 2012 10:18 To: NSI Working Group Subject: [Nsi-wg] Message Delivery Layer

Hi all,

It seems that in the current document we have sections on the MDL and MTL. The MTL is deemed to be out of scope, and it is recommended to be "TCP/IP-HTTP-XML (such as SOAP)". Which I would scrap, and just put in SOAP, since NSI does not support any other TCP/IP-HTTP-XML stack.

Given that TCP, HTTP and SOAP already provide more than enough delivery assurance, I have to wonder why we actually need that extra conceptual layer in the Message Delivery Layer. It already is basically outsourced to those layers anyway. And for the few cases where upper layer logic is required to make sure that a request is handled correctly, we turn to the State Machine in the form of acknowledgements.

I know that the State Machine is currently still being improved. But did we also take the suggestion to make all short exchanges synchronous, and to allow optional acknowledgement for longer asynchronous exchanges? This would make it much simpler to use NSI in cases with limited connectivity (firewalls/clients/NAT etc.)

Jeroen.

Begin forwarded message:

...

From: Jeroen van der Ham <vdham@uva.nl> Subject: Re: [Nsi-wg] Query request Date: 2 November 2012 10:20:34 CET To: Jerry Sobieski <jerry@nordu.net> Cc: <nsi-wg@ogf.org>

Hi,

The NSI has chosen to use SOAP and WSDL as its message transport layer. This is a reasonably standard protocol, with support (more or less) in many different languages. For what we aim to do it is pretty good, and it has proven to work pretty well in the past few demos.

You're saying that we should not be projecting lower layer issues into the NSI layer. In my opinion this is exactly what has happened already. And it is also exactly the reason that we are having all these issues.

Reliable message transport layers are readily available off the shelf. TCP makes sure that a packet gets from the source to the destination. On top of that we're using HTTP, which also provides some additional guarantees that the message gets there, including some error codes regarding malformed messages, messages out of order, authentication errors, et cetera. We've used an additional SOAP layer on top, which provides you with even more guarantees.

Can someone please explain to me that in this scenario we really need to have separate confirmation messages? And even if we do, why do these messages have to be exchanged in a completely separate TCP-HTTP-SOAP session?

I understand that we have a state machine, with transition using confirmation messages. But can't we use any of the TCP/HTTP/SOAP status messages to achieve that?

Jeroen.

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

<MDL.pptx>

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Hi, On 6 Dec 2012, at 16:26, Inder Monga <imonga@es.net> wrote:

The MDL layer essentially handles the case when MTL fails to deliver a message. The scenario being, whichever reliable message transport layer or multiple of them below, tries to deliver the message end to end. In case the message does not reach after multiple retries, the error is handed to MDL layer (and yes it is logical). The MDL layer can have a different timeout, choose to deliver the message again or try a differnet mechanism. That is upto the implementor. This prevents the NSI state machine having to deal with message timeouts and retries at the application layer. It simplifies the state machine .

If there is more confusion lets have a skype call.

The use-case sounds reasonable. The solution to me does not, basically you're saying: We don't want to overcomplicate the state machine, so we add another layer. That layer will have another state machine. I don't see how that simplifies things in this case. Jeroen.

Reliable transport means that there are retries and acknowledgements. If the message cannot be delivered because the other NSA has issues or the intervening network is partitioned for some reason, so the NSA is unreachable, then MTL will pass that failure up to the NSA layer to manage. This failure event is handled within the MDL rather than by the NSI state machine, thats all what the MDL does. Yes, it is simple. MDL can choose to retry again, or have a timeout, before pushing the failure event upto the state machine. An implementors MDL layer can be non-existant and can just forward the MTL error as a delivery error to the state machine. Why call it a different layer? This was what Tomohiro proposed and the group agreed. I do not have special preferences here. Inder On Wed, Dec 12, 2012 at 2:06 AM, Jeroen van der Ham <vdham@uva.nl> wrote:

Hi,

On 7 Dec 2012, at 13:41, Jeroen van der Ham <vdham@uva.nl> wrote:

...

On 6 Dec 2012, at 16:26, Inder Monga <imonga@es.net> wrote:

...

The MDL layer essentially handles the case when MTL fails to deliver a message. The scenario being, whichever reliable message transport layer or multiple of them below, tries to deliver the message end to end. In case the message does not reach after multiple retries, the error is handed to MDL layer (and yes it is logical). The MDL layer can have a different timeout, choose to deliver the message again or try a differnet mechanism. That is upto the implementor. This prevents the NSI state machine having to deal with message timeouts and retries at the application layer. It simplifies the state machine .

If there is more confusion lets have a skype call.

I want to follow up on this further. You state that if the MTL layer fails to deliver a message somehow. We have stated that the MTL should be a reliable transport mechanism, so a delivery failure means something more complicated is going on. You're stating that you want to leave it to the implementor to figure out how to deal with this.

That's all fine with me, but why do we have to call this another layer, instead of just stating the above?

Jeroen.

Hi, On 12 Dec 2012, at 11:34, Inder Monga <imonga@es.net> wrote:

Jeroen,

What solution are you proposing - it is not clear?

I'm proposing to simplify things, to not introduce another layer, and simply describe that the NSI expects the NSA to make an effort in delivering a message. It should use the reliable MTL (which we've defined to be SOAP). If that fails, then it is up to the NSAs discretion to fail directly, or to make an extra effort and try again. Ultimately, it will either fail or it will succeed. To me, it seems like a simple statement that can be added to the document. For example where you describe when you go from a state to the failed state. Jeroen.

Hi, On 12 Dec 2012, at 12:36, Inder Monga <imonga@es.net> wrote:

Thanks Jeroen. The introduction of the separate layer as a concept is to help formalize the separation of concerns by handling errors at the MTL layer. If you look at NSI v1, the state machine was integrated and would mean that the state machine would change if you changed the requirements for the MTL layer. This is not the case with NSI v2.0. When people are talking about different MTL's, then we need to make sure that a change in MTL will not change the NSI state machine. The MDL makes that possible.

From your messages, I believe you agree this functionality is needed in NSA. What you are disagreeing is the formalization of that functionality into a layer. Maybe the NSI group can comment on that as well or you can make a formal proposal to the group to change it?

Exactly. Having thought about this, what I do not like about the layering model of the MDL and MTL is the fact that NSI is drawing them into the whole architecture and representing them as part of NSI. In my mind they are requirements to an outside protocol/transportation mechanism that is already there and should not be represented as part of NSI-CS. Jeroen.

Thanks Tomohiro and Henrik for the details - I had not mentioned the aggregation/coordination of the messages for enabling the 'tree' function of the NSI. -Inder (Mistakes attributed to thumb-typing on a mobile device) On Dec 12, 2012, at 3:44 PM, Tomohiro Kudoh <t.kudoh@aist.go.jp> wrote:

Hi all,

I think MDL is not an appropriate name, and the name should be changed.

In my idea, (former) MDL should support the following things:

1. Aggregate/segregate messages. For a request, message should be sent to children designated by a path finding. For replies and notifications, they should be aggregated appropriately (depending success or fail). Those functions have not been well documented yet.

2. Support uRA initiated message delivery retry. There is a request (especially from John) to support retry by uRA, after message delivery (by MTL and other) is failed. This is to do retry after service plane connection recovery which may occur after a certain period of time (minutes?). This is currently supported by the state machine, but I am proposing to move this to the (former) MDL.

3. To signal state machines for transition. For example, Activate state machine transition is not well described by the state machines only. Such behaviors should be specified as a specification of the (former) MDL.

Note that MTL is under the web services layer, and (former) MDL is on top of the web services layer.

Tomohiro

On Wed, 12 Dec 2012 15:23:48 +0100 Jeroen van der Ham <vdham@uva.nl> wrote:

...

Hi,

On 12 Dec 2012, at 12:36, Inder Monga <imonga@es.net> wrote:

...

Thanks Jeroen. The introduction of the separate layer as a concept is to help formalize the separation of concerns by handling errors at the MTL layer. If you look at NSI v1, the state machine was integrated and would mean that the state machine would change if you changed the requirements for the MTL layer. This is not the case with NSI v2.0. When people are talking about different MTL's, then we need to make sure that a change in MTL will not change the NSI state machine. The MDL makes that possible.

From your messages, I believe you agree this functionality is needed in NSA. What you are disagreeing is the formalization of that functionality into a layer. Maybe the NSI group can comment on that as well or you can make a formal proposal to the group to change it?

Exactly. Having thought about this, what I do not like about the layering model of the MDL and MTL is the fact that NSI is drawing them into the whole architecture and representing them as part of NSI. In my mind they are requirements to an outside protocol/transportation mechanism that is already there and should not be represented as part of NSI-CS.

Jeroen.

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Hi, On 12 Dec 2012, at 16:19, Tomohiro Kudoh <t.kudoh@aist.go.jp> wrote:

Hi all,

This is what I am considering.

Right, this I agree with, the Message Handler is a component of the NSA and deals with the logic of retries, aggregation, etc. Jeroen.

I think we need the MDL/MTL layer - it does things the traditional transport protocol layers do not. Fundamentally, it notifies the NSI layer that a message was unable to be sent within a specified time. That Timer needs to be specified by NSI...i'm not sure you can even specify such a timeout for the other transport protocols. And if that send timer expires, the sending NSI layer needs to recover somehow. The timeout may be different for different messages, and the recovery action will very likely be different depending on the message that failed to send. So, if a response timer expires that says the remote NSA did not respond in a reasonable amount of time, the local NSA's first question should be: Did the remote NSA even get the message? Are we sure? and when? If the message was not sent and acknowledged, then the remote NSA does not know about it so the local NSA can recover unilaterally. If the message *was* sent and acknowledged, then the remote NSA is processing it somehow and the local NSA needs to take that into consideration in recovering. How does web services deal with this? On a connection by connection basis, messages are processed serially. Thus messages must be queued if a prior message has not been processed. Thus, a crash could occur after receipt but before processing losing the queued message. Thus the sending NSA thinks a messages was being processed and the remote NSA has lost it over the crash. A custom MDL could make sure such messages are written to a persistent store before achnowledging the receipt of the message. At the very least - I think we need to define clearly how we expect/want the message delivery process to function in order that the NSI protocols can be deterministic about the distributed state of a connection. If there are existing layers that can do this, perhaps we can use just existing layers. I know there are packages out there that have explicit MDLs - some I think are even defined as web services themselves. So the issues is not unique to this NSI effort - many distributed WS based systems have had similar problems to address. Perhaps these is exsiting library out there to do this..? But we need to define what we think we need first, then decide if the MDL is something new or something that already exists. My thoughts... Jerry On 12/12/12 5:55 AM, Jeroen van der Ham wrote:

Hi,

On 12 Dec 2012, at 11:34, Inder Monga <imonga@es.net> wrote:

...

Jeroen,

What solution are you proposing - it is not clear? I'm proposing to simplify things, to not introduce another layer, and simply describe that the NSI expects the NSA to make an effort in delivering a message. It should use the reliable MTL (which we've defined to be SOAP). If that fails, then it is up to the NSAs discretion to fail directly, or to make an extra effort and try again. Ultimately, it will either fail or it will succeed.

To me, it seems like a simple statement that can be added to the document. For example where you describe when you go from a state to the failed state.

Jeroen. _______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Hi, On 6 Dec 2012, at 11:58, Henrik Thostrup Jensen <htj@nordu.net> wrote:

...

I know that the State Machine is currently still being improved. But did we also take the suggestion to make all short exchanges synchronous, and to allow optional acknowledgement for longer asynchronous exchanges? This would make it much simpler to use NSI in cases with limited connectivity (firewalls/clients/NAT etc.)

Sync/Async doesn't really chage the state machine as I see it. I could be wrong though.

I'd like more synchronous messages, but given the way things are wired together, it is difficult to do. I've argued for a generic state change request (replacing provision, release and terminate), along with a publish-subscribe mechanism earlier. This would also allow third party requests for connection updates, where they are currently forced to do a query.

A feasible first step in my opinion would be to make acknowledgements optional. Combined with a good query interface this already makes it possible to operate a client NSA behind a firewall. Jeroen.

Hi, The problem that I am trying to solve is the situation where the client is possibly behind a firewall/NAT/whatever, where the client is the only one capable of setting up a bidirectional TCP session. Right now the NSI protocol breaks in that situation, because it insists on sending the acknowedgements through a separate channel that is independently setup by the server back to the client. In my opinion the simplest way to solve that is indeed to make the callback optional and allow clients to poll for updates. The reserveConfirmed may or may not be sent. But getting it failed to deliver would not trigger a fault. Jeroen. On 7 Dec 2012, at 10:26, Henrik Thostrup Jensen <htj@nordu.net> wrote:

On Thu, 6 Dec 2012, Jeroen van der Ham wrote:

...

A feasible first step in my opinion would be to make acknowledgements optional. Combined with a good query interface this already makes it possible to operate a client NSA behind a firewall.

Are you suggesting to make callback optional and resolve to polling for updates? I.e., never getting a reserveConfirmed message from a reserve request.

(btw. I think polling is completely acceptable - most people who have build a distributed system with events are painfully aware that they sometime disappear and that one will have to resolve to polling as a fallback).

OR:

Thowing out the callback scheme, i.e., getting a reserveConfirmed as the direct reply to a reseve request. This will mean some potential long-standing requests (not that it is a problem), probabaly some minutes.

This can also be optional (replyTo -> yes to callback, no replyTo -> direct reply). I'd prefer not to have this dual behavior due to implementation complexity.

--

Originally we chose to have the callbacks as some of the commands could take a very long time to complete. I think this was especially for provision, which would not trigger until the link came up (could be weeks), however with the notification mechanism in NSI2, that is no longer the case (provisionConfirmed now indicates that all NSAs have acknowledged the provision request).

If we are willing to handle request delays of a couple of minutes (most will be faster), we could forgo the callbacks for requests, and only deal with callbacks for notifications like active and forcedEnd.

Best regards, Henrik

Henrik Thostrup Jensen <htj at nordu.net> Software Developer, NORDUnet

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

My suggestion is to formalize the Message Transport Layer: The MTL is responsible for taking a message from the NSI protocol layer, placing it in a transmit queue, and ensuring that: a) the msg gets sent, or b) there is an indication that the message cannot be sent. The NSI protocol layer invokes this MTL_Send() function by presenting 1) the NSI protocol message, 2) the *NSA ID* destination (not a web service endpoint), 3) a callback for normal completion, 4) a timeout value, and 5) a callback for timeout error. The MTL places the message on a queue for the indicated target NSA, and transmits the messages in a FIFO order. Thus there is a different queue for each destination NSA. Whenever a message is queued for transmission to a destination NSA, the MTL will check to see if there is an open TCP/SSL session to that destination NSA. If there is, then the message is transmitted on that session using the exchange below. If there is no active session, the MTL will try to find the appropriate addressing information for that target NSA and open a session to it. If successful, the local MTL will proceed with the transmission. If the session is not established(e.g. no target address info is available, or the dest is unreachable, or behind a FW, etc.) the local sending MTL leaves the message on the send queue and waits a period before retrying the session establishment. The session can also be established by the remote MTL. When a TCP/SSL session is received from a remote agent, the MTLs exchange NSA identifiers and the session is bound to the appropriate NSA send queue. (Sessions are bi-directional.) Each NSI protocol message is exchanged by MTLs using the following sequence: Source NSA Dest NSA MsgXmit(msg, msgid)-> Sends the entire message along with a message id/seq number <- MsgRecv(msgid) Responds that the message id has been received, stored, and queued MsgRecvAck(msgid) -> Acknowledges the completion of the transaction. This exchange conclusively indicates to both NSAs that the messages has been moved from the local sending system to the destination system where it has been saved in a persistent store and queued for processing by the destination NSA. Upon successful transmission, the message is removed from the send queue, the timer is canceled, and the successful send callback is invoked. The MTL will continue to try to send the message at the front of the send queue until the timeout expires. When the timeout expires, the message is removed from the queue, and the timeout callback is invoked. Other messages destined for that NSA may exist in the queue as well and are blocked. Their timeouts may also expire and are treated similarly. So any message who's timer expires will be removed from the queue and the timeout callback invoked. For sending a message, this process allows the protocol layer to simply earmark a message for a destination NSA and only be informed of success or failure. The NSI protocol layer does not care why a message was unable to be sent, just that it did not make it after a pre-specified time of trying to do so. The MTL can log attempts or perform other actions so that more detailed forensic information is available or actions can be taken, but these other actions are not NSI protocol layer functions. Further, this MTL mechanism can take advantage of proven protocols such as TCP to assure delivery of the MTL message exchanges - substantially simplifying the MTL and minimizing redundant functionality. However, TCP or SSL or HTTP/S will not do everything - the MTLs are responsible for managing the NSA send queues (note the NSI layer may be mutlithreaded), session establishment/retries, store management, timeout processing, etc. Similar processing is done by the MTL upon receiving a message. The received message is timestamped and entered into a persistent store to enable recovery should the receiving host or process be interrupted. After the message is successfully stored, the message is placed on the input event queue of the local NSA protocol layer. Then the MsgRecv'd message is sent to the source MTL. Note: There are two timers of interest here: the transmission timeout value mentioned above, and the NSI protocol response timeout value that dictates how long the local NSA is willing to wait for a protocol response from the remote NSA. The NSI protocol layer will place a message on the send queue with a transmission timeout. But the NSI layer is actually only concerned with whether the protocol primitive was acted upon by the remote NSA withn a specific timeframe - enter the /response/ timer. If there is no response from the remote NSA, then the local response timer expires and the local NSA has to recover. Then and only then does the *protocol* layer need to recover. And knowing if the remote NSA ever got the message is an important piece of info in determining how to recover from the protocol response timeout. Thus, if the transmission timeout exceeds the response timeout, the NSI layer may timeout before the MTL has given up trying to send the message. Else, if the transmission timeout is small compared to the response timer, the send timeout may occur too fast - not allowing the remote system enough time to establish a session before the sent message is timed out. So, it is suggested that the response timer should be set in sequence with the transmission timer - not overlapping. Thus the response timeout would not begin until the transmission has succeeded. However, issues such as slow session establishment can still impact upstream response timers in the service tree. This remains an issue of concern. Thoughts? Jerry On 12/7/12 5:04 AM, Jeroen van der Ham wrote:

Hi,

The problem that I am trying to solve is the situation where the client is possibly behind a firewall/NAT/whatever, where the client is the only one capable of setting up a bidirectional TCP session.

Right now the NSI protocol breaks in that situation, because it insists on sending the acknowedgements through a separate channel that is independently setup by the server back to the client.

In my opinion the simplest way to solve that is indeed to make the callback optional and allow clients to poll for updates. The reserveConfirmed may or may not be sent. But getting it failed to deliver would not trigger a fault.

Jeroen.

On 7 Dec 2012, at 10:26, Henrik Thostrup Jensen <htj@nordu.net> wrote:

...

On Thu, 6 Dec 2012, Jeroen van der Ham wrote:

...

A feasible first step in my opinion would be to make acknowledgements optional. Combined with a good query interface this already makes it possible to operate a client NSA behind a firewall. Are you suggesting to make callback optional and resolve to polling for updates? I.e., never getting a reserveConfirmed message from a reserve request.

(btw. I think polling is completely acceptable - most people who have build a distributed system with events are painfully aware that they sometime disappear and that one will have to resolve to polling as a fallback).

OR:

Thowing out the callback scheme, i.e., getting a reserveConfirmed as the direct reply to a reseve request. This will mean some potential long-standing requests (not that it is a problem), probabaly some minutes.

This can also be optional (replyTo -> yes to callback, no replyTo -> direct reply). I'd prefer not to have this dual behavior due to implementation complexity.

--

Originally we chose to have the callbacks as some of the commands could take a very long time to complete. I think this was especially for provision, which would not trigger until the link came up (could be weeks), however with the notification mechanism in NSI2, that is no longer the case (provisionConfirmed now indicates that all NSAs have acknowledged the provision request).

If we are willing to handle request delays of a couple of minutes (most will be faster), we could forgo the callbacks for requests, and only deal with callbacks for notifications like active and forcedEnd.

Best regards, Henrik

Henrik Thostrup Jensen <htj at nordu.net> Software Developer, NORDUnet

_______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

---

nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg

Hi Shouldn't the protocol takes care of protocol stuff only? Messaging layer is something out of scope IMHO. There is already multiple more or less standardized approaches on how to send a message in order to assure it's delivered or failed, and also working frameworks are available for this depending on implementations (JMS, or some concepts related to SOA). IMHO we should list the requirements and recommendations for this layer (e.g. security, ciphering, maximum delivery time), which are crucial for protocol operations, but we should not define yet another message transport layer. This in fact is flexibility, so one can adopt NSI to anything which gives the features we need (which are rather common, like please deliver it, and please keep the message order). We are interested in two things: - Send a message to a remote Agent (use timeout for delivery) - Notify if transport failed after timeout - Get a message from a remote Agent Would that be WebServices - actually I don't care, unless all the correct fields are there and it is in align with state machine. Is it written in Java or C++ - actually I don't care, feel free to use brainfuck, OISC, Thue, Whitespace or if those are too easy, use C# ;) The demo cloud, or operational cloud require an agreement to use the message layer tools which can collaborate, i.e. agree to use one (so we use WebServices). But we should not prevent people to use it different way around. Yes, NSI will not be compatible with other NSI agent using different message layer. But protocol definition should not solve all the issues with communication at once. This is implementation independent, not protocol dependent. That's my 5 cents. Disagreements are welcome (and rather expected :) ) Best regards Radek ______________________________________________ Radosław Krzywania Network Department of Poznan Supercomputing and Networking Center http://www.man.poznan.pl ______________________________________________ From: nsi-wg-bounces@ogf.org [mailto:nsi-wg-bounces@ogf.org] On Behalf Of Jerry Sobieski Sent: Friday, December 07, 2012 1:22 PM To: Jeroen van der Ham Cc: NSI Working Group Subject: Re: [Nsi-wg] Message Delivery Layer My suggestion is to formalize the Message Transport Layer: The MTL is responsible for taking a message from the NSI protocol layer, placing it in a transmit queue, and ensuring that: a) the msg gets sent, or b) there is an indication that the message cannot be sent. The NSI protocol layer invokes this MTL_Send() function by presenting 1) the NSI protocol message, 2) the *NSA ID* destination (not a web service endpoint), 3) a callback for normal completion, 4) a timeout value, and 5) a callback for timeout error. The MTL places the message on a queue for the indicated target NSA, and transmits the messages in a FIFO order. Thus there is a different queue for each destination NSA. Whenever a message is queued for transmission to a destination NSA, the MTL will check to see if there is an open TCP/SSL session to that destination NSA. If there is, then the message is transmitted on that session using the exchange below. If there is no active session, the MTL will try to find the appropriate addressing information for that target NSA and open a session to it. If successful, the local MTL will proceed with the transmission. If the session is not established(e.g. no target address info is available, or the dest is unreachable, or behind a FW, etc.) the local sending MTL leaves the message on the send queue and waits a period before retrying the session establishment. The session can also be established by the remote MTL. When a TCP/SSL session is received from a remote agent, the MTLs exchange NSA identifiers and the session is bound to the appropriate NSA send queue. (Sessions are bi-directional.) Each NSI protocol message is exchanged by MTLs using the following sequence: Source NSA Dest NSA MsgXmit(msg, msgid)-> Sends the entire message along with a message id/seq number <- MsgRecv(msgid) Responds that the message id has been received, stored, and queued MsgRecvAck(msgid) -> Acknowledges the completion of the transaction. This exchange conclusively indicates to both NSAs that the messages has been moved from the local sending system to the destination system where it has been saved in a persistent store and queued for processing by the destination NSA. Upon successful transmission, the message is removed from the send queue, the timer is canceled, and the successful send callback is invoked. The MTL will continue to try to send the message at the front of the send queue until the timeout expires. When the timeout expires, the message is removed from the queue, and the timeout callback is invoked. Other messages destined for that NSA may exist in the queue as well and are blocked. Their timeouts may also expire and are treated similarly. So any message who's timer expires will be removed from the queue and the timeout callback invoked. For sending a message, this process allows the protocol layer to simply earmark a message for a destination NSA and only be informed of success or failure. The NSI protocol layer does not care why a message was unable to be sent, just that it did not make it after a pre-specified time of trying to do so. The MTL can log attempts or perform other actions so that more detailed forensic information is available or actions can be taken, but these other actions are not NSI protocol layer functions. Further, this MTL mechanism can take advantage of proven protocols such as TCP to assure delivery of the MTL message exchanges - substantially simplifying the MTL and minimizing redundant functionality. However, TCP or SSL or HTTP/S will not do everything - the MTLs are responsible for managing the NSA send queues (note the NSI layer may be mutlithreaded), session establishment/retries, store management, timeout processing, etc. Similar processing is done by the MTL upon receiving a message. The received message is timestamped and entered into a persistent store to enable recovery should the receiving host or process be interrupted. After the message is successfully stored, the message is placed on the input event queue of the local NSA protocol layer. Then the MsgRecv'd message is sent to the source MTL. Note: There are two timers of interest here: the transmission timeout value mentioned above, and the NSI protocol response timeout value that dictates how long the local NSA is willing to wait for a protocol response from the remote NSA. The NSI protocol layer will place a message on the send queue with a transmission timeout. But the NSI layer is actually only concerned with whether the protocol primitive was acted upon by the remote NSA withn a specific timeframe - enter the response timer. If there is no response from the remote NSA, then the local response timer expires and the local NSA has to recover. Then and only then does the *protocol* layer need to recover. And knowing if the remote NSA ever got the message is an important piece of info in determining how to recover from the protocol response timeout. Thus, if the transmission timeout exceeds the response timeout, the NSI layer may timeout before the MTL has given up trying to send the message. Else, if the transmission timeout is small compared to the response timer, the send timeout may occur too fast - not allowing the remote system enough time to establish a session before the sent message is timed out. So, it is suggested that the response timer should be set in sequence with the transmission timer - not overlapping. Thus the response timeout would not begin until the transmission has succeeded. However, issues such as slow session establishment can still impact upstream response timers in the service tree. This remains an issue of concern. Thoughts? Jerry On 12/7/12 5:04 AM, Jeroen van der Ham wrote: Hi, The problem that I am trying to solve is the situation where the client is possibly behind a firewall/NAT/whatever, where the client is the only one capable of setting up a bidirectional TCP session. Right now the NSI protocol breaks in that situation, because it insists on sending the acknowedgements through a separate channel that is independently setup by the server back to the client. In my opinion the simplest way to solve that is indeed to make the callback optional and allow clients to poll for updates. The reserveConfirmed may or may not be sent. But getting it failed to deliver would not trigger a fault. Jeroen. On 7 Dec 2012, at 10:26, Henrik Thostrup Jensen <mailto:htj@nordu.net> <htj@nordu.net> wrote: On Thu, 6 Dec 2012, Jeroen van der Ham wrote: A feasible first step in my opinion would be to make acknowledgements optional. Combined with a good query interface this already makes it possible to operate a client NSA behind a firewall. Are you suggesting to make callback optional and resolve to polling for updates? I.e., never getting a reserveConfirmed message from a reserve request. (btw. I think polling is completely acceptable - most people who have build a distributed system with events are painfully aware that they sometime disappear and that one will have to resolve to polling as a fallback). OR: Thowing out the callback scheme, i.e., getting a reserveConfirmed as the direct reply to a reseve request. This will mean some potential long-standing requests (not that it is a problem), probabaly some minutes. This can also be optional (replyTo -> yes to callback, no replyTo -> direct reply). I'd prefer not to have this dual behavior due to implementation complexity. -- Originally we chose to have the callbacks as some of the commands could take a very long time to complete. I think this was especially for provision, which would not trigger until the link came up (could be weeks), however with the notification mechanism in NSI2, that is no longer the case (provisionConfirmed now indicates that all NSAs have acknowledged the provision request). If we are willing to handle request delays of a couple of minutes (most will be faster), we could forgo the callbacks for requests, and only deal with callbacks for notifications like active and forcedEnd. Best regards, Henrik Henrik Thostrup Jensen <htj at nordu.net> Software Developer, NORDUnet _______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg _______________________________________________ nsi-wg mailing list nsi-wg@ogf.org https://www.ogf.org/mailman/listinfo/nsi-wg